import reg files to make w2008r2 behave
10 files changed
tree: 1becc8981403a8862e2fc5df06b585065344c972
  1. bin/
  2. bits/
  3. README.md
README.md

#Glazier - automating building CouchDB on Windows


Glazier is a set of scripts designed to help automate as much as practicable the build of CouchDB on Windows, from XP/2003 to Windows 7 or Server 2008.

Current State

  • steps below should produce a working CouchDB build & self-installing .exe
  • the build environment should be fully functional on both 32,64, desktop and server versions of windows from XP/2003 onwards
  • fetching binaries is described and automated
  • installation of development environment is described and automated
  • downloads are not small - get_bits.cmd retrieves approx 7GiB of DVD ISOs for Microsoft's Visual Studio 2008 compiler, related SDKs, the smaller cygwin and mozilla build frameworks, source and misc tools

Installing the Build Environment


  • Building Erlang & CouchDB on Windows requires a custom build environment, which is very sensitive to path order amongst the three different compilers used to build wxWidgets, erlang, javascript, and couchdb
  • Each component is built in differing environments and consolidated via Makefiles
  • This is further complicated by different install locations on 32 vs 64 bit windows versions, and which Microsoft C compiler and Windows SDKs are installed.

Cygwin

The full Cygwin install comprises several GiB of data. Run cygwin's setup.exe using defaults with the following additional modules at a minimum:

  • devel: ALL
  • editors: vim
  • utils: file

After install, run the cygwin shell, and set up a symlink to where you plan to install related binaries, build erlang, and couchdb. I am using C:\relax so:

	mkdir 
	ln -s /cygdrive/c/relax /relax

Mozilla Build

The mozilla build toolchain is needed solely for building our javascript engine.

  • download it from mozbuild and install per defaults

Microsoft Visual C++

  • Erlang and CouchDB can be built using the free VS2008 Express C++ edition from MSDN
  • install Visual C++ 9 only, to the default locations, using the DVD ISO msvc++ excluding optional MSSSQL & Silverlight

Windows 7 SDK

  • The windows 7 SDK is required, as the free VS2008 install is missing the message compiler. Download one of the following version per your requirements & install
  • win7sdk_32bit
  • win7sdk_64bit

Supporting Tools


Both CouchDB and Erlang have dependencies on other opensource tools.

OpenSSL

  • use the 32-bit version even if you are using a 64-bit OS
  • download openssl_bits and install to c:\openssl

Innosoft Installer

  • download the installer inno_bits and install to c:\relax\inno5

NSIS Installer

  • download the installer nsis_bits and install to c:\relax\nsis

set up links

  • to keep our paths clean later, and largely independent of the compiler installs if you have pre-existing ones, start a new cmd.exe prompt with a fresh environment

  • this should have both VS90ComnTools and ProgramFiles environment vars defined from the previous install of Visual Studio

  • setup the following hard links (junction points), using either the included mklink tool (Windows 7 and later), or junction from sysinternals:

      junction c:\relax\openssl c:\openssl
      junction c:\relax\vs90 "%VS90COMNTOOLS%\..\.."
      junction c:\relax\SDKs "%programfiles%\Microsoft SDKs\Windows"
    

or using mklink.exe

    mklink /j c:\relax\openssl c:\openssl
[etc...]

Building pre-requisites for Erlang


wxWidgets

  • two components are used for building Erlang's graphical shell, werl.exe on windows

  • download wxwidgets_bits from WxWidgets website & unzip using cygwin into /relax/

  • the Erlang build expects to see wxWidgets in /opt/local/pgm/wxWidgets-2.8.11 by default

      mkdir -p /opt/local/pgm/
      ln -s /relax/wxMSW-2.8.11 /opt/local/pgm/wxWidgets-2.8.11
    
  • Using a suitable editor (vi in the cygwin suite, or install notepadplus_bits for windows users) and

  • Edit c:\relax\wxMSW-2.8.11\include\wx\msw\setup.h to enable wxUSE_GLCANVAS, wxUSE_POSTSCRIPT and wxUSE_GRAPHICS_CONTEXT

wx.dsw

  • open VSC++ & the project C:\relax\wxMSW-2.8.11\build\msw\wx.dsw, accepting the automatic conversion to the newer VC++ format and save as \relax\wxMSW-2.8.11\build\msw\wx.sln

  • right-click on the project, and set up the dependencies for wx.dsw to achieve the below build order jpeg, png, tiff, zlib, regex, expat, base, net, odbc, core, gl, html, media, qa, adv, dbgrid, xrc, aui, richtext, xml

  • Launch a new prompt from somewhere like Start -> Programs -> Microsoft Visual C++ -> Visual Studio Tools -> VS2008 Cmd Prompt

  • Then build all unicode release (and unicode debug) packages:

      pushd c:\relax\wxMSW*\build\msw
      vcbuild /useenv  /platform:Win32 /M4 wx.sln "Unicode Release|Win32"
      vcbuild /useenv  /platform:Win32 /M4 wx.sln "Unicode Debug|Win32"
    

stc.dsw

  • open VSC++ & convert C:\relax\wxMSW-2.8.11\contrib\build\stc\stc.dsw to C:\relax\wxMSW-2.8.11\contrib\build\stc\stc.sln

      pushd c:\relax\wxMSW*\contrib\build\stc
      set LIB=%LIB%;..\..\..\include..\..\..\lib\vc_lib\mswd
      set LIBPATH=%LIBPATH%;..\..\..\lib\vc_lib
      vcbuild /useenv /platform:Win32 /M4 stc.sln "Unicode Release|Win32"
      vcbuild /useenv /platform:Win32 /M4 stc.sln "Unicode Debug|Win32"
    

Reference URLs


Building Erlang


  • after installing VC++ 2008 Express, and most other Visual Studio solutions, call "%vs90comntools%\..\..\vc\vcvarsall.bat" x86 will automatically find the correct path, and set up our 32-bit build environment correctly, independently if you have installed on 32 or 64bit windows.

  • start a cygwin shell

      cd /relax
      ln -s /cygdrive/d/glazier/bin bin
    

    ln -s /cygdrive/d/glazier/bits bits tar xzf /relax/bits/otp_src_R14A.tar.gz & tar xzf /relax/bits/otp_src_R13B04.tar.gz &

  • then run d:\glazier\bin\relax.cmd

Tk/Tcl

  • optional component

      cd $ERL_TOP && tar xvzf /relax/bits/tcltk85_win32_bin.tar.gz
      # or simply
      cd /relax/otp_src_R14A && tar xvzf /relax/bits/tcltk85_win32_bin.tar.gz
      cd /relax/otp_src_R13B04 && tar xvzf /relax/bits/tcltk85_win32_bin.tar.gz
    
  • or skip the whole damn lot this way

      echo "skipping gs" > lib/gs/SKIP
    
  • check that which cl; which link; which mc return the MS ones, if not then sort them manually

  • build Erlang using /src/glazier/bin/erl_config.sh and /src/glazier/bin/erl_build.sh, or manually as follows

      ./otp_build autoconf
      ./otp_build configure
      ./otp_build boot -a
      ./otp_build release -a
      ./otp_build installer_win32
      # to setup erlang to run from this new source build immediately run:
      ## and not ./release/win32/Install.exe -s
      ./otp_build local_setup
    

erlang INSTALL-Win32.md on github

or using the relax tools:

    start %glazier%\bin\relax.cmd
[select erlang R14A build or R13B04]
/relax/bin/erl_config.sh; /relax/bin/erl_build.sh
[note concurrent builds don't work - try putting a sleep 1200 in front]

CouchDB


minimum requirements

Javascript

The Javascript engine used by CouchDB is built from Seamonkey, using the mozilla build toolkit.

  • get seamonkey_bits

  • run c:\mozilla-build\start-msvc9.bat

      cd /c/relax && mkdir seamonkey-2.0.6
      cd seamonkey-2.0.6
      tar xjf /d/glazier/bits/seamonkey-2.0.6.source.tar.bz2
      cd /c/relax/seamonkey-2.0.6/comm-1.9.1/mozilla/js/src
      autoconf-2.13
      ./configure
      make
    
  • to install from -current do: cd /c/relax && mkdir seamonkey-current cd seamonkey-current hg clone http://hg.mozilla.org/mozilla-central cd mozilla-central/......./comm-1.9.1/mozilla/js/src autoconf-2.13 ./configure make

Inno Installer

from http://www.jrsoftware.org/download.php/ispack-unicode.exe download and install ispack-5.3.10-unicode.exe, including all additional components to c:\relax\inno5 & ensure its in the path

LibCURL

    pushd c:\relax\curl-7*
    vcbuild /upgrade lib\libcurl.vcproj
    vcbuild /useenv /rebuild /platform:Win32 lib\libcurl.vcproj "Release|Win32"
    xcopy lib\Release\libcurl.lib lib\ /y /f
popd

TODO maybe none of this section is needed now

    you need to have libcurl.lib in the ./configure path (CURL_LIBS="$withval/lib/libcurl")
    set OPENSSL_PATH=%systemdrive%\openssl
    set INCLUDE=%INCLUDE%;%OPENSSL_PATH%\include\openssl;
    set LIBPATH=%LIBPATH%;%OPENSSL_PATH%\lib;
    set LIB=%LIB%;%OPENSSL_PATH%\lib;
    nmake vc-ssl
    couldn't get this to work so instead i built curl/vc6curl.sln
    check curl.exe & see what libs it needs - these should be openssl only
    check that /src/curl-7.21.1/lib/libcurl.lib exists

OpenSSL

  • already installed into C:/OpenSSL/ no further steps required

ICU

Make & Build

    ./configure \
    --with-js-include=/cygdrive/c/path_to_seamonkey \
    --with-js-lib=/cygdrive/c/path_to_seamonkey_lib \
    --with-win32-icu-binaries=/cygdrive/c/path_to_icu_binaries_root \
    --with-erlang=$ERL_TOP/release/win32/usr/include \
    --with-win32-curl=/cygdrive/c/path/to/curl/root/directory \
    --with-openssl-bin-dir=/cygdrive/c/openssl/bin \
    --with-msvc-redist-dir=/cygdrive/c/dir/with/vcredist_platform_executable \
    --prefix=$ERL_TOP/release/win32

using seamonkey 2.0.6

    ./configure \
    --prefix=$ERL_TOP/release/win32 \
    --with-erlang=$ERL_TOP/release/win32/usr/include \
    --with-win32-icu-binaries=/relax/icu \
    --with-win32-curl=/relax/curl-7.21.1 \
    --with-openssl-bin-dir=/relax/openssl/bin \
    --with-msvc-redist-dir=/relax \
    --with-js-lib=/relax/seamonkey-2.0.6/comm-1.9.1/mozilla/js/src/dist/lib \
    --with-js-include=/relax/seamonkey-2.0.6/comm-1.9.1/mozilla/js/src/dist/include/js

CURL curl -> win32-openssl curl_bits

Automated Test Bed for Builds

The objective is to take the current manual steps, and have them automated for a successful build off a variety of representative Microsoft current OS using Amazon EC2 infrastructure.

Tested AMIs

These are all sourced from AWS Windows Servers provided by Amazon. The release build is taken from the Windows 2008 64-bit AMI below. You will need to set up an AWS account before using the AWS console and we recommend using spot instances as these are significantly cheaper to run.

TODO // URLs don't go to right AMIs

  • ami-c3e40daa | amazon/Windows-Server2008r1sp2-i386-Base-v103 ** bash.exe dumps core

  • ami-d9e40db0 | amazon/Windows-Server2008r1sp2-x86_64-Base-v103 ** bash.exe and cmd.exe dumps core

  • ami-f11ff098 | amazon/Windows-Server2003r2-i386-Base-v109 ** too bloody slow

  • ami-f51ff09c | amazon/Windows-Server2003r2-x86_64-Base-v109

  • NB both of the Win2008 ones above dump core while installing/running cygwin so we do not build by default

  • reference CouchDB install is therefore Win2003r2 x86_64 ami-f51ff09c l1.large & snap-080bb263 for accompanying binaries

  • reference CouchDB install is therefore Win2003r2 x86_i386 ami-f11ff098 c1.medium & snap-080bb263 for accompanying binaries

Config of bundled W2008R1SP2 64b AMI

  • attach a 30GiB EBS vol for storage of the bits

  • logon, bring up an admin command prompt by right-clicking on any Command Prompt link and choosing “Run as Administrator”

  • execute the following within that

  • diskmgmt.msc -> find the newly attached volume & turn it online

  • a fix for SEHOP security feature causing cygwin & similar unix shell emulations to dump core

      regedit d:\glazier\bundles\disable_sehop_kb956607.reg
      net user couchdb 1dot0 /add
      net localgroup administrators couchdb /add
    
  • disable UAC -> control panel -> user accts -> turn UAC off -> restart later

  • start menu -> server manager -> 2nd pane -> configure IE ESC

  • disable DEP -> bcdedit.exe /set {current} nx AlwaysOff

  • restart now

  • lgoon as couchdb with passwd couchdb1dot0

  • import console_hkcu.reg

  • import


#Manual Build Procedure


This procedure has been tested on the following platforms successfully:

  • Windows XP SP2 32 bit
  • Windows 7 64 bit
  • Amazon Windows 2003 32-bit medium instance

Core Tools

A number of opensource tools are used to unpack or retrieve files.

  • [7zip]

Set up Compilers

Three compiler tools are required to build wxWidgets, Erlang, Javascript, and finally CouchDB.

  • download and install Microsoft Visual Studio 2008 Express, and install C++ only using the DVD ISO msvc++ excluding optional MSSSQL & Silverlight

Licenses



sample environments on different windows platforms


win7 std - default environment

ALLUSERSPROFILE=C:\ProgramData
APPDATA=C:\Users\couchdb\AppData\Roaming
CLIENTNAME=continuity.muse
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=BUILD
ComSpec=C:\Windows\system32\cmd.exe
FP_NO_HOST_CHECK=NO
HOMEDRIVE=C:
HOMEPATH=\Users\couchdb
LOCALAPPDATA=C:\Users\couchdb\AppData\Local
LOGONSERVER=\\BUILD
NUMBER_OF_PROCESSORS=1
OPENSSL_CONF=C:\OpenSSL\bin\openssl.cfg
OS=Windows_NT
Path=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 15 Stepping 6, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=0f06
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PROMPT=$P$G
PSModulePath=C:\Windows\system32\WindowsPowerShell\v1.0\Modules\
PUBLIC=C:\Users\Public
SESSIONNAME=RDP-Tcp#0
SystemDrive=C:
SystemRoot=C:\Windows
TEMP=C:\Users\couchdb\AppData\Local\Temp
TMP=C:\Users\couchdb\AppData\Local\Temp
USERDOMAIN=BUILD
USERNAME=couchdb
USERPROFILE=C:\Users\couchdb
VS90COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools\
windir=C:\Windows

win7 std - vs2008 environment

ALLUSERSPROFILE=C:\ProgramData
APPDATA=C:\Users\couchdb\AppData\Roaming
CLIENTNAME=continuity.muse
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=BUILD
ComSpec=C:\Windows\system32\cmd.exe
DevEnvDir=C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE
FP_NO_HOST_CHECK=NO
FrameworkDir=C:\Windows\Microsoft.NET\Framework
FrameworkVersion=v2.0.50727
HOMEDRIVE=C:
HOMEPATH=\Users\couchdb
INCLUDE=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE;C:\Program Files\Microsoft SDKs\Windows\v6.0A\include;
LIB=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ATLMFC\LIB;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB;C:\Program Files\Microsoft SDKs\Windows\v6.0A\lib;
LIBPATH=C:\Windows\Microsoft.NET\Framework\;C:\Windows\Microsoft.NET\Framework\v2.0.50727;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ATLMFC\LIB;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB;
LOCALAPPDATA=C:\Users\couchdb\AppData\Local
LOGONSERVER=\\BUILD
NUMBER_OF_PROCESSORS=1
OPENSSL_CONF=C:\OpenSSL\bin\openssl.cfg
OS=Windows_NT
Path=C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN;C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools;C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools\bin;C:\Windows\Microsoft.NET\Framework\;C:\Windows\Microsoft.NET\Framework\\Microsoft .NET Framework 3.5 (Pre-Release Version);C:\Windows\Microsoft.NET\Framework\v2.0.50727;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\VCPackages;C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 15 Stepping 6, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=0f06
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PROMPT=$P$G
PSModulePath=C:\Windows\system32\WindowsPowerShell\v1.0\Modules\
PUBLIC=C:\Users\Public
SESSIONNAME=RDP-Tcp#0
SystemDrive=C:
SystemRoot=C:\Windows
TEMP=C:\Users\couchdb\AppData\Local\Temp
TMP=C:\Users\couchdb\AppData\Local\Temp
USERDOMAIN=BUILD
USERNAME=couchdb
USERPROFILE=C:\Users\couchdb
VCINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC
VS90COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools\
VSINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio 9.0
windir=C:\Windows
WindowsSdkDir=C:\Program Files\Microsoft SDKs\Windows\v6.0A\

a perfect path for erlang R13B04 after eval ./otp_build env_win32

export PATH=$ERL_TOP/release/win32/erts-5.7.5/bin:\
$ERL_TOP/erts/etc/win32/cygwin_tools/vc:\
$ERL_TOP/erts/etc/win32/cygwin_tools:\
/cygdrive/c/PROGRA~1/MICROS~1.0/Common7/IDE:\
/cygdrive/c/PROGRA~1/MICROS~1.0/VC/BIN:\
/cygdrive/c/PROGRA~1/MICROS~1.0/Common7/Tools:\
/cygdrive/c/WINDOWS/MICROS~1.NET/FRAMEW~1/:\
/cygdrive/c/WINDOWS/MICROS~1.NET/FRAMEW~1/V20~1.507:\
/cygdrive/c/PROGRA~1/MICROS~1.0/VC/VCPACK~1:\
/cygdrive/c/PROGRA~1/MICROS~3/Windows/v7.0/bin:\
/src/openssl:\
/src/nsis:\
/src/inno5:\
/usr/local/bin:\
/usr/bin:\
/bin:\
/cygdrive/c/WINDOWS/system32:\
/cygdrive/c/WINDOWS:\
/cygdrive/c/WINDOWS/System32/Wbem

export ERL_TOP=/src/otp_src_R13B04
export PATH=$ERL_TOP/release/win32/erts-5.7.5/bin:$ERL_TOP/erts/etc/win32/cygwin_tools/vc:$ERL_TOP/erts/etc/win32/cygwin_tools:/cygdrive/c/PROGRA~1/MICROS~1.0/Common7/IDE:/cygdrive/c/PROGRA~1/MICROS~1.0/VC/BIN:/cygdrive/c/PROGRA~1/MICROS~1.0/Common7/Tools:/cygdrive/c/WINDOWS/MICROS~1.NET/FRAMEW~1/:/cygdrive/c/WINDOWS/MICROS~1.NET/FRAMEW~1/V20~1.507:/cygdrive/c/PROGRA~1/MICROS~1.0/VC/VCPACK~1:/cygdrive/c/PROGRA~1/MICROS~3/Windows/v7.0/bin:/src/openssl:/src/nsis:/src/inno5:/usr/local/bin:/usr/bin:/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem

export ERL_TOP=/src/otp_src_R14A
EXPORT PATH=$ERL_TOP/release/win32/erts-5.8/bin:$ERL_TOP/erts/etc/win32/cygwin_tools/vc:$ERL_TOP/erts/etc/win32/cygwin_tools:/cygdrive/c/PROGRA~1/MICROS~1.0/Common7/IDE:/cygdrive/c/PROGRA~1/MICROS~1.0/VC/BIN:/cygdrive/c/PROGRA~1/MICROS~1.0/Common7/Tools:/cygdrive/c/WINDOWS/MICROS~1.NET/FRAMEW~1/:/cygdrive/c/WINDOWS/MICROS~1.NET/FRAMEW~1/V20~1.507:/cygdrive/c/PROGRA~1/MICROS~1.0/VC/VCPACK~1:/cygdrive/c/PROGRA~1/MICROS~3/Windows/v7.0/bin:/src/openssl:/src/nsis:/src/inno5:/usr/local/bin:/usr/bin:/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem

Download URLs - visible only in raw text view