Since some of the Arrow developers work in the Python ecosystem, we are investing time in maintaining the thirdparty build dependencies for Arrow and related C++ libraries using the conda package manager. Others are free to add other development instructions for Windows here.
Miniconda is a minimal Python distribution including the conda package manager. To get started, download and install a 64-bit distribution.
We recommend using packages from conda-forge. Launch cmd.exe and run following commands:
conda config --add channels conda-forge
Now, you can bootstrap a build environment
conda create -n arrow-dev cmake git boost-cpp flatbuffers rapidjson cmake thrift-cpp snappy zlib brotli gflags lz4-c zstd -c conda-forge
Note:
Make sure to get the
conda-forge
build ofgflags
as the naming of the library differs from that in thedefaults
channel
Activate just created conda environment with pre-installed packages from previous step:
activate arrow-dev
We are using the cmake tool to support Windows builds. To allow cmake to pick up 3rd party dependencies, you should set ARROW_BUILD_TOOLCHAIN
environment variable to contain Library
folder path of new created on previous step arrow-dev
conda environment.
To set ARROW_BUILD_TOOLCHAIN
environment variable visible only for current terminal session you can run following. %CONDA_PREFIX
is set by conda to the current environment root by the activate
script.
set ARROW_BUILD_TOOLCHAIN=%CONDA_PREFIX%\Library
To validate value of ARROW_BUILD_TOOLCHAIN
environment variable you can run following terminal command:
echo %ARROW_BUILD_TOOLCHAIN%
As alternative to ARROW_BUILD_TOOLCHAIN
, it's possible to configure path to each 3rd party dependency separately by setting appropriate environment variable:
FLATBUFFERS_HOME
variable with path to flatbuffers
installation RAPIDJSON_HOME
variable with path to rapidjson
installation GFLAGS_HOME
variable with path to gflags
installation SNAPPY_HOME
variable with path to snappy
installation ZLIB_HOME
variable with path to zlib
installation BROTLI_HOME
variable with path to brotli
installation LZ4_HOME
variable with path to lz4
installation ZSTD_HOME
variable with path to zstd
installation
If you decided to use pre-built 3rd party dependencies libs, it‘s possible to configure Arrow’s cmake build script to search for customized names of 3rd party static libs.
zlib
. Pass -DARROW_ZLIB_VENDORED=OFF
to enable lookup of custom zlib build. Set ZLIB_HOME
environment variable. Pass -DZLIB_MSVC_STATIC_LIB_SUFFIX=%ZLIB_SUFFIX%
to link with z%ZLIB_SUFFIX%.lib
brotli
. Set BROTLI_HOME
environment variable. Pass -DBROTLI_MSVC_STATIC_LIB_SUFFIX=%BROTLI_SUFFIX%
to link with brotli*%BROTLI_SUFFIX%.lib. For brotli versions <= 0.6.0 installed from conda-forge this must be set to _static
, otherwise the default of -static
is used.
snappy
. Set SNAPPY_HOME
environment variable. Pass -DSNAPPY_MSVC_STATIC_LIB_SUFFIX=%SNAPPY_SUFFIX%
to link with snappy%SNAPPY_SUFFIX%.lib.
lz4
. Set LZ4_HOME
environment variable. Pass -LZ4_MSVC_STATIC_LIB_SUFFIX=%LZ4_SUFFIX%
to link with lz4%LZ4_SUFFIX%.lib.
zstd
. Set ZSTD_HOME
environment variable. Pass -ZSTD_MSVC_STATIC_LIB_SUFFIX=%ZSTD_SUFFIX%
to link with zstd%ZSTD_SUFFIX%.lib.
Microsoft provides the free Visual Studio Community edition. When doing development, you must launch the developer command prompt using
"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\VsDevCmd.bat" -arch=amd64
It's easiest to configure a console emulator like cmder to automatically launch this when starting a new development console.
Activate your conda build environment:
activate arrow-dev
Change working directory in cmd.exe to the root directory of Arrow and do an out of source build using nmake
:
cd cpp mkdir build cd build cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release .. nmake
When using conda, only release builds are currently supported.
Activate your conda build environment:
activate arrow-dev
Change working directory in cmd.exe to the root directory of Arrow and do an out of source build by generating a MSVC solution:
cd cpp mkdir build cd build cmake -G "Visual Studio 14 2015 Win64" -DCMAKE_BUILD_TYPE=Release .. cmake --build . --config Release
To build Debug version of Arrow you should have pre-insalled Debug version of boost libs.
It's recommended to configure cmake build with following variables for Debug build:
-DARROW_BOOST_USE_SHARED=OFF
- enables static linking with boost debug libs and simplifies run-time loading of 3rd parties. (Recommended)
-DBOOST_ROOT
- sets the root directory of boost libs. (Optional)
-DBOOST_LIBRARYDIR
- sets the directory with boost lib files. (Optional)
Command line to build Arrow in Debug might look as following:
cd cpp mkdir build cd build cmake -G "Visual Studio 14 2015 Win64" ^ -DARROW_BOOST_USE_SHARED=OFF ^ -DCMAKE_BUILD_TYPE=Debug ^ -DBOOST_ROOT=C:/local/boost_1_63_0 ^ -DBOOST_LIBRARYDIR=C:/local/boost_1_63_0/lib64-msvc-14.0 ^ .. cmake --build . --config Debug
To get the latest build instructions, you can reference msvc-build.bat, which is used by automated Appveyor builds.