A Quick Guide for Maintaining Dependencies with Vcpkg

It is recommended to read vcpkg documents. It is quite easy to understand if you have experience in packaging on linux or mac homebrew.

How to add a new depends?

Please init vcpkg env first:

./dev/vcpkg/init.sh

Vcpkg already maintains a lot of libraries. You can find them by vcpkg cli. (NOTE: Please always use cli because packages on vcpkg.io is outdated).

$ ./.vcpkg/vcpkg search folly
folly                    2022.10.31.00#4  An open-source C++ library developed and used at Facebook. The library is ...
folly[bzip2]                              Support bzip2 for compression
folly[libsodium]                          Support libsodium for cryto
folly[liburing]                           Support compile with liburing
folly[lz4]                                Support lz4 for compression
folly[lzma]                               Support LZMA for compression
folly[snappy]                             Support Snappy for compression
folly[zlib]                               Support zlib for compression
folly[zstd]                               Support zstd for compression

[...] means additional features. Then add the dependency into vcpkg.json.

{
  "velox": {
    "dependencies": [
      {
        "name": "folly",
        "features": ["zstd", "lz4", "lzma", "snappy"]
      }
    ]
  }
}

Run ./dev/vcpkg/init.sh again to verify that everything works. It will only build modified part.

Sometimes the vcpkg port does not meet the requirements. You can modify the packaging logic of vcpkg for different cases:

If vcpkg's port is too new (e.g. fmt). Vcpkg allows you to choose a specific version library. Supported versions can be found in .vcpkg/versions/$package. Then specify version in vcpkg.json. See also Versioning.

{
  "overrides": [
    { "name": "fmt", "version": "8.0.1" }
  ]
}

Otherwise, you must create a new port in ./ports/$package to override the vcpkg's original version.

If a newer version of a library is supported in a later version of vcpkg. Developers can configure vcpkg-configuration.json to allow importing a new port, overriding the corresponding port specified in the builtin-baseline. This approach also applies to libraries that have been removed from vcpkg at the builtin-baseline; by setting a historical vcpkg version, you can import such ports.

If you want to modify port based on vcpkg version. Copy port directory from ./.vcpkg/ports/$package to ./ports/$package.

If you want to create a new port. Create a new directory in ./ports/$package or generate a template via vcpkg cli:

./.vcpkg/vcpkg create $package /url/to/source/or/git
mv ./.vcpkg/ports/$package ./ports/$package

About Boost

Vcpkg splits boost into dozens of packages including header-only libraries. To find out which boost modules you are using, the easiest way is to grep all #include <boost/ in source code. It is not sufficient to just copy the components from find_package(Boost) in cmake scripts. As a result, modifying the boost version is a bit tricky, see more Pin old Boost versions.

Write port files

Create the manifest file

vcpkg.json is a json file describing the package's metadata:

{
  "name": "gsasl",
  "version": "2.2.0",
  "dependencies": [
    "krb5",
    "libntlm"
  ]
}

See vcpkg.json reference;

Create the portfile (aka build script)

portfile.cmake is a cmake script describing how to build and install the package. A typical portfile has 3 stages:

Download and prepare source:

# Download from Github
vcpkg_from_github(
    OUT_SOURCE_PATH SOURCE_PATH
    REPO gflags/gflags
    REF v2.2.2
    SHA512 98c4703aab24e81fe551f7831ab797fb73d0f7dfc516addb34b9ff6d0914e5fd398207889b1ae555bac039537b1d4677067dae403b64903577078d99c1bdb447
    HEAD_REF master
    PATCHES
        0001-patch-dir.patch # gflags was estimating a wrong relative path between the gflags-config.cmake file and the include path; "../.." goes from share/gflags/ to the triplet root
        fix_cmake_config.patch
)

# Download from source archive
vcpkg_download_distfile(ARCHIVE
    URLS "https://ftp.gnu.org/gnu/gsasl/gsasl-2.2.0.tar.gz" "https://www.mirrorservice.org/sites/ftp.gnu.org/gnu/gsasl/gsasl-2.2.0.tar.gz"
    FILENAME "gsasl-2.2.0.tar.gz"
    SHA512 0ae318a8616fe675e9718a3f04f33731034f9a7ba03d83ccb1a72954ded54ced35dc7c7e173fdcb6fa0f0813f8891c6cbcedf8bf70b37d00b8ec512eb9f07f5f
)
vcpkg_extract_source_archive(
    SOURCE_PATH
    ARCHIVE "${ARCHIVE}"
    PATCHES fix-krb5-config.patch
)

Configure, build and install. Vcpkg will add extra flags (e.g. --install-prefix) for common build system automatically.

# CMake
vcpkg_configure_cmake(
    SOURCE_PATH ${SOURCE_PATH}
    OPTIONS
        -DCMAKE_PROGRAM_PATH=${CURRENT_HOST_INSTALLED_DIR}/tools/yasm
        -DWITH_KERBEROS=on
)
vcpkg_install_cmake()

# GNU Autotools (Out of tree)
vcpkg_configure_make(SOURCE_PATH ${SOURCE_PATH} AUTOCONFIG)
vcpkg_install_make()

# Meson
vcpkg_configure_meson(SOURCE_PATH ${SOURCE_PATH})
vcpkg_install_meson()

Some libraries do not support out-of-tree build. There is a hidden option for vcpkg_configure_make():

vcpkg_configure_make(
    SOURCE_PATH ${SOURCE_PATH}
    COPY_SOURCE
)

Post patch. You need to modify the final package in order to make the binary library work in any location. Vcpkg will also validate the final package and warn common issues.

# Remove unused files
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/share")
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include")
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/tools")

# Replace absolute path in pkgconfig
vcpkg_fixup_pkgconfig()

# Missing copyright
vcpkg_install_copyright(FILE_LIST "${SOURCE_PATH}/LICENSE")

See portfile variable and function reference.

Optional Files

usage: A txt file about how to use the package. See example: gflags
vcpkg-cmake-wrapper.cmake: A cmake script to wrapper library's find module. If exists, find_package() will exec it instead of Find*.cmake. See example: gflags

Package Layout

Build intermediate files can be found in .vcpkg/buildtrees/$package. Built packages can be found in .vcpkg/$package:

packages/thrift_x64-linux-avx/
├── BUILD_INFO
├── CONTROL
├── debug
│   └── lib
│       ├── libthriftd.a
│       ├── ...
│       └── pkgconfig
│           ├── thrift.pc
│           └── ...
├── include
│   └── thrift
│       ├── Thrift.h
│       └── ...
├── lib
│   ├── libthrift.a
│   ├── ...
│   └── pkgconfig
│       ├── thrift.pc
│       └── ...
├── share
│   └── thrift
│       ├── copyright
│       ├── ThriftConfig.cmake
│       ├── ...
│       ├── vcpkg_abi_info.txt
│       └── vcpkg.spdx.json
└── tools
    └── thrift
        └── thrift

Most of them are same as the common linux packages layout, except:

debug/lib: debug-build libraries
share/$package: cmake module (aka /usr/lib/cmake/$package), copyright, usage info, vcpkg abi (hash of build scripts).
tools/$package: executables (aka /bin)

About Binary Cache

Binary caching saves copies of library binaries in a shared location that can be accessed by vcpkg for future installation. This means that, as a user, you should only need to build dependencies from source once. If vcpkg is asked to install the same library with the same build configuration in the future, it will copy the built binaries from the cache and finish the operation in seconds.

Binary cache is enabled by default. Cache files is located in ~/.cache/vcpkg/archives. See binary cache.