Google Git
Sign in
apache / pulsar-client-python / 325f852df87dc15d68dd87fd540cf71e61d93281
commit325f852df87dc15d68dd87fd540cf71e61d93281[log] [tgz]
authorYunze Xu <xyzinfernity@163.com>Wed Nov 02 11:41:40 2022 +0800
committerGitHub <noreply@github.com>Tue Nov 01 20:41:40 2022 -0700
tree1172d68df4a60739a9a1b9f639d67ef1374a49cf
parent7c567df4b4f675243f2f7dcafea8ef2da4df4ca0 [diff]
Support building with MSVC (#23)

Fixes https://github.com/apache/pulsar-client-python/issues/7

The Python client cannot be built with MSVC.

CMakeLists.txt:
1. Boost.Python cannot be found on Windows. The component of Boost
   cannot be `python3`. It should be a specific version like
   `python310`. Therefore, find all possible components from 3.10 to 3.7
   until the first component is available.
2. For MSVC, link to `pulsarWithDeps.lib` when `LINK_STATIC` is `ON` and
   set the `MSVC_RUNTIME_LIBRARY` target property to `MultiThreaded`.
3. Change the suffix from `.so` to `.pyd` because Python on Windows
   recognizes `*.pyd` as the dynamic library.
4. Link to Python library with MSVC, otherwise the symbos cannot be
   found when linking `boost-python`.

README: tell users how to build Python client on Windows.

Add a workflow to build Python wheels of versions 3.7 to 3.10 on Windows
and verify the build will succeed.

With this PR, Windows users still need to put the related
`boost_python*.dll` under the system path. In future, the `boost-python`
dependency will be removed. See
https://github.com/apache/pulsar-client-python/issues/24.
12 files changed
tree: 1172d68df4a60739a9a1b9f639d67ef1374a49cf
  1. .github/
  2. build-support/
  3. cmake_modules/
  4. examples/
  5. pkg/
  6. pulsar/
  7. src/
  8. tests/
  9. vcpkg
  10. .asf.yaml
  11. .clang-format
  12. .gitignore
  13. .gitmodules
  14. CMakeLists.txt
  15. CONTRIBUTING.md
  16. dependencies.yaml
  17. LICENSE
  18. NOTICE
  19. pulsar-client-cpp-version.txt
  20. README.md
  21. SECURITY.md
  22. setup.py
  23. vcpkg-3.10.json
  24. vcpkg-3.7.json
  25. vcpkg-3.8.json
  26. vcpkg-3.9.json
  27. vcpkg.json
  28. version.txt
README.md

Pulsar Python client library

Requirements

Install the Python wheel

Windows (with Vcpkg)

First, install the dependencies via Vcpkg.

vcpkg install --feature-flags=manifests --triplet x64-windows

NOTE: For Windows 32-bit library, change x64-windows to x86-windows, see here for all available triplets.

Then, build and install the Python wheel.

# Assuming the Pulsar C++ client has been installed under the `PULSAR_CPP` directory.
cmake -B build -DUSE_VCPKG=ON -DCMAKE_PREFIX_PATH="$env:PULSAR_CPP" -DLINK_STATIC=ON
cmake --build build --config Release
cmake --install build
py setup.py bdist_wheel
py -m pip install ./dist/pulsar_client-*.whl

Since the Python client links to Boost.Python dynamically, you have to copy the dll (e.g. boost_python310-vc142-mt-x64-1_80.dll) into the system path (the PATH environment variable). If the -DLINK_STATIC=ON option is not specified, you have to copy the pulsar.dll into the system path as well.

Linux or macOS

Assuming the Pulsar C++ client and Boost.Python have been installed under the system path.

cmake -B build
cmake --build build -j8
cmake --install build
./setup.py bdist_wheel
pip3 install dist/pulsar_client-*.whl --force-reinstall

NOTE

  1. Here a separate build directory is created to store all CMake temporary files. However, the setup.py requires the _pulsar.so is under the project directory.
  2. Add the --force-reinstall option to overwrite the existing Python wheel in case your system has already installed a wheel before.

Running examples

You can run python3 -c 'import pulsar' to see whether the wheel has been installed successfully. If it failed, check whether dependencies (e.g. libpulsar.so) are in the system path. If not, make sure the dependencies are in LD_LIBRARY_PATH (on Linux) or DYLD_LIBRARY_PATH (on macOS).

Then you can run examples as a simple end-to-end test.

# In terminal 1
python3 ./examples/consumer.py

# In terminal 2
python3 ./examples/producer.py

Before executing the commands above, you must ensure the Pulsar service is running. See here for quick start.

Unit tests

Before running the unit tests, you must run a Pulsar service with all things set up:

./build-support/pulsar-test-service-start.sh

The command above runs a Pulsar standalone in a Docker container. You can run ./build-support/pulsar-test-service-stop.sh to stop it.

Run all unit tests:

./tests/run-unit-tests.sh

Run a single unit test (e.g. PulsarTest.test_tls_auth):

python3 ./tests/pulsar_test.py 'PulsarTest.test_tls_auth'