Google Git
Sign in
apache / pulsar-client-python / c5c177af647997d1182f3d4a69c74824f39af38f
commitc5c177af647997d1182f3d4a69c74824f39af38f[log] [tgz]
authorBaodi Shi <wudixiaolongbao@icloud.com>Wed Nov 16 20:00:21 2022 +0800
committerGitHub <noreply@github.com>Wed Nov 16 20:00:21 2022 +0800
treed176e3078d02d6429efd5f792aa3cb1f0a9a5716
parent1554e39e105fe80a6427801284a6b11fdc23bff2 [diff]
[feat] Support consumer batch receive. (#33)
4 files changed
tree: d176e3078d02d6429efd5f792aa3cb1f0a9a5716
  1. .github/
  2. build-support/
  3. cmake_modules/
  4. examples/
  5. pkg/
  6. pulsar/
  7. src/
  8. tests/
  9. vcpkg
  10. .asf.yaml
  11. .clang-format
  12. .gitignore
  13. .gitmodules
  14. CMakeLists.txt
  15. CONTRIBUTING.md
  16. dependencies.yaml
  17. LICENSE
  18. NOTICE
  19. pulsar-client-cpp-version.txt
  20. README.md
  21. SECURITY.md
  22. setup.py
  23. vcpkg-3.10.json
  24. vcpkg-3.7.json
  25. vcpkg-3.8.json
  26. vcpkg-3.9.json
  27. vcpkg.json
  28. version.txt
README.md

Pulsar Python client library

Requirements

Install the Python wheel

Windows (with Vcpkg)

First, install the dependencies via Vcpkg.

vcpkg install --feature-flags=manifests --triplet x64-windows

NOTE: For Windows 32-bit library, change x64-windows to x86-windows, see here for all available triplets.

Then, build and install the Python wheel.

# Assuming the Pulsar C++ client has been installed under the `PULSAR_CPP` directory.
cmake -B build -DUSE_VCPKG=ON -DCMAKE_PREFIX_PATH="$env:PULSAR_CPP" -DLINK_STATIC=ON
cmake --build build --config Release
cmake --install build
py setup.py bdist_wheel
py -m pip install ./dist/pulsar_client-*.whl

Since the Python client links to Boost.Python dynamically, you have to copy the dll (e.g. boost_python310-vc142-mt-x64-1_80.dll) into the system path (the PATH environment variable). If the -DLINK_STATIC=ON option is not specified, you have to copy the pulsar.dll into the system path as well.

Linux or macOS

Assuming the Pulsar C++ client and Boost.Python have been installed under the system path.

cmake -B build
cmake --build build -j8
cmake --install build
./setup.py bdist_wheel
pip3 install dist/pulsar_client-*.whl --force-reinstall

NOTE

  1. Here a separate build directory is created to store all CMake temporary files. However, the setup.py requires the _pulsar.so is under the project directory.
  2. Add the --force-reinstall option to overwrite the existing Python wheel in case your system has already installed a wheel before.

Running examples

You can run python3 -c 'import pulsar' to see whether the wheel has been installed successfully. If it failed, check whether dependencies (e.g. libpulsar.so) are in the system path. If not, make sure the dependencies are in LD_LIBRARY_PATH (on Linux) or DYLD_LIBRARY_PATH (on macOS).

Then you can run examples as a simple end-to-end test.

# In terminal 1
python3 ./examples/consumer.py

# In terminal 2
python3 ./examples/producer.py

Before executing the commands above, you must ensure the Pulsar service is running. See here for quick start.

Unit tests

Before running the unit tests, you must run a Pulsar service with all things set up:

./build-support/pulsar-test-service-start.sh

The command above runs a Pulsar standalone in a Docker container. You can run ./build-support/pulsar-test-service-stop.sh to stop it.

Run all unit tests:

./tests/run-unit-tests.sh

Run a single unit test (e.g. PulsarTest.test_tls_auth):

python3 ./tests/pulsar_test.py 'PulsarTest.test_tls_auth'

Generate API docs

Pulsar Python Client uses pydoctor to generate API docs. To generate by yourself, run the following command in the root path of this repository:

pip3 install pydoctor
pydoctor --make-html \
  --html-viewsource-base=https://github.com/apache/pulsar-client-python/tree/<release-version-tag> \
  --docformat=numpy --theme=readthedocs \
  --intersphinx=https://docs.python.org/3/objects.inv \
  --html-output=<path-to-apidocs> \
  pulsar