Google Git
Sign in
apache / pulsar-client-python / 39ac8f80d82ca10df5c719b3520985352f578ad4
commit39ac8f80d82ca10df5c719b3520985352f578ad4[log] [tgz]
authorYunze Xu <xyzinfernity@163.com>Fri May 26 08:02:36 2023 +0800
committerGitHub <noreply@github.com>Thu May 25 17:02:36 2023 -0700
tree5a29713558af1f188a7af13138316475393e0cfd
parentd2fac8fb8bbaaaa7d134abc35e7f1f8f89f615be [diff]
Include the C extension when generating API docs (#126)

### Motivation

https://github.com/apache/pulsar-client-python/issues/85#issuecomment-1531069608

Some targets in the API docs are referenced from the `_pulsar` module,
while the `pydoctor` command does not generate API docs for it. It's not
friendly to users, e.g. they cannot find which values could
`_pulsar.ConsumerType` be.

```
pulsar/__init__.py:695: Cannot find link target for "_pulsar.ConsumerType"
```

### Modifications

Fix the documents to describe how to include the API docs for the
`_pulsar` module when generating API docs. It also fixes some other
warnings when running the `pydoctor` command.
3 files changed
tree: 5a29713558af1f188a7af13138316475393e0cfd
  1. .github/
  2. build-support/
  3. cmake_modules/
  4. examples/
  5. pkg/
  6. pulsar/
  7. src/
  8. tests/
  9. pybind11
  10. .asf.yaml
  11. .clang-format
  12. .gitignore
  13. .gitmodules
  14. CMakeLists.txt
  15. CONTRIBUTING.md
  16. dependencies.yaml
  17. LICENSE
  18. NOTICE
  19. README.md
  20. RELEASE.md
  21. SECURITY.md
  22. setup.py
README.md

Pulsar Python client library

Pulsar Python clients support a variety of Pulsar features to enable building applications connecting to your Pulsar cluster. For the supported Pulsar features, see Client Feature Matrix.

Requirements

PyBind11 is a header-only library and a submodule, so you can simply download the submodule so that CMake can find this dependency.

git submodule update --init

You can also download the pybind11 directly like:

pip3 install pyyaml
export PYBIND11_VERSION=$(./build-support/dep-version.py pybind11)
curl -L -O https://github.com/pybind/pybind11/archive/refs/tags/v${PYBIND11_VERSION}.tar.gz
tar zxf v${PYBIND11_VERSION}.tar.gz
mv pybind11-${PYBIND11_VERSION} pybind11

After that, you only need to install the Pulsar C++ client dependency into the system path. You can install the pre-built binaries or build from source.

Install the Python wheel

Make sure the PyBind11 submodule has been downloaded and the Pulsar C++ client has been installed. Then run the following commands:

cmake -B build
cmake --build build
cmake --install build
python3 ./setup.py bdist_wheel
python3 -m pip install dist/pulsar_client-*.whl --force-reinstall

NOTE

  1. The separate build directory is created to store all CMake temporary files. However, the setup.py requires the _pulsar.so to be under the project directory.
  2. Add the --force-reinstall option to overwrite the existing Python wheel in case your system has already installed a wheel before.
  3. On Windows, the Python command is py instead of python3.

Running examples

You can run python3 -c 'import pulsar' to see whether the wheel has been installed successfully. If it fails, check whether dependencies (e.g., libpulsar.so) are in the system path. If not, make sure the dependencies are in LD_LIBRARY_PATH (on Linux) or DYLD_LIBRARY_PATH (on macOS).

Then you can run examples as a simple end-to-end test.

# In terminal 1
python3 ./examples/consumer.py

# In terminal 2
python3 ./examples/producer.py

Before executing the commands above, you must ensure the Pulsar service is running. See here for quick start.

Unit tests

Before running the unit tests, you must run a Pulsar service with all things set up:

./build-support/pulsar-test-service-start.sh

The command above runs a Pulsar standalone in a Docker container. You can run ./build-support/pulsar-test-service-stop.sh to stop it.

Run all unit tests:

./tests/run-unit-tests.sh

Run a single unit test (e.g., PulsarTest.test_tls_auth):

python3 ./tests/pulsar_test.py 'PulsarTest.test_tls_auth'

Generate API docs

Pulsar Python Client uses pydoctor to generate API docs. To generate by yourself, you need to install the Python library first. Then run the following command in the root path of this repository:

sudo python3 -m pip install pydoctor
cp $(python3 -c 'import _pulsar, os; print(_pulsar.__file__)') ./_pulsar.so
pydoctor --make-html \
  --docformat=numpy --theme=readthedocs \
  --intersphinx=https://docs.python.org/3/objects.inv \
  --html-output=<path-to-apidocs> \
  --introspect-c-modules \
  ./_pulsar.so \
  pulsar

Then the index page will be generated in <path-to-apidocs>/index.html.

Contribute

We welcome contributions from the open source community!

If your contribution adds Pulsar features for Python clients, you need to update both the Pulsar docs and the Client Feature Matrix. See Contribution Guide for more details.