Wrap the interruption to a custom exception when a blocking API is interrupted (#99)

### Motivation

Currently, when a blocking API is interrupted by a signal, `SystemError`
will be thrown. However, in this case, `PyErr_SetInterrupt` will be
called and next time a blocking API is called, `std::system_error` will
be somehow thrown.

The failure of
https://lists.apache.org/thread/cmzykd9qz9x1d0s35nc5912o3slwpxpv is
caused by this issue. The `SystemError` is not called, then
`client.close()` will be skipped, which leads to the `bad_weak_ptr`

P.S. Currently we have to call `client.close()` on a `Client` instance,
otherwise, the `bad_weak_ptr` will be thrown.

However, even if we caught the `SystemError` like:

        msg = consumer.receive()
        # ...
    except SystemError:

we would still see the following error:

terminate called after throwing an instance of 'std::system_error'
  what():  Operation not permitted

### Modifications

- Wrap `ResultInterrupted` into the `pulsar.Interrupted` exception.
- Refactor the `waitForAsyncValue` and `waitForAsyncResult` functions
  and raise `pulsar.Interrupted` when `PyErr_CheckSignals` detects a
- Add `InterruptedTest` to cover this case.
- Remove `future.h` since we now use `std::future` instead of the
  manually implemented `Future`.
- Fix the `examples/consumer.py` to support stopping by Ctrl+C.

(cherry picked from commit ec05f50bf489aef85532d61f577c62649a5b71a6)
11 files changed
tree: b042fab3d72e0411255ea5a2b184bf964911b88c
  1. .github/
  2. build-support/
  3. cmake_modules/
  4. examples/
  5. pkg/
  6. pulsar/
  7. src/
  8. tests/
  9. .asf.yaml
  10. .clang-format
  11. .gitignore
  12. .gitmodules
  13. CMakeLists.txt
  15. dependencies.yaml
  17. NOTICE
  18. README.md
  19. RELEASE.md
  20. SECURITY.md
  21. setup.py

Pulsar Python client library


PyBind11 is a header-only library and a submodule, so you can simply download the submodule so that CMake can find this dependency.

git submodule update --init

You can also download the pybind11 directly like:

pip3 install pyyaml
export PYBIND11_VERSION=$(./build-support/dep-version.py pybind11)
curl -L -O https://github.com/pybind/pybind11/archive/refs/tags/v${PYBIND11_VERSION}.tar.gz
tar zxf v${PYBIND11_VERSION}.tar.gz
mv pybind11-${PYBIND11_VERSION} pybind11

After that, you only need to install the Pulsar C++ client dependency into the system path. You can install the pre-built binaries or build from source.

Install the Python wheel

Make sure the PyBind11 submodule has been downloaded and the Pulsar C++ client has been installed. Then run the following commands:

cmake -B build
cmake --build build 
cmake --install build
python3 ./setup.py bdist_wheel
python3 -m pip install dist/pulsar_client-*.whl --force-reinstall


  1. Here a separate build directory is created to store all CMake temporary files. However, the setup.py requires the _pulsar.so is under the project directory.
  2. Add the --force-reinstall option to overwrite the existing Python wheel in case your system has already installed a wheel before.
  3. On Windows, the Python command is py instead of python3.

Running examples

You can run python3 -c 'import pulsar' to see whether the wheel has been installed successfully. If it failed, check whether dependencies (e.g. libpulsar.so) are in the system path. If not, make sure the dependencies are in LD_LIBRARY_PATH (on Linux) or DYLD_LIBRARY_PATH (on macOS).

Then you can run examples as a simple end-to-end test.

# In terminal 1
python3 ./examples/consumer.py
# In terminal 2
python3 ./examples/producer.py

Before executing the commands above, you must ensure the Pulsar service is running. See here for quick start.

Unit tests

Before running the unit tests, you must run a Pulsar service with all things set up:


The command above runs a Pulsar standalone in a Docker container. You can run ./build-support/pulsar-test-service-stop.sh to stop it.

Run all unit tests:


Run a single unit test (e.g. PulsarTest.test_tls_auth):

python3 ./tests/pulsar_test.py 'PulsarTest.test_tls_auth'

Generate API docs

Pulsar Python Client uses pydoctor to generate API docs. To generate by yourself, run the following command in the root path of this repository:

sudo python3 -m pip install pydoctor
pydoctor --make-html \
  --html-viewsource-base=https://github.com/apache/pulsar-client-python/tree/<release-version-tag> \
  --docformat=numpy --theme=readthedocs \
  --intersphinx=https://docs.python.org/3/objects.inv \
  --html-output=<path-to-apidocs> \