commit | c59f8801a1093831ef354e02e0692c4ce5b8dcc0 | [log] [tgz] |
---|---|---|
author | Zike Yang <zike@apache.org> | Mon Apr 10 21:01:02 2023 +0800 |
committer | GitHub <noreply@github.com> | Mon Apr 10 21:01:02 2023 +0800 |
tree | a197e1f61502b19d7758621d76205cce2a002f60 | |
parent | ceaea5006b2e0a318c986f79fa89f0cc1825ab4f [diff] |
[Fix] Fix message listener doesn't respect receiver queue size (#309) Fixes #308 ### Motivation This bug is from the nodejs client but not the c++ client. The root cause is that the message listener calls the user callback in an asynchronous way. It will not wait for the result from the user callback, and then it will process the next messages immediately. ### Modifications * Use Napi::Promise to wait for the user callback to complete in the message listener. The message listener can detect whether the user function is asynchronous or synchronous. * If it is a synchronous function, it will behave the same as before. This is not an issue because `Napi::Function::Call` will wait for the synchronous function to complete. * If it is an asynchronous function, the user function must return a `Promise` object. We use `promise.then()` to determine when the user function is finished. The message listener will wait in the C++ client thread. We should not wait in the Node thread because it will block the entire Node main thread. This way, we can also utilize the feature of `MessageListenerThreads`. Previously, the configuration `MessageListenerThreads` did not work for the asynchronous callback. This PR also fixes this issue.
The Pulsar Node.js client can be used to create Pulsar producers and consumers in Node.js.
This library works only in Node.js 10.x or later because it uses the node-addon-api module to wrap the C++ library.
Note
These instructions are only available for versions after 1.8.0. For versions previous to 1.8.0, you need to install the C++ client first. Please switch to the corresponding version branch of this repo to read the specific instructions.
To run the examples, skip this section.
To use the Pulsar Node.js client in your project, run:
npm install pulsar-client
or
yarn add pulsar-client
Then you can run the following simple end-to-end example:
const Pulsar = require('pulsar-client'); (async () => { // Create a client const client = new Pulsar.Client({ serviceUrl: 'pulsar://localhost:6650' }); // Create a producer const producer = await client.createProducer({ topic: 'persistent://public/default/my-topic', }); // Create a consumer const consumer = await client.subscribe({ topic: 'persistent://public/default/my-topic', subscription: 'sub1' }); // Send a message producer.send({ data: Buffer.from("hello") }); // Receive the message const msg = await consumer.receive(); console.log(msg.getData().toString()); consumer.acknowledge(msg); await producer.close(); await consumer.close(); await client.close(); })();
You should find the output as:
hello
You can see more examples in the examples directory. However, since these examples might use an API that was not released yet, you need to build this module. See the next section.
Note
Build from source code requires the Node.js version greater than 16.18
First, clone the repository.
git clone https://github.com/apache/pulsar-client-node.git cd pulsar-client-node
Since this client is a C++ addon that depends on the Pulsar C++ client, you need to install the C++ client first. You need to ensure there is a C++ compiler that supports C++11 installed in your system.
pkg/linux/download-cpp-client.sh
pkg\windows\download-cpp-client.bat
pkg/mac/build-cpp-deps-lib.sh pkg/mac/build-cpp-lib.sh
After the C++ client is installed, run the following command to build this C++ addon.
npm install
To verify it has been installed successfully, you can run an example like:
Note
A running Pulsar server is required. The example uses
pulsar://localhost:6650
to connect to the server.
node examples/producer
You should find the output as:
Sent message: my-message-0 Sent message: my-message-1 Sent message: my-message-2 Sent message: my-message-3 Sent message: my-message-4 Sent message: my-message-5 Sent message: my-message-6 Sent message: my-message-7 Sent message: my-message-8 Sent message: my-message-9
You can generate the API docs by:
npm install npx typedoc # Documentation generated at ./apidocs