The Netty Source connector opens a port that accept incoming data via the configured network protocol and publish it to a user-defined Pulsar topic. Also, this connector is suggested to be used in a containerized (e.g. k8s) deployment. Otherwise, if the connector is running in process or thread mode, the instances may be conflicting on listening to ports.
Name | Required | Default | Description |
---|---|---|---|
type | false | tcp | The network protocol over which data is trasmitted to netty. Valid values include HTTP, TCP, and UDP |
host | false | 127.0.0.1 | The host name or address that the source instance to listen on. |
port | false | 10999 | The port that the source instance to listen on. |
numberOfThreads | false | 1 | The number of threads of Netty Tcp Server to accept incoming connections and handle the traffic of the accepted connections. |
Here is a configuration Json example:
{ "type": "tcp", "host": "127.0.0.1", "port": "10911", "numberOfThreads": "5" }
Here is a configuration Yaml example:
configs: type: "tcp" host: "127.0.0.1" port: 10999 numberOfThreads: 1
docker pull apachepulsar/pulsar:2.4.0 docker run -d -it -p 6650:6650 -p 8080:8080 -v $PWD/data:/pulsar/data --name pulsar-netty-standalone apachepulsar/pulsar:2.4.0 bin/pulsar standalone
configs: type: "tcp" host: "127.0.0.1" port: 10999 numberOfThreads: 1
docker cp netty-source-config.yaml pulsar-netty-standalone:/pulsar/conf/
docker exec -it pulsar-netty-standalone /bin/bash curl -O http://mirror-hk.koddos.net/apache/pulsar/pulsar-2.4.0/connectors/pulsar-io-netty-2.4.0.nar ./bin/pulsar-admin sources localrun --archive pulsar-io-{{pulsar:version}}.nar --tenant public --namespace default --name netty --destination-topic-name netty-topic --source-config-file netty-source-config.yaml --parallelism 1
docker exec -it pulsar-netty-standalone /bin/bash ./bin/pulsar-client consume -t Exclusive -s netty-sub netty-topic -n 0
docker exec -it pulsar-netty-standalone /bin/bash apt-get update apt-get -y install telnet root@1d19327b2c67:/pulsar# telnet 127.0.0.1 10999 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. hello world
In the consumer window just opened, you can see the following data
----- got message ----- hello ----- got message ----- world
configs: type: "http" host: "127.0.0.1" port: 10999 numberOfThreads: 1
docker exec -it pulsar-netty-standalone /bin/bash ./bin/pulsar-admin sources localrun --archive pulsar-io-{{pulsar:version}}.nar --tenant public --namespace default --name netty --destination-topic-name netty-topic --source-config-file netty-source-config.yaml --parallelism 1
curl -X POST --data 'hello, world!' http://127.0.0.1:10999/
In the consumer window just opened, you can see the following data
----- got message ----- hello, world!