| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" |
| "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| |
| |
| |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> |
| <title>PLC4X – </title> |
| <script src="../../js/jquery.slim.min.js" type="text/javascript"></script> |
| <!--script src="../../js/popper.min.js" type="javascript"></script--> |
| <script src="../../js/bootstrap.bundle.min.js" type="text/javascript"></script> |
| <!-- The tooling for adding images and links to Apache events --> |
| <script src="https://www.apachecon.com/event-images/snippet.js" type="text/javascript"></script> |
| <!-- FontAwesome --> |
| <link rel="stylesheet" href="../../css/all.min.css" type="text/css"/> |
| <!-- Bootstrap --> |
| <link rel="stylesheet" href="../../css/bootstrap.min.css" type="text/css"/> |
| <!-- Some Maven Site defaults --> |
| <link rel="stylesheet" href="../../css/maven-base.css" type="text/css"/> |
| <link rel="stylesheet" href="../../css/maven-theme.css" type="text/css"/> |
| <!-- The PLC4X version of a bootstrap theme --> |
| <link rel="stylesheet" href="../../css/themes/plc4x.css" type="text/css" id="pagestyle"/> |
| <!-- A custom style for printing content --> |
| <link rel="stylesheet" href="../../css/print.css" type="text/css" media="print"/> |
| |
| <meta http-equiv="Content-Language" content="en"/> |
| |
| </head> |
| <body class="composite"> |
| <nav class="navbar navbar-light navbar-expand-md bg-faded justify-content-center border-bottom"> |
| <!--a href="/" class="navbar-brand d-flex w-50 mr-auto">Navbar 3</a--> |
| <a href="https://plc4x.apache.org/" id="bannerLeft"><img src="../../images/apache_plc4x_logo_small.png" alt="Apache PLC4X"/></a> |
| <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#collapsingNavbar3"> |
| <span class="navbar-toggler-icon"></span> |
| </button> |
| <div class="navbar-collapse collapse w-100" id="collapsingNavbar3"> |
| <ul class="navbar-nav w-100 justify-content-center"> |
| <li class="nav-item"> |
| <a class="nav-link" href="../../index.html">Home</a> |
| </li> |
| <li class="nav-item active"> |
| <a class="nav-link" href="../../users/index.html">Users</a> |
| </li> |
| <li class="nav-item"> |
| <a class="nav-link" href="../../developers/index.html">Developers</a> |
| </li> |
| <li class="nav-item"> |
| <a class="nav-link" href="../../apache/index.html">Apache</a> |
| </li> |
| </ul> |
| <ul class="nav navbar-nav ml-auto justify-content-end"> |
| <li class="nav-item row valign-middle"> |
| <a class="acevent" data-format="wide" data-mode="light" data-event="random" style="width:240px;height:60px;"></a> |
| </li> |
| </ul> |
| </div> |
| </nav> |
| <div class="container-fluid"> |
| <div class="row h-100"> |
| |
| |
| <nav class="col-sm-push col-md-2 pt-3 sidebar"> |
| <div class="sidebar-sticky"> |
| <ul class="nav flex-column"> |
| <li class="nav-item"> |
| <a href="../../users/index.html" class="nav-link">Section Home</a> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/download.html" class="nav-link">Download</a> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/adopters.html" class="nav-link">Adopters</a> |
| </li> |
| |
| |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/commercial-support.html" class="nav-link">Commercial support</a> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/gettingstarted.html" class="nav-link">Getting Started</a> |
| <ul class="flex-column pl-4 nav"> |
| <li class="nav-item"> |
| <a href="../../users/getting-started/plc4go.html" class="nav-link">Go</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/getting-started/plc4j.html" class="nav-link">Java</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/getting-started/using-snapshots.html" class="nav-link">Using SNAPSHOTS</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/getting-started/general-concepts.html" class="nav-link">General Concepts</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/getting-started/virtual-modbus.html" class="nav-link">Virtual Modbus</a> |
| </li> |
| </ul> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/blogs-videos-and-slides.html" class="nav-link">Blogs, Videos and Slides</a> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/protocols/index.html" class="nav-link">Protocols</a> |
| <ul class="flex-column pl-4 nav"> |
| <li class="nav-item"> |
| <a href="../../users/protocols/ab-eth.html" class="nav-link">AB-ETH</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/ads.html" class="nav-link">ADS/AMS</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/bacnetip.html" class="nav-link">BACnet/IP</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/canopen.html" class="nav-link">CANopen</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/deltav.html" class="nav-link">DeltaV</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/df1.html" class="nav-link">DF1</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/ethernet-ip.html" class="nav-link">EtherNet/IP</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/firmata.html" class="nav-link">Firmata</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/knxnetip.html" class="nav-link">KNXnet/IP</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/modbus.html" class="nav-link">Modbus</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/opc-ua.html" class="nav-link">OPC UA</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/plc4x.html" class="nav-link">PLC4X (Proxy)</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/s7.html" class="nav-link">S7 (Step7)</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/protocols/simulated.html" class="nav-link">Simulated</a> |
| </li> |
| </ul> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/transports/index.html" class="nav-link">Transports</a> |
| <ul class="flex-column pl-4 nav"> |
| <li class="nav-item"> |
| <a href="../../users/transports/tcp.html" class="nav-link">TCP</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/transports/udp.html" class="nav-link">UDP</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/transports/serial.html" class="nav-link">Serial</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/transports/socketcan.html" class="nav-link">SocketCAN</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/transports/raw-socket.html" class="nav-link">Raw Socket</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/transports/pcap-replay.html" class="nav-link">PCAP Replay</a> |
| </li> |
| </ul> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/integrations/index.html" class="nav-link">Integrations</a> |
| <ul class="flex-column pl-4 nav"> |
| <li class="nav-item"> |
| <a href="../../users/integrations/apache-calcite.html" class="nav-link">Apache Calcite</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/integrations/apache-camel.html" class="nav-link">Apache Camel</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/integrations/apache-edgent.html" class="nav-link">Apache Edgent</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/integrations/apache-iotdb.html" class="nav-link">Apache IoTDB</a> |
| </li> |
| <li class="nav-item"> |
| <strong class="nav-link">Apache Kafka</strong> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/integrations/apache-nifi.html" class="nav-link">Apache NiFi</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/integrations/apache-streampipes.html" class="nav-link">Apache StreamPipes</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/integrations/eclipse-ditto.html" class="nav-link">Eclipse Ditto</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/integrations/eclipse-milo.html" class="nav-link">Eclipse Milo OPC UA Server</a> |
| </li> |
| </ul> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/tools/index.html" class="nav-link">Tools</a> |
| <ul class="flex-column pl-4 nav"> |
| <li class="nav-item"> |
| <a href="../../users/tools/capture-replay.html" class="nav-link">Capture Replay</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/tools/connection-pool.html" class="nav-link">Connection Pool</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/tools/connection-cache.html" class="nav-link">Connection Cache</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/tools/opm.html" class="nav-link">Object PLC Mapping (OPM)</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/tools/scraper.html" class="nav-link">Scraper</a> |
| </li> |
| <li class="nav-item"> |
| <a href="../../users/tools/testing.html" class="nav-link">PLC4X without a PLC and Unit Testing</a> |
| </li> |
| </ul> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/industry40.html" class="nav-link">Industry 4.0 with Apache</a> |
| </li> |
| |
| |
| <li class="nav-item"> |
| <a href="../../users/security.html" class="nav-link">Security</a> |
| </li> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| </ul> |
| </div> |
| </nav> |
| <main role="main" class="ml-sm-auto px-4 col-sm-pull col-md-9 col-lg-10 h-100"> |
| <div class="sect1"> |
| <h2 id="apache_kafka"><a href="https://kafka.apache.org/">Apache Kafka</a></h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>Apache Kafka is an open-source distributed event streaming platform used by thousands of |
| companies for high-performance data pipelines, streaming analytics, data integration, and |
| mission-critical applications.</p> |
| </div> |
| </div> |
| </div> |
| <h1 id="plc4x_kafka_connectors" class="sect0">PLC4X Kafka Connectors</h1> |
| <div class="paragraph"> |
| <p>The PLC4X connectors have the ability to pass data between Kafka and devices using industrial protocols. |
| They can be built from source from the latest <a href="https://plc4x.apache.org/users/download.html">release</a> of |
| PLC4X or from the latest snapshot from <a href="https://github.com/apache/plc4x">github</a>. |
| They can also be downloaded from the Confluent <a href="https://www.confluent.io/hub/apache/kafka-connect-plc4x-plc4j">hub</a>.</p> |
| </div> |
| <div class="sect1"> |
| <h2 id="introduction">Introduction</h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>A connect worker is basically a producer or consumer process with a standard api that Kafka can use to manage it. It is |
| able to be run in two modes:-</p> |
| </div> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>Standalone</p> |
| </li> |
| <li> |
| <p>Distributed</p> |
| </li> |
| </ul> |
| </div> |
| <div class="paragraph"> |
| <p>Standalone allows you to run the connector locally from the command line without having to install the jar file on your |
| Kafka brokers. |
| In distributed mode the connector runs on the Kafka brokers, which requires you to install the jar file on all of your |
| brokers. It allows the worker to be distrubuted across the Kafka brokers to provide redundancy and load balancing.</p> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="quickstart">Quickstart</h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>In order to start a Kafka Connect system the following steps have to be performed:</p> |
| </div> |
| <div class="paragraph"> |
| <p>1) Download the latest version of Apache Kafka binaries from here: <a href="https://kafka.apache.org/downloads" class="bare">https://kafka.apache.org/downloads</a>.</p> |
| </div> |
| <div class="paragraph"> |
| <p>2) Unpack the archive.</p> |
| </div> |
| <div class="paragraph"> |
| <p>3) Copy the <code>target/plc4j-apache-kafka-0.8.0-uber-jar.jar</code> to the Kafka <code>libs</code> or plugin directory specified |
| in the config/connect-distributed.properties file.</p> |
| </div> |
| <div class="paragraph"> |
| <p>4) Copy the files in the <code>config</code> to Kafka’s <code>config</code> directory.</p> |
| </div> |
| <div class="paragraph"> |
| <p>5) Make sure that the host name that the OPCUA server advertises during the discovery process is able |
| to be resolved from the Kafka Connect server. The easiest way to do this is to add the hostname to your |
| hosts file.</p> |
| </div> |
| <div class="sect2"> |
| <h3 id="start_a_kafka_broker">Start a Kafka Broker</h3> |
| <div class="paragraph"> |
| <p>1) Open 4 console windows and change directory into that directory |
| 2) Start Zookeeper:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin/zookeeper-server-start.sh config/zookeeper.properties</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>3) Start Kafka:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin/kafka-server-start.sh config/server.properties</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>4) Create the "test" topic:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>5) Start the consumer:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning</pre> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="source_connector">Source Connector</h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>The starting configuration for your connect worker is provided by a configuration file. However, once the worker has |
| started the configuration can be changed using the connect REST API which is generally available on |
| <a href="http://localhost:8083/connectors" class="bare">http://localhost:8083/connectors</a>. When running in distributed mode all the configuration needs to be done via the REST API.</p> |
| </div> |
| <div class="paragraph"> |
| <p>A sample configuration file is provided in the PLC4X Kafka integration repository in the <code>config/plc4x-source.properties</code> directory.. |
| This includes comments as well as meaningful properties that can be used with the worker.</p> |
| </div> |
| <div class="paragraph"> |
| <p>The configuration of the connectors via the REST interface expects the same properties as are specified within the |
| example <a href="https://github.com/apache/plc4x/tree/develop/plc4j/integrations/apache-kafka/config">config/plc4x-source.properties</a> file. These will need to be in JSON format and included with a couple of headers. |
| An example below shows the format it expects:-</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>curl -X POST -H "Content-Type: application/json" --data '{"name": "plc-source-test", "config": {"connector.class":"org.apache.plc4x.kafka.Plc4xSourceConnector", |
| // TODO: Continue here ... |
| "tasks.max":"1", "file":"test.sink.txt", "topics":"connect-test" }}' http://localhost:8083/connectors</pre> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="start_a_kafka_connect_source_worker_standalone">Start a Kafka Connect Source Worker (Standalone)</h3> |
| <div class="paragraph"> |
| <p>Ideal for testing.</p> |
| </div> |
| <div class="paragraph"> |
| <p>1) Start Kafka connect:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin/connect-standalone.sh config/connect-standalone.properties config/plc4x-source.properties</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>Now watch the console window with the "kafka-console-consumer".</p> |
| </div> |
| <div class="paragraph"> |
| <p>If you want to debug the connector, be sure to set some environment variables before starting Kafka-Connect:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>export KAFKA_DEBUG=y; export DEBUG_SUSPEND_FLAG=y;</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>In this case the startup will suspend till an IDE is connected via a remote-debugging session.</p> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="start_kafka_connect_source_worker_distributed_mode">Start Kafka Connect Source Worker (Distributed Mode)</h3> |
| <div class="paragraph"> |
| <p>Ideal for production.</p> |
| </div> |
| <div class="paragraph"> |
| <p>In this case the state of the node is handled by Zookeeper and the configuration of the connectors are distributed via Kafka topics.</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin/kafka-topics --create --zookeeper localhost:2181 --topic connect-configs --replication-factor 3 --partitions 1 --config cleanup.policy=compact |
| bin/kafka-topics --create --zookeeper localhost:2181 --topic connect-offsets --replication-factor 3 --partitions 50 --config cleanup.policy=compact |
| bin/kafka-topics --create --zookeeper localhost:2181 --topic connect-status --replication-factor 3 --partitions 10 --config cleanup.policy=compact</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>Starting the worker is then as simple as this:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin /connect-distributed.sh config/connect-distributed.properties</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>The configuration of the Connectors is then provided via REST interface:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>curl -X POST -H "Content-Type: application/json" --data '{"name": "plc-source-test", "config": {"connector.class":"org.apache.plc4x.kafka.Plc4xSourceConnector", |
| // TODO: Continue here ... |
| "tasks.max":"1", "file":"test.sink.txt", "topics":"connect-test" }}' http://localhost:8083/connectors</pre> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="sink_connector">Sink Connector</h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>See <a href="https://github.com/apache/plc4x/tree/develop/plc4j/integrations/apache-kafka/config">config/sink.properties</a> for an example configuration.</p> |
| </div> |
| <div class="sect2"> |
| <h3 id="start_a_kafka_connect_sink_worker_standalone">Start a Kafka Connect Sink Worker (Standalone)</h3> |
| <div class="paragraph"> |
| <p>Ideal for testing.</p> |
| </div> |
| <div class="paragraph"> |
| <p>1) Start Kafka connect:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin/connect-standalone.sh config/connect-standalone.properties config/plc4x-sink.properties</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>Now open console window with "kafka-console-producer".</p> |
| </div> |
| <div class="paragraph"> |
| <p>Producing to the kafka topic using the sample packet shown below should result all the values included in the payload |
| being sent to the PLC using the mapping defined in the sink properties.</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>{"schema": |
| {"type":"struct","fields": |
| [{"type":"struct","fields": |
| [{"type":"boolean","optional":true,"field":"running"}, |
| {"type":"boolean","optional":true,"field":"conveyorLeft"}, |
| {"type":"boolean","optional":true,"field":"conveyorRight"}, |
| {"type":"boolean","optional":true,"field":"load"}, |
| {"type":"int32","optional":true,"field":"numLargeBoxes"}, |
| {"type":"boolean","optional":true,"field":"unload"}, |
| {"type":"boolean","optional":true,"field":"transferRight"}, |
| {"type":"boolean","optional":true,"field":"transferLeft"}, |
| {"type":"boolean","optional":true,"field":"conveyorEntry"}, |
| {"type":"int32","optional":true,"field":"numSmallBoxes"}], |
| "optional":false,"name":"org.apache.plc4x.kafka.schema.Field","field":"fields"}, |
| {"type":"int64","optional":false,"field":"timestamp"}, |
| {"type":"int64","optional":true,"field":"expires"}], |
| "optional":false,"name":"org.apache.plc4x.kafka.schema.JobResult", |
| "doc":"PLC Job result. This contains all of the received PLCValues as well as a recieved timestamp"}, |
| "payload": |
| {"fields": |
| {"running":false,"conveyorLeft":true, |
| "conveyorRight":true,"load":false, |
| "numLargeBoxes":1630806456, |
| "unload":true, |
| "transferRight":false, |
| "transferLeft":true, |
| "conveyorEntry":false, |
| "numSmallBoxes":-1135309911}, |
| "timestamp":1606047842350, |
| "expires":null}}</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>If you want to debug the connector, be sure to set some environment variables before starting Kafka-Connect:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>export KAFKA_DEBUG=y; export DEBUG_SUSPEND_FLAG=y;</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>In this case the startup will suspend till an IDE is connected via a remote-debugging session.</p> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="start_kafka_connect_sink_worker_distributed_mode">Start Kafka Connect Sink Worker (Distributed Mode)</h3> |
| <div class="paragraph"> |
| <p>Ideal for production.</p> |
| </div> |
| <div class="paragraph"> |
| <p>In this case the state of the node is handled by Zookeeper and the configuration of the connectors are distributed via Kafka topics.</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin/kafka-topics --create --zookeeper localhost:2181 --topic connect-configs --replication-factor 3 --partitions 1 --config cleanup.policy=compact |
| bin/kafka-topics --create --zookeeper localhost:2181 --topic connect-offsets --replication-factor 3 --partitions 50 --config cleanup.policy=compact |
| bin/kafka-topics --create --zookeeper localhost:2181 --topic connect-status --replication-factor 3 --partitions 10 --config cleanup.policy=compact</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>Starting the worker is then as simple as this:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>bin /connect-distributed.sh config/connect-distributed.properties</pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>The configuration of the Connectors is then provided via REST interface:</p> |
| </div> |
| <div class="literalblock"> |
| <div class="content"> |
| <pre>curl -X POST -H "Content-Type: application/json" --data '{"name": "plc-sink-test", "config": {"connector.class":"org.apache.plc4x.kafka.Plc4xSinkConnector", |
| // TODO: Continue here ... |
| "tasks.max":"1", "file":"test.sink.txt", "topics":"connect-test" }}' http://localhost:8083/connectors</pre> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="graceful_backoff">Graceful Backoff</h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>If an error occurs when reading or writing PLC addresses a graceful backoff has been implemented so that the PLC isn’t |
| bombarded with requests. However as the number of connectors for each PLC should be limited to reduce the load on the PLC, |
| the graceful backoff shouldn’t have a major impact.</p> |
| </div> |
| <div class="paragraph"> |
| <p>For the source connector the PLC4X scraper logic is able to handle randomized polling rates on failures, this is buffered within the |
| connector, the poll rate of the connector has no affect on the PLC poll rate.</p> |
| </div> |
| <div class="paragraph"> |
| <p>For the sink connector, if a write fails it is retried a configurable number of times with a timeout between each time. |
| A Retriable Exception is raised which provides jitter for the timing of the retries.</p> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="schema_compatability">Schema Compatability</h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>PLC4X specifies a very basic schema and leaves the majority of the implementation to the user. It contains the |
| following fields:-</p> |
| </div> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>"fields": - This is a customized structure that is formed by the fields defined in the connector configuration. |
| This allows the user to defined arbitary fields within here all based on the PLC4X data types.</p> |
| </li> |
| <li> |
| <p>"timestamp": - This is the timestamp at which the PLC4X connector processed the PLC request.</p> |
| </li> |
| <li> |
| <p>"expires": - This field is used by the sink connector. It allows it to discard the record if it is too old. A value |
| of 0 or null indicates that the record some never be discarded no matter how old it is.</p> |
| </li> |
| </ul> |
| </div> |
| <div class="paragraph"> |
| <p>As the majority of the schema is left to the user to define we expect to be able to provide backward compatiblity |
| between the base schemas.</p> |
| </div> |
| <div class="paragraph"> |
| <p>The schemas for the sink and source connectors are the same. This allows us to producer from one PLC and send the |
| data to a sink.</p> |
| </div> |
| </div> |
| </div> |
| </main> |
| <footer class="pt-4 my-md-5 pt-md-5 w-100 border-top"> |
| <div class="row justify-content-md-center" style="font-size: 13px"> |
| <div class="col col-6 text-center"> |
| Copyright © 2017–2022 <a href="https://www.apache.org/">The Apache Software Foundation</a>. |
| All rights reserved.<br/> |
| Apache PLC4X, PLC4X, Apache, the Apache feather logo, and the Apache PLC4X project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries. All other marks mentioned may be trademarks or registered trademarks of their respective owners. |
| <br/><div style="text-align:center;">Home screen image taken from <a |
| href="https://flic.kr/p/chEftd">Flickr</a>, "Tesla Robot Dance" by Steve Jurvetson, licensed |
| under <a href="https://creativecommons.org/licenses/by/2.0/">CC BY 2.0 Generic</a>, image cropped |
| and blur effect added.</div> |
| </div> |
| </div> |
| </footer> |
| </div> |
| </div> |
| |
| <!-- Bootstrap core JavaScript |
| ================================================== --> |
| <!-- Placed at the end of the document so the pages load faster --> |
| <script src="../../js/jquery.slim.min.js"></script> |
| <script src="../../js/popper.min.js"></script> |
| <script src="../../js/bootstrap.min.js"></script> |
| <script type="text/javascript"> |
| $('.carousel .carousel-item').each(function(){ |
| var next = $(this).next(); |
| if (!next.length) { |
| next = $(this).siblings(':first'); |
| } |
| next.children(':first-child').clone().appendTo($(this)); |
| |
| for (let i = 0; i < 3; i++) { |
| next=next.next(); |
| if (!next.length) { |
| next = $(this).siblings(':first'); |
| } |
| next.children(':first-child').clone().appendTo($(this)); |
| } |
| }); |
| </script> |
| </body> |
| </html> |
