docs/components/modules/ROOT/pages/flink-component.adoc

[[flink-component]]
= Flink Component
//THIS FILE IS COPIED: EDIT THE SOURCE FILE:
:page-source: components/camel-flink/src/main/docs/flink-component.adoc
:docTitle: Flink
:artifactId: camel-flink
:description: Send DataSet jobs to an Apache Flink cluster.
:since: 2.18
:supportLevel: Stable
:component-header: Only producer is supported

*Since Camel {since}*

*{component-header}*

*Since Camel 2.18*


This documentation page covers the https://flink.apache.org[Apache Flink]
component for the Apache Camel. The *camel-flink* component provides a
bridge between Camel connectors and Flink tasks. +
This Camel Flink connector provides a way to route message from various
transports, dynamically choosing a flink task to execute, use incoming
message as input data for the task and finally deliver the results back to
the Camel pipeline.

Maven users will need to add the following dependency to
their `pom.xml` for this component:

[source,xml]
------------------------------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-flink</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>
------------------------------------------------------------

== URI Format

Currently, the Flink Component supports only Producers. One can create DataSet, DataStream jobs.

[source,java]
-------------------------------------------------
flink:dataset?dataset=#myDataSet&dataSetCallback=#dataSetCallback
flink:datastream?datastream=#myDataStream&dataStreamCallback=#dataStreamCallback
-------------------------------------------------

[[Flink-FlinkEndpointOptions]]
FlinkEndpoint Options



// endpoint options: START
The Flink endpoint is configured using URI syntax:

----
flink:endpointType
----

with the following path and query parameters:

=== Path Parameters (1 parameters):


[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *endpointType* | *Required* Type of the endpoint (dataset, datastream). The value can be one of: dataset, datastream |  | EndpointType
|===


=== Query Parameters (8 parameters):


[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *collect* (producer) | Indicates if results should be collected or counted. | true | boolean
| *dataSet* (producer) | DataSet to compute against. |  | DataSet
| *dataSetCallback* (producer) | Function performing action against a DataSet. |  | DataSetCallback
| *dataStream* (producer) | DataStream to compute against. |  | DataStream
| *dataStreamCallback* (producer) | Function performing action against a DataStream. |  | DataStreamCallback
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *basicPropertyBinding* (advanced) | Whether the endpoint should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *synchronous* (advanced) | Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported). | false | boolean
|===
// endpoint options: END




== FlinkComponent Options




// component options: START
The Flink component supports 5 options, which are listed below.



[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *dataSetCallback* (producer) | Function performing action against a DataSet. |  | DataSetCallback
| *dataStream* (producer) | DataStream to compute against. |  | DataStream
| *dataStreamCallback* (producer) | Function performing action against a DataStream. |  | DataStreamCallback
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *basicPropertyBinding* (advanced) | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
|===
// component options: END





== Flink DataSet Callback

[source,java]
-----------------------------------
@Bean
public DataSetCallback<Long> dataSetCallback() {
    return new DataSetCallback<Long>() {
        public Long onDataSet(DataSet dataSet, Object... objects) {
            try {
                 dataSet.print();
                 return new Long(0);
            } catch (Exception e) {
                 return new Long(-1);
            }
        }
    };
}
-----------------------------------

== Flink DataStream Callback

[source,java]
---------------------------
@Bean
public VoidDataStreamCallback dataStreamCallback() {
    return new VoidDataStreamCallback() {
        @Override
        public void doOnDataStream(DataStream dataStream, Object... objects) throws Exception {
            dataStream.flatMap(new Splitter()).print();

            environment.execute("data stream test");
        }
    };
}
---------------------------

== Camel-Flink Producer call

[source,java]
-----------------------------------
CamelContext camelContext = new SpringCamelContext(context);

String pattern = "foo";

try {
    ProducerTemplate template = camelContext.createProducerTemplate();
    camelContext.start();
    Long count = template.requestBody("flink:dataSet?dataSet=#myDataSet&dataSetCallback=#countLinesContaining", pattern, Long.class);
    } finally {
        camelContext.stop();
    }
-----------------------------------

include::camel-spring-boot::page$flink-starter.adoc[]