This example demonstrates how to get started with Camel based Knative sources by showing you some of the most important features that you should know before trying to develop more complex examples. This extends the ideas presented on the AWS Kinesis Source Basic example and shows how to customize the Kinesis client. In this example we show how to configure a custom Kinesis client for testing using a LocalStack instance instead of a real AWS Kinesis instance. Although this example focuses on a general scenario aimed at testing, the overall idea and design is also applicable for scenarios when a custom client configuration is necessary.
You can find more information about Apache Camel and Apache Camel K on the official Camel website.
Read the general instructions in the root README.md file for setting up your environment and the Kubernetes cluster before looking at this example.
Make sure you've read the installation instructions for your specific cluster before starting the example.
You should open this file with Didact if available on your IDE.
You need a Maven repository where the custom client can be deployed.
Validate all Requirements at Once!
Kubectl CLI
The Kubernetes kubectl CLI tool will be used to interact with the Kubernetes cluster.
Check if the Kubectl CLI is installed{.didact}
Connection to a Kubernetes cluster
You need to connect to a Kubernetes cluster in order to run the example.
Check if you're connected to a Kubernetes cluster{.didact}
Apache Camel K CLI (“kamel”)
You need the Apache Camel K CLI (“kamel”) in order to access all Camel K features.
Check if the Apache Camel K CLI (“kamel”) is installed{.didact}
Knative installed on the cluster
The cluster also needs to have Knative installed and working. Refer to the official Knative documentation for information on how to install it in your cluster.
Check if the Knative Serving is installed{.didact}
Check if the Knative Eventing is installed{.didact}
Knative Camel Source installed on the cluster
The cluster also needs to have installed the Knative Camel Source from the camel.yaml in the Eventing Sources release page
Check if the Knative Camel Source is installed{.didact}
The following requirements are optional. They don't prevent the execution of the demo, but may make it easier to follow.
VS Code Extension Pack for Apache Camel
The VS Code Extension Pack for Apache Camel provides a collection of useful tools for Apache Camel K developers, such as code completion and integrated lifecycle management. They are recommended for the tutorial, but they are not required.
You can install it from the VS Code Extensions marketplace.
Check if the VS Code Extension Pack for Apache Camel by Red Hat is installed{.didact}
Let's open a terminal and go to the example directory:
cd 90-aws-kinesis-customized-event-source
(^ execute{.didact})
We're going to create a namespace named aws-kinesis-customized-event-source for running the example. To create it, execute the following command:
kubectl create namespace aws-kinesis-customized-event-source
(^ execute{.didact})
Now we can set the aws-kinesis-customized-event-source namespace as default namespace for the following commands:
kubectl config set-context --current --namespace=aws-kinesis-customized-event-source
(^ execute{.didact})
You need to install Camel K in the aws-kinesis-customized-event-source namespace (or globally in the whole cluster). For this example to run, you must configure Camel K to include your Maven repository in its configuration, so that dependencies can be downloaded from it. To do so, execute the following command replacing https://url.of.the.repository with the actual URL of the repository:
kamel install --maven-repository https://url.of.the.repository
NOTE: The kamel install command requires some prerequisites to be successful in some situations, e.g. you need to enable the registry addon on Minikube. Refer to the Camel K install guide for cluster-specific instructions.
To check that Camel K is installed we'll retrieve the IntegrationPlatform object from the namespace:
kubectl get integrationplatform
(^ execute{.didact})
You should find an IntegrationPlatform in status Ready.
You can now proceed to the next section.
Deploy the custom client to a maven repository, adjusting the command below so that the repository ID and the URL matches the ones used in your Maven repository.
mvn -f extra/custom-kinesis-configuration/pom.xml -DaltDeploymentRepository="id-of-the-repository::default::https://url.of.the.repository" deploy
Deploy the a LocalStack instance for testing:
kubectl apply -f extra/localstack.yaml
(^ execute{.didact})
This repository contains a simple aws-kinesis.properties{.didact} that contains the access key and secret key for accessing the AWS Kinesis stream.
kubectl create secret generic aws-kinesis --from-file=aws-kinesis.properties
(^ execute{.didact})
As the example levareges Knative Eventing channels, we need to create the one that the example will use:
kubectl apply -f aws-kinesis-channel.yaml
(^ execute{.didact})
This repository contains a simple Camel Source based on the AWS Kinesis component that forward streaming events received on the AWS Kinesis stream to a Knative channel named aws-kinesis.
Use the following command to deploy the Camel Source:
kubectl apply -f aws-kinesis-source.yaml
(^ execute{.didact})
You need a producer adding data to a Kinesis stream to try this example. This integration comes with a sample producer that will send 100 messages with the text Hello Camel K every 3 seconds.
kamel run --secret aws-kinesis --dependency mvn:org.apache.camel.k.examples:custom-kinesis-configuration:1.0.3 --property camel.component.aws-kinesis.configuration=#class:org.apache.camel.k.examples.CustomKinesisConfiguration --property amazon.host=localstack:4568 aws-kinesis-producer.groovy
(^ execute{.didact})
If everything is ok, after the build phase finishes, you should see the Camel integration running.
kamel run aws-kinesis-consumer.groovy --dev
(^ execute{.didact})
If everything is ok, after the build phase finishes, you should see the Camel integration running.
To cleanup everything, execute the following command:
kubectl delete namespace aws-kinesis-customized-event-source
(^ execute{.didact})