fix oAuth bearer and set openshift container for openshift deployment (#47)
* fix oAuth bearer and set openshift container for openshift deployment
fixes #2872
* Update kafka/README.adoc
Co-authored-by: Peter Palaga <ppalaga@redhat.com>
* Update kafka/README.adoc
Co-authored-by: Peter Palaga <ppalaga@redhat.com>
* Update kafka/README.adoc
Co-authored-by: Peter Palaga <ppalaga@redhat.com>
* Update kafka/README.adoc
Co-authored-by: Peter Palaga <ppalaga@redhat.com>
* remove oAuth2 + Strimzi testcontainers already managed by dev services
Co-authored-by: Peter Palaga <ppalaga@redhat.com>
diff --git a/docs/modules/ROOT/attachments/examples.json b/docs/modules/ROOT/attachments/examples.json
index 51c6a35..6875aa8 100644
--- a/docs/modules/ROOT/attachments/examples.json
+++ b/docs/modules/ROOT/attachments/examples.json
@@ -26,7 +26,7 @@
},
{
"title": "Kafka example ",
- "description": "Shows how to produce and consume messages in a Kafka topic",
+ "description": "Shows how to produce and consume messages in a Kafka topic, using Strimzi Operator",
"link": "https://github.com/apache/camel-quarkus-examples/tree/main/kafka"
},
{
diff --git a/kafka/README.adoc b/kafka/README.adoc
index 9e61b10..51b2034 100644
--- a/kafka/README.adoc
+++ b/kafka/README.adoc
@@ -1,5 +1,5 @@
= Kafka example : A Camel Quarkus example
-:cq-example-description: An example that shows how to produce and consume messages in a Kafka topic
+:cq-example-description: An example that shows how to produce and consume messages in a Kafka topic, using Strimzi Operator
{cq-description}
@@ -9,27 +9,16 @@
== Prerequisites
-The example application requires a running Kafka instance. For simplicity, you can launch the Kafka instance using the docker-compose.yaml.
+The example application requires a running Kafka instance.
-----
-$ cd conf && docker-compose up
-----
-
-Next : uncomment the The section Kafka instance without Authentication in `src/main/resources/application.properties` and set :
-----
-camel.component.kafka.brokers=localhost:9092
-----
-
-If you prefer to use a different Kafka instance uncomment and adjust the corresponding commented section in `src/main/resources/application.properties`.
-
-- The section Kafka instance without Authentication if no Authentication required.
-- The section Kafka instance with SASL Plain is using SASL.
-- The section Kafka instance with SASL Oauth Bearer if using Oauth Bearer.
+For simplicity, this example uses Quarkus DevServices in dev mode. It will automatically start Strimzi container for your tests, and you don’t even need to configure anything: the container will be automatically wired to the Quarkus dev mode configuration.
== Start in Development mode
Run the application in development mode.
+TIP: If you want to use another running instance, in dev mode. Uncomment the corresponding Kafka configuration section in `src/main/resources/application.properties` and change `%prod` profile to `%dev`.
+
[source,shell]
----
$ mvn clean compile quarkus:dev
@@ -64,13 +53,48 @@
----
-=== Package and run the application
+=== Configure Kafka client, package and run the application
-Once you are done with developing you may want to package and run the application.
+Once you are done with developing you may want to configure your kafka client, package and run the application.
TIP: Find more details about the JVM mode and Native mode in the Package and run section of
https://camel.apache.org/camel-quarkus/latest/first-steps.html#_package_and_run_the_application[Camel Quarkus User guide]
+==== Configure kafka client
+Uncomment the corresponding commented section in `src/main/resources/application.properties`.
+
+- The section Kafka instance without Authentication if no Authentication required.
+- The section Kafka instance with SASL Plain if using SASL.
+- The section Kafka instance with SASL Oauth Bearer if using Oauth Bearer.
+
+You need to set the corresponding environment variables:
+- Without Authentication
+[source,shell]
+----
+$ export brokers=<YOUR_KAFKA_BROKERS_URL>
+----
+- SASL Plain
+[source,shell]
+----
+$ export brokers=<YOUR_KAFKA_BROKERS_URL>
+$ export id=<YOUR_KAFKA_SASL_CLIENT_ID>
+$ export secret=<YOUR_KAFKA_SASL_CLIENT_SECRET>
+----
+-SASL Oauth Bearer
+[source,shell]
+----
+$ export brokers=<YOUR_KAFKA_BROKERS_URL>
+$ export id=<YOUR_KAFKA_SASL_CLIENT_ID>
+$ export secret=<YOUR_KAFKA_SASL_CLIENT_SECRET>
+$ export token=<YOUR_KAFKA_SASL_OAUTHBEARER_TOKEN_URL>
+----
+
+If you want to deploy on Kubernetes or Openshift, you'd need to define those in a secret named `camel-kafka`. Set the needed values in the `kubefiles/secret-example.yml`, then add the secret :
+[source,shell]
+----
+$ kubectl apply -f kubefiles/secret-example.yml
+----
+
==== JVM mode
[source,shell]
@@ -96,18 +120,38 @@
You can build a container image for the application like this. Refer to the https://quarkus.io/guides/deploying-to-kubernetes[Quarkus Kubernetes guide] for options around customizing image names, registries etc.
-Uncomment the container build section. Set the proper image group.
+This example uses Jib to create the container image for Kubernetes deployment.
+
+Uncomment the creating container with jib and secrets, in the Kubernetes specific section in `src/main/resources/application.properties`. Set image group and image registry.
+
+Build the application using the `kubernetes` profile.
[source,shell]
----
-$ mvn clean package -DskipTests
+$ mvn clean package -DskipTests -Dkubernetes
+----
+
+The `kubernetes` profile uses quarkus kubernetes and jib container extensions, as described in the `pom.xml`.
+
+[source,shell]
+----
+<dependencies>
+ <dependency>
+ <groupId>io.quarkus</groupId>
+ <artifactId>quarkus-kubernetes</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>io.quarkus</groupId>
+ <artifactId>quarkus-container-image-jib</artifactId>
+ </dependency>
+</dependencies>
----
If you are using a local development cluster like Kind or k3s, you can use host the container image on your local host. Or, with minikube, use the Docker daemon from the cluster virtual machine `eval $(minikube docker-env)`. Otherwise, you'll need to push the image to a registry of your choosing.
-TIP: You can build & deploy in one single step by doing `mvn clean package -DskipTests -Dquarkus.kubernetes.deploy=true`
+TIP: You can build & deploy in one single step by doing `mvn clean package -DskipTests -Dkubernetes -Dquarkus.kubernetes.deploy=true`
-Check pods are running.
+Check that the pods are running.
Example when using Strimzi operator, with a Kafka instance named `Test` :
@@ -138,6 +182,7 @@
[source,shell]
----
$ kubectl delete all -l app.kubernetes.io/name=camel-quarkus-examples-kafka
+$ kubectl delete secret camel-kafka
----
[NOTE]
@@ -145,13 +190,33 @@
If you need to configure container resource limits & requests, or enable the Quarkus Kubernetes client to trust self signed certificates, you can find these configuration options in `src/main/resources/application.properties`. Simply uncomment them and set your desired values.
====
+
==== Deploying to OpenShift
+Uncomment the creating container with openshift and secrets, in the Openshift specific section in `src/main/resources/application.properties`.
+
+
[source,shell]
----
$ mvn clean package -DskipTests -Dquarkus.kubernetes.deploy=true -Dopenshift
----
+The `openshift` profile uses quarkus openshift and openshift-container extensions, as described in the `pom.xml`.
+
+[source,shell]
+----
+<dependencies>
+ <dependency>
+ <groupId>io.quarkus</groupId>
+ <artifactId>quarkus-openshift</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>io.quarkus</groupId>
+ <artifactId>quarkus-container-image-openshift</artifactId>
+ </dependency>
+</dependencies>
+----
+
You can check the pod status and tail logs using the commands mentioned above in the Kubernetes section. Use the `oc` binary instead of `kubectl` if preferred.
== Feedback
diff --git a/kafka/conf/docker-compose.yaml b/kafka/conf/docker-compose.yaml
deleted file mode 100644
index de99cae..0000000
--- a/kafka/conf/docker-compose.yaml
+++ /dev/null
@@ -1,47 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-version: '2'
-
-services:
-
- zookeeper:
- image: quay.io/strimzi/kafka:0.21.1-kafka-2.7.0
- command: [
- "sh", "-c",
- "bin/zookeeper-server-start.sh config/zookeeper.properties"
- ]
- ports:
- - "2181:2181"
- environment:
- LOG_DIR: /tmp/logs
-
- kafka:
- image: quay.io/strimzi/kafka:0.21.1-kafka-2.7.0
- command: [
- "sh", "-c",
- "bin/kafka-server-start.sh config/server.properties --override listeners=$${KAFKA_LISTENERS} --override advertised.listeners=$${KAFKA_ADVERTISED_LISTENERS} --override zookeeper.connect=$${KAFKA_ZOOKEEPER_CONNECT}"
- ]
- depends_on:
- - zookeeper
- ports:
- - "9092:9092"
- environment:
- LOG_DIR: "/tmp/logs"
- KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
- KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
- KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
\ No newline at end of file
diff --git a/kafka/kubefiles/secret-example.yml b/kafka/kubefiles/secret-example.yml
new file mode 100644
index 0000000..72499b2
--- /dev/null
+++ b/kafka/kubefiles/secret-example.yml
@@ -0,0 +1,28 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+apiVersion: v1
+kind: Secret
+metadata:
+ name: camel-kafka
+ namespace: test
+type: Opaque
+stringData:
+ brokers: "<YOUR_KAFKA_BROKERS_URL>"
+ id: "<YOUR_KAFKA_SASL_CLIENT_ID>"
+ secret: "<YOUR_KAFKA_SASL_CLIENT_SECRET>"
+ token: "<YOUR_KAFKA_SASL_OAUTHBEARER_TOKEN_URL>"
\ No newline at end of file
diff --git a/kafka/pom.xml b/kafka/pom.xml
index 405bf17..7cc2742 100644
--- a/kafka/pom.xml
+++ b/kafka/pom.xml
@@ -45,7 +45,7 @@
<maven-resources-plugin.version>3.1.0</maven-resources-plugin.version>
<maven-surefire-plugin.version>2.22.2</maven-surefire-plugin.version>
<mycila-license.version>3.0</mycila-license.version>
- <strimzi.testcontainers.version>0.20.1</strimzi.testcontainers.version>
+ <kafka-oauth-client.version>0.7.2</kafka-oauth-client.version>
</properties>
<dependencyManagement>
@@ -65,6 +65,11 @@
<type>pom</type>
<scope>import</scope>
</dependency>
+ <dependency>
+ <groupId>io.strimzi</groupId>
+ <artifactId>kafka-oauth-client</artifactId>
+ <version>${kafka-oauth-client.version}</version>
+ </dependency>
</dependencies>
</dependencyManagement>
@@ -95,11 +100,13 @@
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
- <artifactId>quarkus-kubernetes</artifactId>
+ <artifactId>quarkus-kubernetes-config</artifactId>
</dependency>
+
+ <!-- For oauth use case -->
<dependency>
- <groupId>io.quarkus</groupId>
- <artifactId>quarkus-container-image-docker</artifactId>
+ <groupId>io.strimzi</groupId>
+ <artifactId>kafka-oauth-client</artifactId>
</dependency>
<!-- Test -->
@@ -123,12 +130,6 @@
<artifactId>testcontainers</artifactId>
<scope>test</scope>
</dependency>
- <dependency>
- <groupId>io.strimzi</groupId>
- <artifactId>test-container</artifactId>
- <version>${strimzi.testcontainers.version}</version>
- <scope>test</scope>
- </dependency>
</dependencies>
<build>
@@ -305,7 +306,6 @@
<profile>
<id>kubernetes</id>
<activation>
- <activeByDefault>true</activeByDefault>
<property>
<name>kubernetes</name>
</property>
@@ -315,6 +315,10 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes</artifactId>
</dependency>
+ <dependency>
+ <groupId>io.quarkus</groupId>
+ <artifactId>quarkus-container-image-jib</artifactId>
+ </dependency>
</dependencies>
</profile>
<profile>
@@ -329,6 +333,10 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-openshift</artifactId>
</dependency>
+ <dependency>
+ <groupId>io.quarkus</groupId>
+ <artifactId>quarkus-container-image-openshift</artifactId>
+ </dependency>
</dependencies>
</profile>
</profiles>
diff --git a/kafka/src/main/resources/application.properties b/kafka/src/main/resources/application.properties
index 12c8d81..3d39dd6 100644
--- a/kafka/src/main/resources/application.properties
+++ b/kafka/src/main/resources/application.properties
@@ -23,52 +23,69 @@
timer.delay = 10000
#uncomment to set Kafka instance without Authentication
-#camel.component.kafka.brokers=<YOUR-KAFKA-BOOTSTRAP-URL-HERE>
+#camel.component.kafka.brokers=${brokers}
# uncomment to set Kafka instance with SASL Plain
-#camel.component.kafka.brokers=<YOUR-KAFKA-BOOTSTRAP-URL-HERE>
+#camel.component.kafka.brokers=${brokers}
#camel.component.kafka.security-protocol=SASL_SSL
#camel.component.kafka.sasl-mechanism=PLAIN
-#camel.component.kafka.sasl-jaas-config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<YOUR-SERVICE-ACCOUNT-ID-HERE>" password="<YOUR-SERVICE-ACCOUNT-SECRET-HERE>";
+#camel.component.kafka.sasl-jaas-config=org.apache.kafka.common.security.plain.PlainLoginModule required username="${id}" password="${secret}";
# uncomment to set Kafka instance with SASL Oauth Bearer
-#camel.component.kafka.brokers = <YOUR-KAFKA-BOOTSTRAP-URL-HERE>
+#camel.component.kafka.brokers = ${brokers}
#camel.component.kafka.security-protocol = SASL_SSL
#camel.component.kafka.sasl-mechanism = OAUTHBEARER
#camel.component.kafka.sasl-jaas-config = org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
-# oauth.client.id=<YOUR-SERVICE-ACCOUNT-ID-HERE> \
-# oauth.client.secret=<YOUR-SERVICE-ACCOUNT-SECRET-HERE> \
-# oauth.token.endpoint.uri="<TOKEN_ENDPOINT_URI>" ;
+# oauth.client.id="${id}" \
+# oauth.client.secret="${secret}" \
+# oauth.token.endpoint.uri="${token}" ;
#camel.component.kafka.additional-properties[sasl.login.callback.handler.class] = io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler
-# Kubernetes
+###################################
+# Kubernetes specific
+###################################
+# secrets
+#quarkus.kubernetes-config.enabled=true
+#getting secrets while deploying to kubernetes
+#quarkus.kubernetes-config.namespace=test
+#quarkus.kubernetes-config.secrets.enabled=true
+#quarkus.kubernetes-config.secrets=camel-kafka
+
+# creating container with jib
+#quarkus.container-image.build=true
+#quarkus.kubernetes.deployment-target=kubernetes
+#quarkus.container-image.group=<YOUR_IMAGE_GROUP>
+#quarkus.container-image.registry=<YOUR_REGISTRY_URL>
# Uncomment to trust self signed certificates if they are presented by the Kubernetes API server
#quarkus.kubernetes-client.trust-certs=true
-quarkus.kubernetes.image-pull-policy=IfNotPresent
-
# Uncomment to set resource limits
#quarkus.kubernetes.resources.requests.memory=64Mi
#quarkus.kubernetes.resources.requests.cpu=250m
#quarkus.kubernetes.resources.limits.memory=512Mi
#quarkus.kubernetes.resources.limits.cpu=1000m
-# OpenShift
-quarkus.openshift.image-pull-policy=IfNotPresent
+###################################
+# OpenShift specific
+###################################
+# secrets
+#quarkus.kubernetes-config.enabled=true
+#getting secrets while deploying to kubernetes
+#quarkus.kubernetes-config.namespace=test
+#quarkus.kubernetes-config.secrets.enabled=true
+#quarkus.kubernetes-config.secrets=camel-kafka
+# creating container for openshift
+#quarkus.container-image.build=true
+#quarkus.kubernetes.deployment-target=openshift
+
+# OpenShift
+#quarkus.openshift.image-pull-policy=IfNotPresent
# Uncomment to set resource limits
#quarkus.openshift.resources.requests.memory=64Mi
#quarkus.openshift.resources.requests.cpu=250m
#quarkus.openshift.resources.limits.memory=512Mi
#quarkus.openshift.resources.limits.cpu=1000m
-
-
-#uncomment to set container build
-#quarkus.container-image.builder=docker
-#quarkus.kubernetes.deployment-target=kubernetes
-#quarkus.container-image.push=true
-#quarkus.container-image.group=<YOUR_IMAGE_GROUP>
-
diff --git a/kafka/src/test/java/org/apache/camel/example/KafkaTest.java b/kafka/src/test/java/org/apache/camel/example/KafkaTest.java
index 235ea9b..bc5cf9f 100644
--- a/kafka/src/test/java/org/apache/camel/example/KafkaTest.java
+++ b/kafka/src/test/java/org/apache/camel/example/KafkaTest.java
@@ -16,22 +16,22 @@
*/
package org.apache.camel.example;
-import io.quarkus.test.common.QuarkusTestResource;
+import java.util.concurrent.TimeUnit;
+
import io.quarkus.test.junit.QuarkusTest;
import io.restassured.RestAssured;
import org.junit.jupiter.api.Test;
-import static org.hamcrest.Matchers.containsString;
+import static org.awaitility.Awaitility.await;
@QuarkusTest
-@QuarkusTestResource(StrimziTestResource.class)
public class KafkaTest {
@Test
public void testKafka() {
- RestAssured.get("/example")
- .then()
- .statusCode(200)
- .body(containsString("Message #"));
+ await().atMost(10, TimeUnit.SECONDS).until(() -> {
+ String message = RestAssured.get("/example").asString();
+ return message != null && message.contains("Message #");
+ });
}
}
diff --git a/kafka/src/test/java/org/apache/camel/example/StrimziTestResource.java b/kafka/src/test/java/org/apache/camel/example/StrimziTestResource.java
deleted file mode 100644
index 12e0552..0000000
--- a/kafka/src/test/java/org/apache/camel/example/StrimziTestResource.java
+++ /dev/null
@@ -1,49 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.camel.example;
-
-import java.util.Map;
-
-import io.quarkus.test.common.QuarkusTestResourceLifecycleManager;
-import io.strimzi.StrimziKafkaContainer;
-import org.apache.camel.util.CollectionHelper;
-
-public class StrimziTestResource implements QuarkusTestResourceLifecycleManager {
-
- private StrimziKafkaContainer strimziKafkaContainer;
- private static final int KAFKA_PORT = 9092;
- private static final String KAFKA_STRIMZI_VERSION = "0.20.1-kafka-2.5.0";
-
- @Override
- public Map<String, String> start() {
- strimziKafkaContainer = new StrimziKafkaContainer(KAFKA_STRIMZI_VERSION);
- strimziKafkaContainer.start();
-
- String bootstrap_servers = strimziKafkaContainer.getContainerIpAddress() + ":"
- + strimziKafkaContainer.getMappedPort(KAFKA_PORT);
-
- return CollectionHelper.mapOf(
- "camel.component.kafka.brokers", bootstrap_servers);
- }
-
- @Override
- public void stop() {
- if (strimziKafkaContainer != null) {
- strimziKafkaContainer.stop();
- }
- }
-}
