blob: b8f73e6f0913f913139dfa77d48d72bbe160c43f [file] [log] [blame]
== Timer to Elasticsearch Index Sink
In this sample you'll use the Elasticsearch Index Sink Kamelet based on camel-elasticsearch component.
=== Install JBang
First install JBang according to https://www.jbang.dev
When JBang is installed then you should be able to run from a shell:
[source,sh]
----
$ jbang --version
----
This will output the version of JBang.
To run this example you can either install Camel on JBang via:
[source,sh]
----
$ jbang app install camel@apache/camel
----
Which allows to run CamelJBang with `camel` as shown below.
=== Setup Elasticsearch
We are going to use the official Docker image for Elasticsearch.
We can run the following:
[source,sh]
----
docker run -d --name elasticsearch --net host -e "discovery.type=single-node" elasticsearch:8.5.3
----
We should now reset the password:
[source,sh]
----
docker exec -it elasticsearch /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
WARNING: Owner of file [/usr/share/elasticsearch/config/users] used to be [root], but now is [elasticsearch]
WARNING: Owner of file [/usr/share/elasticsearch/config/users_roles] used to be [root], but now is [elasticsearch]
This tool will reset the password of the [elastic] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y
Password for the [elastic] user successfully reset.
New value: xxxxx
----
Take note of the password, open the `timer-es-index.yaml` file and add the password in substitution of <password>
Now we need to download the self-signed certificate to communicate with the Elasticsearch cluster
[source,sh]
----
docker cp elasticsearch:/usr/share/elasticsearch/config/certs/http_ca.crt .
----
To verify the connection is fine run the following command:
[source,sh]
----
[oscerd@ghost timer-elasticsearch-index]$ curl --cacert http_ca.crt -u elastic https://localhost:9200
Enter host password for user 'elastic':
{
"name" : "ghost",
"cluster_name" : "docker-cluster",
"cluster_uuid" : "PWP3KUNARhCqR69yPhLilA",
"version" : {
"number" : "8.5.3",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "4ed5ee9afac63de92ec98f404ccbed7d3ba9584e",
"build_date" : "2022-12-05T18:22:22.226119656Z",
"build_snapshot" : false,
"lucene_version" : "9.4.2",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}
----
As you may see you'll be prompted for the elastic user password, inserted it an it should work.
We need now to encode the http_ca.crt into base64 to be used in the kamelet
[source,sh]
----
[oscerd@ghost timer-elasticsearch-index]$ cat http_ca.crt | base64 -w 0
----
Take note of the value, open the `timer-es-index.yaml` file and add the value in substitution of <certificate>
We should be ready to test our route.
=== How to run
Then you can run this example using:
[source,sh]
----
$ jbang run --fresh -Dcamel.jbang.version=3.21.0-SNAPSHOT camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> timer-es-index.yaml
----
=== Developer Web Console
You can enable the developer console via `--console` flag as show:
[source,sh]
----
$ jbang run --fresh -Dcamel.jbang.version=3.21.0-SNAPSHOT camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> timer-es-index.yaml --console
----
Then you can browse: http://localhost:8080/q/dev to introspect the running Camel applicaton.
=== Running integration
We are going to insert just one record into the index.
[source,sh]
----
[jbang] Resolving dependencies...
[jbang] org.apache.camel:camel-bom:3.21.0-SNAPSHOT@pom
[jbang] org.apache.camel:camel-jbang-core:3.21.0-SNAPSHOT
[jbang] org.apache.camel.kamelets:camel-kamelets:3.20.0
[jbang] Dependencies resolved
[jbang] Building jar...
2023-01-11 10:54:46.232 INFO 71956 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 3.21.0-SNAPSHOT is starting
2023-01-11 10:54:46.447 INFO 71956 --- [ main] org.apache.camel.main.MainSupport : Using Java 11.0.16.1 with PID 71956. Started by oscerd in /home/oscerd/workspace/apache-camel/camel-kamelets-examples/jbang/timer-elasticsearch-index
2023-01-11 10:54:46.462 INFO 71956 --- [ main] mel.cli.connector.LocalCliConnector : Camel CLI enabled (local)
2023-01-11 10:54:48.964 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 3.21.0-SNAPSHOT (timer-es-index) is starting
2023-01-11 10:54:49.300 INFO 71956 --- [ main] .component.es.ElasticsearchProducer : Connecting to the ElasticSearch cluster: docker-cluster
2023-01-11 10:54:49.528 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : Property-placeholders summary
2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] repeatCount=1
2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] message={"id":"1","message":"Camel Rocks"}
2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] contentType=application/json
2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] local-es=local-es-1
2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] clusterName=docker-cluster
2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] certificate=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZXVENDQTBHZ0F3SUJBZ0lVSEdoMmdJSFVFSkxJUlZyT0NpSHNEMWdad1lRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1BERTZNRGdHQTFVRUF4TXhSV3hoYzNScFkzTmxZWEpqYUNCelpXTjFjbWwwZVNCaGRYUnZMV052Ym1acApaM1Z5WVhScGIyNGdTRlJVVUNCRFFUQWVGdzB5TXpBeE1URXdPVE00TWpsYUZ3MHlOakF4TVRBd09UTTRNamxhCk1Ed3hPakE0QmdOVkJBTVRNVVZzWVhOMGFXTnpaV0Z5WTJnZ2MyVmpkWEpwZEhrZ1lYVjBieTFqYjI1bWFXZDEKY21GMGFXOXVJRWhVVkZBZ1EwRXdnZ0lpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElDRHdBd2dnSUtBb0lDQVFDaQpaQzZENlRrZldFLzZMVWptY2IrbmhXMys3amJvVlBkM2haTlRjc3RsM3VFV0ZTd3BHNW5YUWttMmpIZTc0cFc5Cnd3VFZJeUFPZGpaMFlTRG1Mbmt3UlZqUVlKNDY3Sm5wbnpXd2hvUXpYMjUzRlc4RmVGZDVNdTdpeTZzdGErTUwKN2dzYi9PZ0ZVQnpEYTNhb3BFK3lkN2tNOWpHbXVkZEJQMEtBeHpTZ0FTSUNOR2dqY3NqZ2Y2cTFDUms5Y2ZQQwp5S0tJUjl1TUtVWit3ZHRDNFczc2RqNWgrMWo2QitYaDFoWndCVjlZTytXWU5uYmJnNFlGSHdVR3k5disxZUN4CmxVVjBVUmhwa2dUN3J5UlZCNWpkTlJaYWVLeVQ3c2VlVWI1a25TTXR0WCtuTHIyemU3cnExd3JVQ0k2ZHYvcjAKVWZGUXQyMCtRcUF3TWVGa2NmdGtpWlRYanhWMlN4TmIrNFlxeDlkS3VWaWZaZVVWWGtyaUs0V3NmbE9NNmlObgp6L3oySVBGMmdDR3JPdHZWdTJnc1VReFd0YkJ0aDBBQU5LWUQ5NXN0MTBJbnU3VUVldTlmMUVFbktVQU5oTmNZClY3a3Rka2F6VGdZVEI0dWhKc3hqNm5XTGlBUUwxTDNqNzhuS1pERVV5aHlSY3VUUjJFcTFlNnlpR1BBbmFmUHIKOVhaMTJwMVY0cUE0NVRQc1BqdnFzM2NkSWxiNDRMUTdHUnRqalkxdnVXMkJGanlodW1mMi83N1lGeHpHZUVHbgowcG9xZDZENHQwS1Y4Q3RLVXllM3orcEhIWHpkaXZ2dGNwWnh1RTRybldpSHRQaURpcm5UUWttTEpydWlnWGZHCkVHQnRPZnFRUEtpMWV3TkVpTWlKYnBYOVZUU3JCc3JLK2ZJWXh0V3VUd0lEQVFBQm8xTXdVVEFkQmdOVkhRNEUKRmdRVWpKNTFydGZSNnV6RnpSWVRLUUltNUdRcHI1VXdId1lEVlIwakJCZ3dGb0FVako1MXJ0ZlI2dXpGelJZVApLUUltNUdRcHI1VXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FnRUFBb1JnCm4ybU5KRUQxZWQwbTdBMDBHeFJQMzVWYzBXS2o1M0hoZXRlVHBKeDJDSmZ4THBlNy9tUFlIcC9oSkdnbmIxUDMKWUZJS1ZmV1ZKTG1lV0hVSWlMTjJFVG5VeTFFNTNuSGZiWGY2UWhkK3o4YSsrd1V0aHVxUHBaWTIyOVdCMXVTVwpTT0tUT3BQdEtaRTVIMHkva25XdlhLNzdOb2lpSlhOYmphbXptSU1TRzRmSWxFY1lJKzEyZ3ZmeVpzdjhWNHViCnpEckhmRVV1dUg3QmtBaXN4dWlqcWhRVzZCMjZJdlFrTEJUTEtxOHkzSk5Jbm1OMkxhdlhOc3Q2TWFoSW9xNXUKdUtoWW1zUnVCMHE1QmQ1b1poZVVOeEJKaUc5aUlya0NOMjlCRmtkdHVqc0tkMFd0bkErTUdGSzNyMkFTcGhaeQpIbldZMFV3RXVSR2doMzdDQ3ViYVdIUzJYbTluTldlSEs2NVpDZHNsVGgxcjR4c1JFUjNDRk1IV1lEb1U2UXJWCjNmUkk1eXdkVjlxakVtd1d0NktKaEJnelBQeXQwRGx2U0xkcEcxNmZDMmZ3ZGdERi9ONDJLMmtqSDBFQ2k1bUQKWkdkakxUMk9POFczeVp1WTVENmNOZTNyZitBS09wTFQ4YVQrSXJvcWwxb081b2tLYU9QRkRIQ0dZT20yNlVMawp0YjJSelovWUJrbWpwVGlSNVdRSXA3d1M3SldmNFByRjdmR2NOVWtRN1FqdFg3Sk5Qc29HSThoSmg3R3BsbzFxCmtIbTh6LzRhRnVsdTJJK3BLVksybEFMb0xPc0FuYit2Z0FDV3dNTHlGY1FGVE5uQXpwdUJGZnFEREdVTWY2T0MKWlBPa1pBdHBZRmI2QktMVVR0Y0krTTkwaWtoS0Z0eWhIa3hsbTE0PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
2023-01-11 10:54:49.530 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] indexName=test_index
2023-01-11 10:54:49.530 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] hostAddresses=localhost
2023-01-11 10:54:49.530 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] password=xxxxxx
2023-01-11 10:54:49.530 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] user=xxxxxx
2023-01-11 10:54:49.546 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Routes startup (started:3)
2023-01-11 10:54:49.546 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Started route1 (kamelet://timer-source)
2023-01-11 10:54:49.547 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Started timer-source-1 (timer://tick)
2023-01-11 10:54:49.547 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Started elasticsearch-index-sink-2 (kamelet://source)
2023-01-11 10:54:49.547 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 3.21.0-SNAPSHOT (timer-es-index) started in 2s574ms (build:107ms init:1s885ms start:582ms JVM-uptime:4s)
^C2023-01-11 10:55:23.091 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 3.21.0-SNAPSHOT (timer-es-index) is shutting down (timeout:10s)
2023-01-11 10:55:23.104 INFO 71956 --- [ - ShutdownTask] .component.es.ElasticsearchProducer : Disconnecting from ElasticSearch cluster: docker-cluster
2023-01-11 10:55:23.128 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Routes stopped (stopped:3)
2023-01-11 10:55:23.129 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Stopped elasticsearch-index-sink-2 (kamelet://source)
2023-01-11 10:55:23.129 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Stopped timer-source-1 (timer://tick)
2023-01-11 10:55:23.129 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Stopped route1 (kamelet://timer-source)
2023-01-11 10:55:23.136 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 3.21.0-SNAPSHOT (timer-es-index) shutdown in 44ms (uptime:34s JVM-uptime:37s)
2023-01-11 10:55:23.137 INFO 71956 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 3.21.0-SNAPSHOT shutdown
----
=== Verify the sink worked
To verify everything is fine we should run the following command
[source,sh]
----
$ curl --cacert http_ca.crt -u elastic -X GET https://localhost:9200/test_index/_search
Enter host password for user 'elastic':
{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"test_index","_id":"0oZAoIUBi5pH2-mF-Qi5","_score":1.0,"_source":{"id":"1","message":"Camel Rocks"}}]}}
----
=== Help and contributions
If you hit any problem using Camel or have some feedback, then please
https://camel.apache.org/community/support/[let us know].
We also love contributors, so
https://camel.apache.org/community/contributing/[get involved] :-)
The Camel riders!