| == Timer to Elasticsearch Index Sink |
| |
| In this sample you'll use the Elasticsearch Index Sink Kamelet based on camel-elasticsearch component. |
| |
| === Install JBang |
| |
| First install JBang according to https://www.jbang.dev |
| |
| When JBang is installed then you should be able to run from a shell: |
| |
| [source,sh] |
| ---- |
| $ jbang --version |
| ---- |
| |
| This will output the version of JBang. |
| |
| To run this example you can either install Camel on JBang via: |
| |
| [source,sh] |
| ---- |
| $ jbang app install camel@apache/camel |
| ---- |
| |
| Which allows to run CamelJBang with `camel` as shown below. |
| |
| === Setup Elasticsearch |
| |
| We are going to use the official Docker image for Elasticsearch. |
| |
| We can run the following: |
| |
| [source,sh] |
| ---- |
| docker run -d --name elasticsearch --net host -e "discovery.type=single-node" elasticsearch:8.5.3 |
| ---- |
| |
| We should now reset the password: |
| |
| [source,sh] |
| ---- |
| docker exec -it elasticsearch /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic |
| WARNING: Owner of file [/usr/share/elasticsearch/config/users] used to be [root], but now is [elasticsearch] |
| WARNING: Owner of file [/usr/share/elasticsearch/config/users_roles] used to be [root], but now is [elasticsearch] |
| This tool will reset the password of the [elastic] user to an autogenerated value. |
| The password will be printed in the console. |
| Please confirm that you would like to continue [y/N]y |
| |
| |
| Password for the [elastic] user successfully reset. |
| New value: xxxxx |
| ---- |
| |
| Take note of the password, open the `timer-es-index.yaml` file and add the password in substitution of <password> |
| |
| Now we need to download the self-signed certificate to communicate with the Elasticsearch cluster |
| |
| [source,sh] |
| ---- |
| docker cp elasticsearch:/usr/share/elasticsearch/config/certs/http_ca.crt . |
| ---- |
| |
| To verify the connection is fine run the following command: |
| |
| [source,sh] |
| ---- |
| [oscerd@ghost timer-elasticsearch-index]$ curl --cacert http_ca.crt -u elastic https://localhost:9200 |
| Enter host password for user 'elastic': |
| { |
| "name" : "ghost", |
| "cluster_name" : "docker-cluster", |
| "cluster_uuid" : "PWP3KUNARhCqR69yPhLilA", |
| "version" : { |
| "number" : "8.5.3", |
| "build_flavor" : "default", |
| "build_type" : "docker", |
| "build_hash" : "4ed5ee9afac63de92ec98f404ccbed7d3ba9584e", |
| "build_date" : "2022-12-05T18:22:22.226119656Z", |
| "build_snapshot" : false, |
| "lucene_version" : "9.4.2", |
| "minimum_wire_compatibility_version" : "7.17.0", |
| "minimum_index_compatibility_version" : "7.0.0" |
| }, |
| "tagline" : "You Know, for Search" |
| } |
| ---- |
| |
| As you may see you'll be prompted for the elastic user password, inserted it an it should work. |
| |
| We need now to encode the http_ca.crt into base64 to be used in the kamelet |
| |
| [source,sh] |
| ---- |
| [oscerd@ghost timer-elasticsearch-index]$ cat http_ca.crt | base64 -w 0 |
| ---- |
| |
| Take note of the value, open the `timer-es-index.yaml` file and add the value in substitution of <certificate> |
| |
| We should be ready to test our route. |
| |
| === How to run |
| |
| Then you can run this example using: |
| |
| [source,sh] |
| ---- |
| $ jbang run --fresh -Dcamel.jbang.version=3.21.0-SNAPSHOT camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> timer-es-index.yaml |
| ---- |
| |
| === Developer Web Console |
| |
| You can enable the developer console via `--console` flag as show: |
| |
| [source,sh] |
| ---- |
| $ jbang run --fresh -Dcamel.jbang.version=3.21.0-SNAPSHOT camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> timer-es-index.yaml --console |
| ---- |
| |
| Then you can browse: http://localhost:8080/q/dev to introspect the running Camel applicaton. |
| |
| === Running integration |
| |
| We are going to insert just one record into the index. |
| |
| [source,sh] |
| ---- |
| [jbang] Resolving dependencies... |
| [jbang] org.apache.camel:camel-bom:3.21.0-SNAPSHOT@pom |
| [jbang] org.apache.camel:camel-jbang-core:3.21.0-SNAPSHOT |
| [jbang] org.apache.camel.kamelets:camel-kamelets:3.20.0 |
| [jbang] Dependencies resolved |
| [jbang] Building jar... |
| 2023-01-11 10:54:46.232 INFO 71956 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 3.21.0-SNAPSHOT is starting |
| 2023-01-11 10:54:46.447 INFO 71956 --- [ main] org.apache.camel.main.MainSupport : Using Java 11.0.16.1 with PID 71956. Started by oscerd in /home/oscerd/workspace/apache-camel/camel-kamelets-examples/jbang/timer-elasticsearch-index |
| 2023-01-11 10:54:46.462 INFO 71956 --- [ main] mel.cli.connector.LocalCliConnector : Camel CLI enabled (local) |
| 2023-01-11 10:54:48.964 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 3.21.0-SNAPSHOT (timer-es-index) is starting |
| 2023-01-11 10:54:49.300 INFO 71956 --- [ main] .component.es.ElasticsearchProducer : Connecting to the ElasticSearch cluster: docker-cluster |
| 2023-01-11 10:54:49.528 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : Property-placeholders summary |
| 2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] repeatCount=1 |
| 2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] message={"id":"1","message":"Camel Rocks"} |
| 2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] contentType=application/json |
| 2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] local-es=local-es-1 |
| 2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] clusterName=docker-cluster |
| 2023-01-11 10:54:49.529 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] certificate=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZXVENDQTBHZ0F3SUJBZ0lVSEdoMmdJSFVFSkxJUlZyT0NpSHNEMWdad1lRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1BERTZNRGdHQTFVRUF4TXhSV3hoYzNScFkzTmxZWEpqYUNCelpXTjFjbWwwZVNCaGRYUnZMV052Ym1acApaM1Z5WVhScGIyNGdTRlJVVUNCRFFUQWVGdzB5TXpBeE1URXdPVE00TWpsYUZ3MHlOakF4TVRBd09UTTRNamxhCk1Ed3hPakE0QmdOVkJBTVRNVVZzWVhOMGFXTnpaV0Z5WTJnZ2MyVmpkWEpwZEhrZ1lYVjBieTFqYjI1bWFXZDEKY21GMGFXOXVJRWhVVkZBZ1EwRXdnZ0lpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElDRHdBd2dnSUtBb0lDQVFDaQpaQzZENlRrZldFLzZMVWptY2IrbmhXMys3amJvVlBkM2haTlRjc3RsM3VFV0ZTd3BHNW5YUWttMmpIZTc0cFc5Cnd3VFZJeUFPZGpaMFlTRG1Mbmt3UlZqUVlKNDY3Sm5wbnpXd2hvUXpYMjUzRlc4RmVGZDVNdTdpeTZzdGErTUwKN2dzYi9PZ0ZVQnpEYTNhb3BFK3lkN2tNOWpHbXVkZEJQMEtBeHpTZ0FTSUNOR2dqY3NqZ2Y2cTFDUms5Y2ZQQwp5S0tJUjl1TUtVWit3ZHRDNFczc2RqNWgrMWo2QitYaDFoWndCVjlZTytXWU5uYmJnNFlGSHdVR3k5disxZUN4CmxVVjBVUmhwa2dUN3J5UlZCNWpkTlJaYWVLeVQ3c2VlVWI1a25TTXR0WCtuTHIyemU3cnExd3JVQ0k2ZHYvcjAKVWZGUXQyMCtRcUF3TWVGa2NmdGtpWlRYanhWMlN4TmIrNFlxeDlkS3VWaWZaZVVWWGtyaUs0V3NmbE9NNmlObgp6L3oySVBGMmdDR3JPdHZWdTJnc1VReFd0YkJ0aDBBQU5LWUQ5NXN0MTBJbnU3VUVldTlmMUVFbktVQU5oTmNZClY3a3Rka2F6VGdZVEI0dWhKc3hqNm5XTGlBUUwxTDNqNzhuS1pERVV5aHlSY3VUUjJFcTFlNnlpR1BBbmFmUHIKOVhaMTJwMVY0cUE0NVRQc1BqdnFzM2NkSWxiNDRMUTdHUnRqalkxdnVXMkJGanlodW1mMi83N1lGeHpHZUVHbgowcG9xZDZENHQwS1Y4Q3RLVXllM3orcEhIWHpkaXZ2dGNwWnh1RTRybldpSHRQaURpcm5UUWttTEpydWlnWGZHCkVHQnRPZnFRUEtpMWV3TkVpTWlKYnBYOVZUU3JCc3JLK2ZJWXh0V3VUd0lEQVFBQm8xTXdVVEFkQmdOVkhRNEUKRmdRVWpKNTFydGZSNnV6RnpSWVRLUUltNUdRcHI1VXdId1lEVlIwakJCZ3dGb0FVako1MXJ0ZlI2dXpGelJZVApLUUltNUdRcHI1VXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FnRUFBb1JnCm4ybU5KRUQxZWQwbTdBMDBHeFJQMzVWYzBXS2o1M0hoZXRlVHBKeDJDSmZ4THBlNy9tUFlIcC9oSkdnbmIxUDMKWUZJS1ZmV1ZKTG1lV0hVSWlMTjJFVG5VeTFFNTNuSGZiWGY2UWhkK3o4YSsrd1V0aHVxUHBaWTIyOVdCMXVTVwpTT0tUT3BQdEtaRTVIMHkva25XdlhLNzdOb2lpSlhOYmphbXptSU1TRzRmSWxFY1lJKzEyZ3ZmeVpzdjhWNHViCnpEckhmRVV1dUg3QmtBaXN4dWlqcWhRVzZCMjZJdlFrTEJUTEtxOHkzSk5Jbm1OMkxhdlhOc3Q2TWFoSW9xNXUKdUtoWW1zUnVCMHE1QmQ1b1poZVVOeEJKaUc5aUlya0NOMjlCRmtkdHVqc0tkMFd0bkErTUdGSzNyMkFTcGhaeQpIbldZMFV3RXVSR2doMzdDQ3ViYVdIUzJYbTluTldlSEs2NVpDZHNsVGgxcjR4c1JFUjNDRk1IV1lEb1U2UXJWCjNmUkk1eXdkVjlxakVtd1d0NktKaEJnelBQeXQwRGx2U0xkcEcxNmZDMmZ3ZGdERi9ONDJLMmtqSDBFQ2k1bUQKWkdkakxUMk9POFczeVp1WTVENmNOZTNyZitBS09wTFQ4YVQrSXJvcWwxb081b2tLYU9QRkRIQ0dZT20yNlVMawp0YjJSelovWUJrbWpwVGlSNVdRSXA3d1M3SldmNFByRjdmR2NOVWtRN1FqdFg3Sk5Qc29HSThoSmg3R3BsbzFxCmtIbTh6LzRhRnVsdTJJK3BLVksybEFMb0xPc0FuYit2Z0FDV3dNTHlGY1FGVE5uQXpwdUJGZnFEREdVTWY2T0MKWlBPa1pBdHBZRmI2QktMVVR0Y0krTTkwaWtoS0Z0eWhIa3hsbTE0PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== |
| 2023-01-11 10:54:49.530 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] indexName=test_index |
| 2023-01-11 10:54:49.530 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] hostAddresses=localhost |
| 2023-01-11 10:54:49.530 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] password=xxxxxx |
| 2023-01-11 10:54:49.530 INFO 71956 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] user=xxxxxx |
| 2023-01-11 10:54:49.546 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Routes startup (started:3) |
| 2023-01-11 10:54:49.546 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Started route1 (kamelet://timer-source) |
| 2023-01-11 10:54:49.547 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Started timer-source-1 (timer://tick) |
| 2023-01-11 10:54:49.547 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Started elasticsearch-index-sink-2 (kamelet://source) |
| 2023-01-11 10:54:49.547 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 3.21.0-SNAPSHOT (timer-es-index) started in 2s574ms (build:107ms init:1s885ms start:582ms JVM-uptime:4s) |
| ^C2023-01-11 10:55:23.091 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 3.21.0-SNAPSHOT (timer-es-index) is shutting down (timeout:10s) |
| 2023-01-11 10:55:23.104 INFO 71956 --- [ - ShutdownTask] .component.es.ElasticsearchProducer : Disconnecting from ElasticSearch cluster: docker-cluster |
| 2023-01-11 10:55:23.128 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Routes stopped (stopped:3) |
| 2023-01-11 10:55:23.129 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Stopped elasticsearch-index-sink-2 (kamelet://source) |
| 2023-01-11 10:55:23.129 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Stopped timer-source-1 (timer://tick) |
| 2023-01-11 10:55:23.129 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Stopped route1 (kamelet://timer-source) |
| 2023-01-11 10:55:23.136 INFO 71956 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 3.21.0-SNAPSHOT (timer-es-index) shutdown in 44ms (uptime:34s JVM-uptime:37s) |
| 2023-01-11 10:55:23.137 INFO 71956 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 3.21.0-SNAPSHOT shutdown |
| ---- |
| |
| === Verify the sink worked |
| |
| To verify everything is fine we should run the following command |
| |
| [source,sh] |
| ---- |
| $ curl --cacert http_ca.crt -u elastic -X GET https://localhost:9200/test_index/_search |
| Enter host password for user 'elastic': |
| {"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"test_index","_id":"0oZAoIUBi5pH2-mF-Qi5","_score":1.0,"_source":{"id":"1","message":"Camel Rocks"}}]}} |
| ---- |
| |
| === Help and contributions |
| |
| If you hit any problem using Camel or have some feedback, then please |
| https://camel.apache.org/community/support/[let us know]. |
| |
| We also love contributors, so |
| https://camel.apache.org/community/contributing/[get involved] :-) |
| |
| The Camel riders! |