google-mail-stream/google-mail-stream-source/README.adoc

= Camel-Kafka-connector Google Mail Stream Source Example

This is an example for Camel-Kafka-connector Google Mail Stream Source Example

== Standalone

=== What is needed

- A google account
- A google application authorized to deal with gmail API

=== Setting up Google Application

The step-by-step guide is a bit long, so follow it with attention:

1. Create a project
2. Note down clientSecret and clientId
2. Create an OAuth consent screen
3. Create OAuth Client credentials (OAuth for WebApp)
4. Add https://developers.google.com/oauthplayground to Authorized redirect URIs
5. Setup https://developers.google.com/oauthplayground using credentials from above
6. Authorize APIs for Gmail API v1 scope https://mail.google.com
7. Exchange authorization code for tokens
8. Note down Access token & Refresh token
9. Mark some email as unread in your account

We auto-sent an email with the following body "Camel Kafka Connector is great!" and marked it as unread.

=== Running Kafka

[source]
----
$KAFKA_HOME/bin/zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties
$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties
$KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic mytopic
----

=== Download the connector package

Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/`

[source]
----
> cd /home/oscerd/connectors/
> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-google-mail-stream-kafka-connector/0.11.5/camel-google-mail-stream-kafka-connector-0.11.5-package.tar.gz
> untar.gz camel-google-mail-stream-kafka-connector-0.11.5-package.tar.gz
----

=== Configuring Kafka Connect

You'll need to set up the `plugin.path` property in your kafka

Open the `$KAFKA_HOME/config/connect-standalone.properties` and set the `plugin.path` property to your choosen location:

[source]
----
...
plugin.path=/home/oscerd/connectors
...
----

=== Setup the connectors

Open the Git configuration file at `$EXAMPLES/google-mail-stream/google-mail-stream-source/config/CamelGoogleMailStreamSourceConnector.properties`

[source]
----
name=CamelGoogle-mail-streamSourceConnector
connector.class=org.apache.camel.kafkaconnector.googlemailstream.CamelGooglemailstreamSourceConnector
tasks.max=1

key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter

topics=mytopic

camel.source.path.index=ckc
camel.source.endpoint.clientId=<client_id>
camel.source.endpoint.accessToken=<access_token>
camel.source.endpoint.refreshToken=<refresh_token>
camel.source.endpoint.clientSecret=<client_secret>
camel.source.endpoint.applicationName=ckc
camel.source.endpoint.delay=60000
camel.source.endpoint.markAsRead=true
camel.source.endpoint.labels=inbox
camel.source.endpoint.query=is:unread -category:(promotions OR social)
camel.source.endpoint.maxResults=20
----

=== Running the example

Run the kafka connect with the Google Mail Stream Source connector:

[source]
----
$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties $EXAMPLES/google-mail-stream/google-mail-stream-source/config/CamelGoogleMailStreamSourceConnector.properties
----

On a different terminal run the kafkacat consumer

[source]
----
> ./kafkacat -b localhost:9092 -t mytopic
Camel Kafka Connector is great!
% Reached end of topic mytopic [0] at offset 1
----

In the google mail account you should now see the email marked as read.