commit | 6ea42a8965d98773a342ad5cd31aab6b64e9d9bd | [log] [tgz] |
---|---|---|
author | Esteban Laver <emlaver@us.ibm.com> | Sun Dec 17 13:30:04 2017 -0500 |
committer | Luciano Resende <lresende@apache.org> | Fri Jan 26 07:43:07 2018 -0800 |
tree | 74384a098445185c5ca9e50567708fb481c5c408 | |
parent | 785ee1e1acfb129bb0524d79df3372968b9e95a7 [diff] |
[BAHIR-154] Refactor sql-cloudant to use cloudant-client library - Use java-cloudant’s executeRequest for HTTP requests against _all_docs endpoint - Added HTTP 429 backoff with default settings - Simplified caught exception and message for schema size - Replaced scala http library with okhttp library for changes receiver - Updated streaming CloudantReceiver class to use improved ChangesRowScanner method - Replaced Play JSON with GSON library - Updated save operation to use java-cloudant bulk API - Use _changes feed filter option for Cloudant/CouchDB 2.x and greater Closes #61
Apache Bahir provides extensions to distributed analytics platforms such as Apache Spark & Apache Flink.
The Initial Bahir source code (see issue BAHIR-1) containing the source for the Apache Spark streaming connectors for akka, mqtt, twitter, zeromq extracted from Apache Spark revision 8301fad (before the deletion of the streaming connectors akka, mqtt, twitter, zeromq).
Source code folder structure:
- streaming-akka - examples/src/main/... - src/main/... - streaming-mqtt - examples - src - python - ...
Bahir is built using Apache Maven. To build Bahir and its example programs, run:
mvn -DskipTests clean install
Testing first requires building Bahir. Once Bahir is built, tests can be run using:
mvn test
Each extension currently available in Apache Bahir has an example application located under the “examples” folder.
Currently, each submodule has its own README.md, with information on example usages and API.
Furthermore, to generate scaladocs for each module:
$ mvn package
Scaladocs is generated in, MODULE_NAME/target/site/scaladocs/index.html
. __ Where MODULE_NAME
is one of, sql-streaming-mqtt
, streaming-akka
, streaming-mqtt
, streaming-zeromq
, streaming-twitter
. __
Currently, each module in Bahir is available through spark packages. Please follow linking sub section in module specific README.md for more details.