Google Git
Sign in
apache / bahir / 86ded930e4af769e8191c8f415fe48193dd4914b
commit86ded930e4af769e8191c8f415fe48193dd4914b[log] [tgz]
authorClemens Wolff <clewolff@microsoft.com>Thu May 04 12:52:54 2017 -0700
committerLuciano Resende <lresende@apache.org>Wed May 24 09:42:53 2017 -0700
tree4b615907c77c6e8b1b989562f98f7ee86e0a5e22
parent68ed2d448c91ca2a79697649d8bae7b94c3a05bc [diff]
[BAHIR-117] Expand filtering options for TwitterInputDStream

Adds a new method to TwitterUtils that enables users to pass
an arbitrary FilterQuery down to the TwitterReceiver.

This enables use-cases like receiving Tweets based on location,
based on handle, etc. Previously users were only able to receive
Tweets based on disjunctive keyword queries.

Closes #43.
6 files changed
tree: 4b615907c77c6e8b1b989562f98f7ee86e0a5e22
  1. bin/
  2. dev/
  3. distribution/
  4. sql-cloudant/
  5. sql-streaming-akka/
  6. sql-streaming-mqtt/
  7. streaming-akka/
  8. streaming-mqtt/
  9. streaming-twitter/
  10. streaming-zeromq/
  11. .gitattributes
  12. .gitignore
  13. LICENSE
  14. NOTICE
  15. pom.xml
  16. README.md
  17. scalastyle-config.xml
README.md

Apache Bahir

Apache Bahir provides extensions to distributed analytics platforms such as Apache Spark & Apache Flink.

http://bahir.apache.org/

Apache Bahir origins

The Initial Bahir source code (see issue BAHIR-1) containing the source for the Apache Spark streaming connectors for akka, mqtt, twitter, zeromq extracted from Apache Spark revision 8301fad (before the deletion of the streaming connectors akka, mqtt, twitter, zeromq).

Source code structure

Source code folder structure:

- streaming-akka
  - examples/src/main/...
  - src/main/...
- streaming-mqtt
  - examples
  - src
  - python
- ...

Building Bahir

Bahir is built using Apache Maven. To build Bahir and its example programs, run:

mvn -DskipTests clean install

Running tests

Testing first requires building Bahir. Once Bahir is built, tests can be run using:

mvn test

Example programs

Each extension currently available in Apache Bahir has an example application located under the “examples” folder.

Documentation

Currently, each submodule has its own README.md, with information on example usages and API.

Furthermore, to generate scaladocs for each module:

$ mvn package

Scaladocs is generated in, MODULE_NAME/target/site/scaladocs/index.html. __ Where MODULE_NAME is one of, sql-streaming-mqtt, streaming-akka, streaming-mqtt, streaming-zeromq, streaming-twitter. __

A note about Apache Spark integration

Currently, each module in Bahir is available through spark packages. Please follow linking sub section in module specific README.md for more details.