Google Git
Sign in
apache / incubator-toree / v0.1.2
tag16919853d79b35534caab332a6e95786e1d25c45
taggerChip Senkbeil <rcsenkbe@us.ibm.com>Fri May 08 15:04:59 2015 -0500
object9fa97c821d7ac76ed7ace6f3c2648ede9e45e4d3 
Final version of 0.1.2 (Apache Spark 1.2.1) before updating to Spark 1.3.1
commit9fa97c821d7ac76ed7ace6f3c2648ede9e45e4d3[log] [tgz]
authorChip Senkbeil <chip.senkbeil@gmail.com>Thu May 07 12:04:36 2015 -0500
committerChip Senkbeil <chip.senkbeil@gmail.com>Thu May 07 12:04:36 2015 -0500
tree189b3eaa43860f3b1554b93fac0dec87020bcd70
parent119911ba00c52afdadb0404ac1fc140bd5e0ff64 [diff]
parent0c0129ff2a94efe5c2954e8bc8d7f1ccfb96c379 [diff]
Merge pull request #107 from ibm-et/ParallelizeTaskManager

Parallelize task manager
tree: 189b3eaa43860f3b1554b93fac0dec87020bcd70
  1. client/
  2. kernel/
  3. kernel-api/
  4. macros/
  5. project/
  6. protocol/
  7. resources/
  8. src/
  9. .gitignore
  10. .pairs
  11. .travis.yml
  12. Dockerfile
  13. LICENSE
  14. Makefile
  15. NOTICE
  16. README.md
  17. Vagrantfile
README.md

Spark Kernel

Build Status Coverage Status Scaladoc License

The Spark Kernel has one main goal: provide the foundation for interactive applications to connect and use Apache Spark.

Overview

The kernel provides several key features for applications:

  1. Define and run Spark Tasks

    • Executing Scala code dynamically in a similar fashion to the Scala REPL and Spark Shell

  2. Collect Results without a Datastore

    • Send execution results and streaming data back via the Spark Kernel to your applications

    • Use the Comm API - an abstraction of the IPython protocol - for more detailed data communication and synchronization between your applications and the Spark Kernel

  3. Host and Manage Applications Separately from Apache Spark

    • The Spark Kernel serves as a proxy for requests to the Apache Spark cluster

The project intends to provide applications with the ability to send both packaged jars and code snippets. As it implements the latest IPython message protocol (5.0), the Spark Kernel can easily plug into the 3.x branch of IPython for quick, interactive data exploration. The Spark Kernel strives to be extensible, providing a pluggable interface for developers to add their own functionality.

If you are new to the Spark Kernel, please see the Getting Started section.

For more information, please visit the Spark Kernel wiki.

For bug reporting and feature requests, please visit the Spark Kernel issue list.