| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> |
| <meta name="description" content="Livy is an open source REST service for Apache Spark"> |
| <meta name="author" content="Cloudera"> |
| |
| |
| <title>Livy for Apache Spark Overview</title> |
| |
| <!-- Bootstrap core CSS --> |
| <link href="css/bootstrap.min.css" rel="stylesheet"> |
| |
| <!-- Custom styles for this template --> |
| <link href="css/justified-nav.css" rel="stylesheet"> |
| |
| |
| |
| <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> |
| <!--[if lt IE 9]> |
| <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> |
| <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| <![endif]--> |
| </head> |
| |
| <body> |
| <div class="container"> |
| |
| <!-- Static navbar --> |
| <nav class="navbar navbar-default"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="index.html">Livy</a> |
| </div> |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav"> |
| <li><a href="index.html">Home</a></li> |
| <li class="active"><a href="overview.html">Overview</a></li> |
| <li><a href="quickstart.html">Quick Start</a></li> |
| <li><a href="https://github.com/cloudera/livy#rest-api">API Docs</a></li> |
| <li><a href="https://github.com/cloudera/livy">Repository</a></li> |
| <li><a href="https://github.com/cloudera/livy/wiki/Contributing-to-Livy">Contributions</a></li> |
| </ul> |
| |
| </div><!--/.nav-collapse --> |
| </div><!--/.container-fluid --> |
| </nav> |
| <div class="row"> |
| <div class="col-lg-12"> |
| <h2>Overview</h2> |
| <p>Livy (currently an alpha release) is a service that enables easy interaction with an Apache Spark cluster over a REST interface. It enables easy submission of Spark jobs or snippets of Spark code, synchronous or asynchronous result retrieval, as well as SparkContext management, all via a simple REST interface or a RPC client library. Livy also simplifies the interaction between Spark from application servers, thus enabling the use of Spark for interactive web/mobile applications. Additional features include:</p> |
| <ul> |
| <li>Have long running SparkContexts that can be used for multiple Spark jobs, by multiple clients</li> |
| <li>Share cached RDDs or Dataframes across multiple jobs and clients</li> |
| <li>Multiple SparkContexts can be managed simultaneously, and they run on the cluster (YARN/Mesos) instead of the Livy Server for good fault tolerance and concurrency</li> |
| <li>Jobs can be submitted as precompiled jars, snippets of code, or via Java/Scala client API</li> |
| <li>Ensure security via secure authenticated communication</li> |
| <li>Apache License, 100% open source</li> |
| </ul> |
| <p>To learn more, <a href="https://youtu.be/C_3iEf_KNv8">watch this tech session video</a> from Spark Summit West 2016.</p> |
| <p><br> |
| <br> |
| </p> |
| <div align="center"> |
| <h4><img src="img/livy-architecture.png" width="800" height="425"><br> |
| <br> |
| Livy Architecture </h4> |
| </div> |
| </div> |
| </div> |
| <footer class="footer"> |
| <p class="pull-right"> </p> |
| <p class="small">Copyright © 2016 Cloudera. All rights reserved.</p> |
| </footer> |
| |
| </div> |
| |
| |
| |
| |
| <script src="js/bootstrap.js"></script> |
| |
| |
| </body> |
| </html> |
| |
