| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <title>Apache Aurora</title> |
| <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css"> |
| <link href="/assets/css/main.css" rel="stylesheet"> |
| <!-- Analytics --> |
| <script type="text/javascript"> |
| var _gaq = _gaq || []; |
| _gaq.push(['_setAccount', 'UA-45879646-1']); |
| _gaq.push(['_setDomainName', 'apache.org']); |
| _gaq.push(['_trackPageview']); |
| |
| (function() { |
| var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; |
| ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; |
| var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); |
| })(); |
| </script> |
| </head> |
| <body> |
| <div class="container-fluid section-header"> |
| <div class="container"> |
| <div class="nav nav-bar"> |
| <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="/documentation/latest/">Documentation</a></li> |
| <li><a href="/community/">Community</a></li> |
| <li><a href="/downloads/">Downloads</a></li> |
| <li><a href="/blog/">Blog</a></li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| |
| <div class="container-fluid"> |
| <div class="container content"> |
| <div class="col-md-12 documentation"> |
| <h5 class="page-header text-uppercase">Documentation |
| <select onChange="window.location.href='/documentation/' + this.value + '/development/client/'" |
| value="0.17.0"> |
| <option value="0.22.0" |
| > |
| 0.22.0 |
| (latest) |
| </option> |
| <option value="0.21.0" |
| > |
| 0.21.0 |
| </option> |
| <option value="0.20.0" |
| > |
| 0.20.0 |
| </option> |
| <option value="0.19.1" |
| > |
| 0.19.1 |
| </option> |
| <option value="0.19.0" |
| > |
| 0.19.0 |
| </option> |
| <option value="0.18.1" |
| > |
| 0.18.1 |
| </option> |
| <option value="0.18.0" |
| > |
| 0.18.0 |
| </option> |
| <option value="0.17.0" |
| selected="selected"> |
| 0.17.0 |
| </option> |
| <option value="0.16.0" |
| > |
| 0.16.0 |
| </option> |
| <option value="0.15.0" |
| > |
| 0.15.0 |
| </option> |
| <option value="0.14.0" |
| > |
| 0.14.0 |
| </option> |
| <option value="0.13.0" |
| > |
| 0.13.0 |
| </option> |
| <option value="0.12.0" |
| > |
| 0.12.0 |
| </option> |
| <option value="0.11.0" |
| > |
| 0.11.0 |
| </option> |
| <option value="0.10.0" |
| > |
| 0.10.0 |
| </option> |
| <option value="0.9.0" |
| > |
| 0.9.0 |
| </option> |
| <option value="0.8.0" |
| > |
| 0.8.0 |
| </option> |
| <option value="0.7.0-incubating" |
| > |
| 0.7.0-incubating |
| </option> |
| <option value="0.6.0-incubating" |
| > |
| 0.6.0-incubating |
| </option> |
| <option value="0.5.0-incubating" |
| > |
| 0.5.0-incubating |
| </option> |
| </select> |
| </h5> |
| <h1 id="developing-the-aurora-client">Developing the Aurora Client</h1> |
| |
| <p>The client is written in Python, and uses the |
| <a href="http://pantsbuild.github.io/python-readme.html">Pants</a> build tool.</p> |
| |
| <h2 id="building-and-testing">Building and Testing</h2> |
| |
| <p>Building and testing the client code are both done using Pants. The relevant targets to know about |
| are:</p> |
| |
| <ul> |
| <li>Build a client executable: <code>./pants binary src/main/python/apache/aurora/client:aurora</code></li> |
| <li>Test client code: <code>./pants test src/test/python/apache/aurora/client/cli:cli</code></li> |
| </ul> |
| |
| <p>If you want to build a source distribution of the client, you need to run <code>./build-support/release/make-python-sdists</code>.</p> |
| |
| <h2 id="creating-custom-builds">Creating Custom Builds</h2> |
| |
| <p>There are situations where you may want to plug in custom logic to the Client that may not be |
| applicable to the open source codebase. Rather than create a whole CLI from scratch, you can |
| easily create your own custom, drop-in replacement aurora.pex using the pants build tool.</p> |
| |
| <p>First, create an AuroraCommandLine implementation as an entry-point for registering customizations:</p> |
| <pre class="highlight objective_c"><code><span style="background-color: #f8f8f8">from</span> <span style="background-color: #f8f8f8">apache</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">aurora</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">client</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">cli</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">client</span> <span style="background-color: #f8f8f8">import</span> <span style="background-color: #f8f8f8">AuroraCommandLine</span> |
| |
| <span style="background-color: #f8f8f8">class</span> <span style="background-color: #f8f8f8">CustomAuroraCommandLine</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">AuroraCommandLine</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> |
| <span style="color: #d14">"""Custom AuroraCommandLine for your needs"""</span> |
| |
| <span style="color: #000000;font-weight: bold">@property</span> |
| <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">name</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> |
| <span style="color: #000000;font-weight: bold">return</span> <span style="color: #d14">"your-company-aurora"</span> |
| |
| <span style="color: #a61717;background-color: #e3d2d2">@classmethod</span> |
| <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">get_description</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">cls</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> |
| <span style="color: #000000;font-weight: bold">return</span> <span style="color: #a61717;background-color: #e3d2d2">'</span><span style="background-color: #f8f8f8">Your</span> <span style="background-color: #f8f8f8">Company</span> <span style="background-color: #f8f8f8">internal</span> <span style="background-color: #f8f8f8">Aurora</span> <span style="background-color: #f8f8f8">client</span> <span style="background-color: #f8f8f8">command</span> <span style="background-color: #f8f8f8">line</span><span style="color: #a61717;background-color: #e3d2d2">'</span> |
| |
| <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">__init__</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> |
| <span style="background-color: #f8f8f8">super</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">CustomAuroraCommandLine</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">).</span><span style="background-color: #f8f8f8">__init__</span><span style="background-color: #f8f8f8">()</span> |
| <span style="color: #999999;font-weight: bold"># Add custom plugins.. |
| </span> <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">register_plugin</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">YourCustomPlugin</span><span style="background-color: #f8f8f8">())</span> |
| |
| <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">register_nouns</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> |
| <span style="background-color: #f8f8f8">super</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">CustomAuroraCommandLine</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">).</span><span style="background-color: #f8f8f8">register_nouns</span><span style="background-color: #f8f8f8">()</span> |
| <span style="color: #999999;font-weight: bold"># You can even add new commands / sub-commands! |
| </span> <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">register_noun</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">YourStartUpdateProxy</span><span style="background-color: #f8f8f8">())</span> |
| <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">register_noun</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">YourDeployWorkflowCommand</span><span style="background-color: #f8f8f8">())</span> |
| </code></pre> |
| |
| <p>Secondly, create a main entry point:</p> |
| <pre class="highlight plaintext"><code>def proxy_main(): |
| client = CustomAuroraCommandLine() |
| if len(sys.argv) == 1: |
| sys.argv.append("-h") |
| sys.exit(client.execute(sys.argv[1:])) |
| </code></pre> |
| |
| <p>Finally, you can wire everything up with a pants BUILD file in your project directory:</p> |
| <pre class="highlight plaintext"><code>python_binary( |
| name='aurora', |
| entry_point='your_company.aurora.client:proxy_main', |
| dependencies=[ |
| ':client_lib' |
| ] |
| ) |
| |
| python_library( |
| name='client_lib', |
| sources = [ |
| 'client.py', |
| 'custom_plugin.py', |
| 'custom_command.py', |
| ], |
| dependencies = [ |
| # The Apache Aurora client |
| # Any other dependencies for your custom code |
| ], |
| ) |
| </code></pre> |
| |
| <p>Using the same commands to build the client as above (but obviously pointing to this BUILD file |
| instead), you will have a drop-in replacement aurora.pex file with your customizations.</p> |
| |
| <h2 id="running-debugging">Running/Debugging</h2> |
| |
| <p>For manually testing client changes against a cluster, we use <a href="https://www.vagrantup.com/">Vagrant</a>. |
| To start a virtual cluster, you need to install Vagrant, and then run <code>vagrant up</code> for the root of |
| the aurora workspace. This will create a vagrant host named “devcluster”, with a Mesos master, a set |
| of Mesos agents, and an Aurora scheduler.</p> |
| |
| <p>If you have a change you would like to test in your local cluster, you’ll rebuild the client:</p> |
| <pre class="highlight plaintext"><code>vagrant ssh -c 'aurorabuild client' |
| </code></pre> |
| |
| <p>Once this completes, the <code>aurora</code> command will reflect your changes.</p> |
| |
| <h2 id="running-debugging-in-pycharm">Running/Debugging in PyCharm</h2> |
| |
| <p>It’s possible to use PyCharm to run and debug both the client and client tests in an IDE. In order |
| to do this, first run:</p> |
| <pre class="highlight plaintext"><code>build-support/python/make-pycharm-virtualenv |
| </code></pre> |
| |
| <p>This script will configure a virtualenv with all of our Python requirements. Once the script |
| completes it will emit instructions for configuring PyCharm:</p> |
| <pre class="highlight plaintext"><code>Your PyCharm environment is now set up. You can open the project root |
| directory with PyCharm. |
| |
| Once the project is loaded: |
| - open project settings |
| - click 'Project Interpreter' |
| - click the cog in the upper-right corner |
| - click 'Add Local' |
| - select 'build-support/python/pycharm.venv/bin/python' |
| - click 'OK' |
| </code></pre> |
| |
| <h3 id="running-debugging-tests">Running/Debugging Tests</h3> |
| |
| <p>After following these instructions, you should now be able to run/debug tests directly from the IDE |
| by right-clicking on a test (or test class) and choosing to run or debug:</p> |
| |
| <p><a href="/documentation/0.18.1/images/debug-client-test.png"><img alt="Debug Client Test" src="../../images/debug-client-test.png" /></a></p> |
| |
| <p>If you’ve set a breakpoint, you can see the run will now stop and let you debug:</p> |
| |
| <p><a href="/documentation/0.18.1/images/debugging-client-test.png"><img alt="Debugging Client Test" src="../../images/debugging-client-test.png" /></a></p> |
| |
| <h3 id="running-debugging-the-client">Running/Debugging the Client</h3> |
| |
| <p>Actually running and debugging the client is unfortunately a bit more complex. You’ll need to create |
| a Run configuration:</p> |
| |
| <ul> |
| <li>Go to Run → Edit Configurations</li> |
| <li>Click the + icon to add a new configuration.</li> |
| <li>Choose python and name the configuration ‘client’.</li> |
| <li>Set the script path to <code>/your/path/to/aurora/src/main/python/apache/aurora/client/cli/client.py</code></li> |
| <li>Set the script parameters to the command you want to run (e.g. <code>job status <job key></code>)</li> |
| <li>Expand the Environment section and click the ellipsis to add a new environment variable</li> |
| <li>Click the + at the bottom to add a new variable named AURORA<em>CONFIG</em>ROOT whose value is the |
| path where the your cluster configuration can be found. For example, to talk to the scheduler |
| running in the vagrant image, it would be set to <code>/your/path/to/aurora/examples/vagrant</code> (this |
| is the directory where our example clusters.json is found).</li> |
| <li>You should now be able to run and debug this configuration!</li> |
| </ul> |
| |
| </div> |
| |
| </div> |
| </div> |
| <div class="container-fluid section-footer buffer"> |
| <div class="container"> |
| <div class="row"> |
| <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> |
| <ul> |
| <li><a href="/downloads/">Downloads</a></li> |
| <li><a href="/community/">Mailing Lists</a></li> |
| <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li> |
| <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> |
| </ul> |
| </div> |
| <div class="col-md-2"><h3>The ASF</h3> |
| <ul> |
| <li><a href="http://www.apache.org/licenses/">License</a></li> |
| <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li> |
| <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li> |
| <li><a href="http://www.apache.org/security/">Security</a></li> |
| </ul> |
| </div> |
| <div class="col-md-6"> |
| <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> |
| </div> |
| </div> |
| </div> |
| |
| </body> |
| </html> |
