| <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "https://www.w3.org/TR/html4/loose.dtd"> |
| |
| |
| <!-- ====================================================================== --> |
| <!-- GENERATED FILE, DO NOT EDIT, EDIT THE XML FILE IN xdocs INSTEAD! --> |
| <!-- ====================================================================== --> |
| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/> |
| <style type="text/css">@import "stylesheets/base.css";</style> |
| <meta name="author" value=" |
| Apache UIMA Team |
| "> |
| <meta name="email" value="dev@uima.apache.org"> |
| |
| |
| |
| <title>Apache UIMA - Running UIMA DUCC in a Cloud environment</title> |
| |
| <!-- Begin Cookie Consent plugin by Silktide - https://silktide.com/cookieconsent --> |
| <!-- Commented out because implied consent is not compatible with GDPR --> |
| <!-- |
| <script type="text/javascript"> |
| window.cookieconsent_options = {"message":"This website uses cookies to ensure you get the best experience on our website","dismiss":"Got it!","learnMore":"More info","link":"https://uima.apache.org/privacy-policy.html","theme":"dark-bottom"}; |
| </script> |
| |
| <script type="text/javascript" src="/cookieconsent2/cookieconsent.min.js"></script> |
| --> |
| <!-- End Cookie Consent plugin --> |
| |
| <!-- Begin Google Analytics --> |
| <!-- Commented out because GA requires consent according to GDPR --> |
| <!-- |
| <script> |
| (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ |
| (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), |
| m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) |
| })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); |
| |
| ga('create', 'UA-70846351-1', 'auto'); |
| ga('set', 'anonymizeIp', true); |
| ga('send', 'pageview'); |
| |
| </script> |
| --> |
| <!-- End Google Analytics --> |
| </head> |
| |
| <body> |
| <div class="topLogos"> |
| <table border="0" width="100%" cellspacing="0"> |
| <!-- TOP IMAGE --> |
| <tr> |
| <td align='LEFT'> |
| <a href="index.html"> |
| <img style="border: 1px solid black;" src="./images/UIMA_banner2tlpTm.png" alt="UIMA project logo" border="0"/> |
| </a> |
| </td> |
| <td align='CENTER'> |
| <div class="pageBanner">Running UIMA DUCC in a Cloud environment</div> |
| </td> |
| <td align='RIGHT'> |
| <a href="https://www.apache.org"> |
| <img src="./images/asf-logo-on-white-smallTm.png" alt="Apache UIMA" border="0"/> |
| </a> |
| </td> |
| </tr> |
| </table> |
| <hr noshade="" size="1"/> |
| </div> |
| <table border="0" width="100%" cellspacing="4"> |
| <tr> |
| <td align='RIGHT' colspan="2"> |
| <form method="get" action="https://www.google.com/search"> |
| Search the site |
| <input type="text" name="q" size="25" maxlength="255" value="" /> |
| <input type="hidden" name="sitesearch" value="https://uima.apache.org/" /> |
| <input name="Search" value="Search Site" type="submit"/> |
| </form> |
| </td> |
| </tr> |
| <tr> <!-- LEFT SIDE NAVIGATION --> |
| <td width="20%" valign="top"> |
| |
| |
| |
| |
| |
| |
| <!-- regular menu --> |
| <div class="navBar"> |
| <br/> |
| <div class="navBarItem"> <div class="navPartHeading">General</div> |
| </div> |
| <div class="navBar"> |
| <div class="navBarItem"> <a href="./index.html">Home</a> |
| </div> |
| <div class="navBarItem"> <a href="./downloads.cgi">Downloads</a> |
| </div> |
| <div class="navBarItem"> <a href="./documentation.html">Documentation</a> |
| </div> |
| <div class="navBarItem"> <a href="./news.html">News</a> |
| </div> |
| <div class="navBarItem"> <a href="./publications.html">Publications</a> |
| </div> |
| <br style="line-height: .5em"/> |
| <div class="navBarItem"> <a href="https://issues.apache.org/jira/browse/uima" target="_blank" rel="noopener">Issue tracker <img src="images/offsitelink.png"/></a> |
| </div> |
| <div class="navBarItem"> <a href="https://cwiki.apache.org/confluence/display/UIMA/" target="_blank" rel="noopener">Wiki <img src="images/offsitelink.png"/></a> |
| </div> |
| <br style="line-height: .5em"/> |
| <div class="navBarItem"> <a href="https://cwiki.apache.org/confluence/display/UIMA/Powered+by+Apache+UIMA" target="_blank" rel="noopener">Powered By UIMA <img src="images/offsitelink.png"/></a> |
| </div> |
| </div> |
| <br/> |
| <div class="navBarItem"> <div class="navPartHeading">Community</div> |
| </div> |
| <div class="navBar"> |
| <div class="navBarItem"> <a href="./get-involved.html">Get Involved</a> |
| </div> |
| <div class="navBarItem"> <a href="./mail-lists.html">Mailing Lists</a> |
| </div> |
| <div class="navBarItem"> <a href="./contribution-policy.html">Contribution Policies</a> |
| </div> |
| <div class="navBarItem"> <a href="./faq.html">FAQ</a> |
| </div> |
| <div class="navBarItem"> <a href="./project-guidelines.html">Project Guidelines</a> |
| </div> |
| </div> |
| <br/> |
| <div class="navBarItem"> <div class="navPartHeading">Scaleout Frameworks</div> |
| </div> |
| <div class="navBar"> |
| <div class="navBarItem"> <a href="./doc-uimaas-what.html">UIMA-AS</a> |
| </div> |
| <div class="navBarItem"> <a href="./doc-uimaducc-whatitam.html">UIMA-DUCC</a> |
| </div> |
| <div class="navBarItem"> <a href="./doc-uimaducc-demo.html">..Demo Page</a> |
| </div> |
| <div class="navBarItem"> <a href="http://uima-ducc-demo.apache.org:42133" target="_blank" rel="noopener">..Demo Live <img src="images/offsitelink.png"/></a> |
| </div> |
| </div> |
| <br/> |
| <div class="navBarItem"> <div class="navPartHeading">Components & Tools</div> |
| </div> |
| <div class="navBar"> |
| <div class="navBarItem"> <a href="./sandbox.html#uima-addons-annotators">Annotators</a> |
| </div> |
| <div class="navBarItem"> <a href="./toolsServers.html">Tools & Servers</a> |
| </div> |
| <div class="navBarItem"> <a href="./sandbox.html">Addons and Sandbox</a> |
| </div> |
| <div class="navBarItem"> <a href="./ruta.html">UIMA Ruta</a> |
| </div> |
| <div class="navBarItem"> <a href="./uimafit.html">uimaFIT</a> |
| </div> |
| <div class="navBarItem"> <a href="./external-resources.html">External Resources</a> |
| </div> |
| </div> |
| <br/> |
| <div class="navBarItem"> <div class="navPartHeading">Development</div> |
| </div> |
| <div class="navBar"> |
| <div class="navBarItem"> <a href="./dev-quick.html">Quick Start: building</a> |
| </div> |
| <div class="navBarItem"> <a href="./building-uima.html">Building from Source</a> |
| </div> |
| <div class="navBarItem"> <a href="./one-time-setup.html">One-time setups</a> |
| </div> |
| <div class="navBarItem"> <a href="./svn.html">Source Code</a> |
| </div> |
| <div class="navBarItem"> <a href="./release.html">Doing a UIMA release</a> |
| </div> |
| <div class="navBarItem"> <a href="https://www.apache.org/security/committers.html" target="_blank" rel="noopener">Doing a CVE (Apache) <img src="images/offsitelink.png"/></a> |
| </div> |
| <div class="navBarItem"> <a href="./eclipse-update-site.html">Eclipse Update Sites</a> |
| </div> |
| <div class="navBarItem"> <a href="./git.html">GIT</a> |
| </div> |
| <div class="navBarItem"> <a href="./codeConventions.html">Code Conventions</a> |
| </div> |
| <div class="navBarItem"> <a href="./uima-specification.html">UIMA Specification (OASIS)</a> |
| </div> |
| <div class="navBarItem"> <a href="./team-list.html">Project Team</a> |
| </div> |
| <div class="navBarItem"> <a href="./maven-design.html">Maven Use</a> |
| </div> |
| <div class="navBarItem"> <a href="./updating-website.html">Updating this Website</a> |
| </div> |
| </div> |
| <br/> |
| <div class="navBarItem"> <div class="navPartHeading">Events and Conferences</div> |
| </div> |
| <div class="navBar"> |
| <div class="navBarItem"> <a href="./coling14.html">COLING 2014</a> |
| </div> |
| <div class="navBarItem"> <a href="./gscl13.html">GSCL 2013</a> |
| </div> |
| <div class="navBarItem"> <a href="./iks09.html">IKS 2009</a> |
| </div> |
| <div class="navBarItem"> <a href="./gscl09.html">GSCL 2009</a> |
| </div> |
| <div class="navBarItem"> <a href="./lsm09.html">LSM 2009</a> |
| </div> |
| <div class="navBarItem"> <a href="./lrec08.html">LREC 2008</a> |
| </div> |
| <div class="navBarItem"> <a href="./gldv07.html">GLDV 2007</a> |
| </div> |
| </div> |
| <br/> |
| <div class="navBarItem"> <div class="navPartHeading">ASF</div> |
| </div> |
| <div class="navBar"> |
| <div class="navBarItem"> <a href="https://www.apache.org/licenses/" target="_blank" rel="noopener">License <img src="images/offsitelink.png"/></a> |
| </div> |
| <div class="navBarItem"> <a href="https://www.apache.org/foundation/thanks.html" target="_blank" rel="noopener">ASF Sponsors <img src="images/offsitelink.png"/></a> |
| </div> |
| <div class="navBarItem"> <a href="https://www.apache.org/foundation/sponsorship.html" target="_blank" rel="noopener">ASF Sponsorship <img src="images/offsitelink.png"/></a> |
| </div> |
| <div class="navBarItem"> <a href="./security_report">Security</a> |
| </div> |
| </div> |
| </div> |
| </td> |
| <td width="80%" align="left" valign="top"> |
| <div class="sectionTable"> |
| <table class="sectionTable"> |
| <tr><td> |
| <a name="DUCC in Clouds"><h1><img src="images/UIMA_4sq50tightCropSolid.png"/> DUCC in Clouds</h1></a> |
| </td></tr> |
| <tr><td> |
| <blockquote class="sectionBody"> |
| <table class="subsectionTable"> |
| <tr><td> |
| |
| |
| |
| <a name="Preliminary"> |
| <h2>Preliminary |
| </h2> |
| </a> |
| </td></tr> |
| <tr><td> |
| <blockquote class="subsectionBody"> |
| <p> |
| This page collects information on running DUCC |
| in Cloud environments. |
| </p> |
| <p>It is preliminary, and is missing details. |
| Nevertheless, it may have some potentially useful references and ideas. |
| More will be added as it becomes available.</p> |
| </blockquote> |
| </td></tr> |
| </table> |
| <table class="subsectionTable"> |
| <tr><td> |
| |
| |
| |
| <a name="External Links"> |
| <h2>External Links |
| </h2> |
| </a> |
| </td></tr> |
| <tr><td> |
| <blockquote class="subsectionBody"> |
| <ul> |
| <li><p> |
| <a target="_blank" rel="nofollow noopener" href="https://github.com/aleksey-hariton/uima-ducc-docker"> |
| https://github.com/aleksey-hariton/uima-ducc-docker</a> |
| is a github repository with one approach. |
| </p></li> |
| </ul> |
| </blockquote> |
| </td></tr> |
| </table> |
| <table class="subsectionTable"> |
| <tr><td> |
| |
| |
| |
| <a name="Considerations"> |
| <h2>Considerations |
| </h2> |
| </a> |
| </td></tr> |
| <tr><td> |
| <blockquote class="subsectionBody"> |
| <p style="margin-left: 2rem">NOTE: This information is from an email to the uima-dev list by one of the DUCC committers. |
| In this email, they reference a "glusterfs" which is a shared file system cross mounted |
| on various machines. This "shared file system" approach is the conventional way DUCC is |
| configured, but it can be configured without a shared file system; see the DUCC docs.</p> |
| <p>We've been running DUCC in a cloud environment for almost a year now. The |
| DUCC master and a |
| <a target="_blank" rel="nofollow noopener" href="https://www.google.com/search?q=glusterfs">glusterfs</a> |
| servers run on bare metal and all of the |
| workstations and worker machines run on VMs. Cluster users add VMs to the |
| cluster as needed. A job can be started on one more workers and then |
| additional VMs dynamically added to which the job will automatically scale |
| out to use. A common system image is maintained on all VM machines via an |
| LDAP server and shared filesystem data. Users belong to groups and share |
| machines allocated by members of the group. |
| </p> |
| <p> |
| A DUCC VM-image is used to automatically connect new VMs to the DUCC master |
| and glusterfs. The DUCC master configuration may be updated anytime, for |
| example to add new groups or even update master software. VMs automatically |
| sync DUCC software and configuration each time they start their DUCC agent. |
| The VM image supports three different machine types: a graphical |
| workstation, a CPU worker and a GPU worker (used typically for deep learning |
| training). DUCC spawns work on specified |
| worker machine types and even specific machines. Workstations are optional |
| as DUCC requests can be submitted from worker machines. Docker images are |
| supported using <a target="_blank" rel="nofollow noopener" href="https://www.google.com/search?q=podman">Podman</a>. Podman runs rootless and only allows access to all |
| mounted file systems with user credentials. |
| </p> |
| <p> |
| |
| In order to keep some level of data security, a group directory is only |
| mounted on the VMs created by members of the group. Individual users |
| maintain file permissions as desired, but, as anyone that creates a VM has |
| root access, they could become any other user and access data from other |
| group members.There is a self-service glusterfs webapp that is used to |
| export group data to new VMs and manage quotas. |
| </p> |
| <p> |
| |
| The VM-image builder and glusterfs webapp are not yet part of Apache DUCC. |
| |
| </p> |
| </blockquote> |
| </td></tr> |
| </table> |
| </blockquote> |
| </p> |
| </td></tr> |
| </table> |
| </td> |
| </tr> |
| <!-- FOOTER --> |
| <tr><td colspan="2"> |
| <hr noshade="" size="1"/> |
| </td></tr> |
| <tr><td colspan="2"> |
| <table class="pageFooter"> |
| <tr> |
| <td><a href="index.html">Home</a></td> |
| <td><a href="privacy-policy.html">Privacy Policy</a></td> |
| <td style="font-size:75%"> |
| Copyright © 2006-2013, The Apache Software Foundation.<br/> |
| Apache UIMA, UIMA, the Apache UIMA logo and the Apache Feather logo are trademarks of The Apache Software Foundation.<br/> |
| All other marks mentioned may be trademarks or registered trademarks of their respective owners. |
| </td> |
| <td><a href="mailto:dev@uima.apache.org">Contact us</a></td> |
| </tr> |
| </table> |
| </td></tr> |
| </table> |
| </body> |
| </html> |
| |
