| <!doctype html> |
| <html class="docs-version-1.1.0" lang="en" dir="ltr"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="viewport" content="width=device-width,initial-scale=1"> |
| <meta name="generator" content="Docusaurus v2.0.0-beta.14"> |
| <link rel="alternate" type="application/rss+xml" href="/blog/rss.xml" title="Apache Linkis Blog RSS Feed"> |
| <link rel="alternate" type="application/atom+xml" href="/blog/atom.xml" title="Apache Linkis Blog Atom Feed"> |
| <link rel="search" type="application/opensearchdescription+xml" title="Apache Linkis" href="/opensearch.xml"><title data-react-helmet="true">JAVA SDK Manual | Apache Linkis</title><meta data-react-helmet="true" name="twitter:card" content="summary_large_image"><meta data-react-helmet="true" property="og:url" content="https://linkis.incubator.apache.org/docs/1.1.0/user_guide/sdk_manual"><meta data-react-helmet="true" name="docsearch:language" content="en"><meta data-react-helmet="true" name="docsearch:version" content="1.1.0"><meta data-react-helmet="true" name="docsearch:docusaurus_tag" content="docs-default-1.1.0"><meta data-react-helmet="true" property="og:title" content="JAVA SDK Manual | Apache Linkis"><meta data-react-helmet="true" name="description" content="Linkis provides a convenient interface for calling JAVA and SCALA. It can be used only by introducing the linkis-computation-client module. After 1.0, the method of submitting with Label is added. The following will introduce both ways that compatible with 0.X and newly added in 1.0."><meta data-react-helmet="true" property="og:description" content="Linkis provides a convenient interface for calling JAVA and SCALA. It can be used only by introducing the linkis-computation-client module. After 1.0, the method of submitting with Label is added. The following will introduce both ways that compatible with 0.X and newly added in 1.0."><link data-react-helmet="true" rel="shortcut icon" href="/img/favicon.ico"><link data-react-helmet="true" rel="canonical" href="https://linkis.incubator.apache.org/docs/1.1.0/user_guide/sdk_manual"><link data-react-helmet="true" rel="alternate" href="https://linkis.incubator.apache.org/docs/1.1.0/user_guide/sdk_manual" hreflang="en"><link data-react-helmet="true" rel="alternate" href="https://linkis.incubator.apache.org/zh-CN/docs/1.1.0/user_guide/sdk_manual" hreflang="zh-CN"><link data-react-helmet="true" rel="alternate" href="https://linkis.incubator.apache.org/docs/1.1.0/user_guide/sdk_manual" hreflang="x-default"><link data-react-helmet="true" rel="preconnect" href="https://AE29KQB3IA-dsn.algolia.net" crossorigin="anonymous"><link rel="stylesheet" href="/assets/css/styles.92d87943.css"> |
| <link rel="preload" href="/assets/js/runtime~main.a4c60f50.js" as="script"> |
| <link rel="preload" href="/assets/js/main.42471b13.js" as="script"> |
| </head> |
| <body> |
| <script>!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}()</script><div id="__docusaurus"> |
| <div><a href="#" class="skipToContent_OuoZ">Skip to main content</a></div><nav class="navbar navbar--fixed-top"><div class="navbar__inner"><div class="navbar__items"><button aria-label="Navigation bar toggle" class="navbar__toggle clean-btn" type="button" tabindex="0"><svg width="30" height="30" viewBox="0 0 30 30" aria-hidden="true"><path stroke="currentColor" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M4 7h22M4 15h22M4 23h22"></path></svg></button><a class="navbar__brand" href="/"><img src="/img/logo.png" alt="Apache Linkis Logo" class="themedImage_TMUO themedImage--light_4Vu1 navbar__logo"><img src="/img/logo.png" alt="Apache Linkis Logo" class="themedImage_TMUO themedImage--dark_uzRr navbar__logo"><b class="navbar__title">Apache Linkis(Incubating)</b></a><a class="navbar__item navbar__link" href="/">Home</a><a class="navbar__item navbar__link" href="/faq/main">FAQ</a><a class="navbar__item navbar__link" href="/download/main">Download</a><a class="navbar__item navbar__link" href="/community/how-to-subscribe">Community</a><a class="navbar__item navbar__link" href="/blog">Blog</a><a class="navbar__item navbar__link" href="/team">Team</a><a aria-current="page" class="navbar__item navbar__link navbar__link--active" href="/user">Users</a><div class="navbar__item dropdown dropdown--hoverable"><a class="navbar__link">ASF</a><ul class="dropdown__menu"><li><a href="https://www.apache.org/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Foundation</a></li><li><a href="https://www.apache.org/licenses/" target="_blank" rel="noopener noreferrer" class="dropdown__link">License</a></li><li><a href="https://www.apache.org/events/current-event" target="_blank" rel="noopener noreferrer" class="dropdown__link">Events</a></li><li><a href="https://www.apache.org/security/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Security</a></li><li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Sponsorship</a></li><li><a href="https://www.apache.org/foundation/policies/privacy.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Privacy</a></li><li><a href="https://www.apache.org/foundation/thanks.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Thanks</a></li></ul></div></div><div class="navbar__items navbar__items--right"><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a class="navbar__link">Doc</a><ul class="dropdown__menu"><li><a class="dropdown__link" href="/docs/1.1.3/introduction">Next(1.1.3)</a></li><li><a class="dropdown__link" href="/docs/latest/introduction">1.1.2</a></li><li><a class="dropdown__link" href="/docs/1.1.1/introduction">1.1.1</a></li><li><a class="dropdown__link" href="/docs/1.1.0/introduction">1.1.0</a></li><li><a class="dropdown__link" href="/docs/1.0.3/introduction">1.0.3</a></li><li><a class="dropdown__link" href="/docs/1.0.2/introduction">1.0.2</a></li><li><a class="dropdown__link" href="/docs/0.11.0/introduction">0.11.0</a></li><li><a class="dropdown__link" href="/versions">All Version</a></li></ul></div><a href="https://github.com/apache/incubator-linkis" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link header-github-link" aria-label="GitHub"></a><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" class="navbar__link"><span><svg t="1631348384596" class="iconLanguage_EbrZ" viewBox="0 0 1024 1024" version="1.1" p-id="557" width="20" height="20"><path d="M547.797333 638.208l-104.405333-103.168 1.237333-1.28a720.170667 720.170667 0 0 0 152.490667-268.373333h120.448V183.082667h-287.744V100.906667H347.605333v82.218666H59.818667V265.386667h459.178666a648.234667 648.234667 0 0 1-130.304 219.946666 643.242667 643.242667 0 0 1-94.976-137.728H211.541333a722.048 722.048 0 0 0 122.453334 187.434667l-209.194667 206.378667 58.368 58.368 205.525333-205.525334 127.872 127.829334 31.232-83.84m231.424-208.426667h-82.218666l-184.96 493.312h82.218666l46.037334-123.306667h195.242666l46.464 123.306667h82.218667l-185.002667-493.312m-107.690666 287.744l66.56-178.005333 66.602666 178.005333z" fill="currentColor" p-id="558"></path></svg><span>English</span></span></a><ul class="dropdown__menu"><li><a href="/docs/1.1.0/user_guide/sdk_manual" target="_self" rel="noopener noreferrer" class="dropdown__link dropdown__link--active" style="text-transform:capitalize">English</a></li><li><a href="/zh-CN/docs/1.1.0/user_guide/sdk_manual" target="_self" rel="noopener noreferrer" class="dropdown__link" style="text-transform:capitalize">简体中文</a></li></ul></div><div class="searchBox_Bc3W"><button type="button" class="DocSearch DocSearch-Button" aria-label="Search"><span class="DocSearch-Button-Container"><svg width="20" height="20" class="DocSearch-Search-Icon" viewBox="0 0 20 20"><path d="M14.386 14.386l4.0877 4.0877-4.0877-4.0877c-2.9418 2.9419-7.7115 2.9419-10.6533 0-2.9419-2.9418-2.9419-7.7115 0-10.6533 2.9418-2.9419 7.7115-2.9419 10.6533 0 2.9419 2.9418 2.9419 7.7115 0 10.6533z" stroke="currentColor" fill="none" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round"></path></svg><span class="DocSearch-Button-Placeholder">Search</span></span><span class="DocSearch-Button-Keys"></span></button></div></div></div><div role="presentation" class="navbar-sidebar__backdrop"></div></nav><div class="main-wrapper docs-wrapper docs-doc-page"><div class="docPage_lDyR"><button class="clean-btn backToTopButton_i9tI" type="button"><svg viewBox="0 0 24 24" width="28"><path d="M7.41 15.41L12 10.83l4.59 4.58L18 14l-6-6-6 6z" fill="currentColor"></path></svg></button><aside class="docSidebarContainer_0YBq"><div class="sidebar_a3j0"><nav class="menu thin-scrollbar menu_cyFh menuWithAnnouncementBar_+O1J"><ul class="theme-doc-sidebar-menu menu__list"><li class="theme-doc-sidebar-item-link menu__list-item"><a class="menu__link" href="/docs/1.1.0/introduction">Introduction</a></li><li class="theme-doc-sidebar-item-link menu__list-item"><a class="menu__link" href="/docs/1.1.0/release">Version Overview</a></li><li class="theme-doc-sidebar-item-category menu__list-item menu__list-item--collapsed"><a class="menu__link menu__link--sublist" href="#">Deployment</a></li><li class="theme-doc-sidebar-item-category menu__list-item"><a class="menu__link menu__link--sublist menu__link--active" href="#">User Guide</a><ul style="display:block;overflow:visible;height:auto" class="menu__list"><li class="theme-doc-sidebar-item-link menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.1.0/user_guide/overview">Overview</a></li><li class="theme-doc-sidebar-item-link menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.1.0/user_guide/how_to_use">How to Use</a></li><li class="theme-doc-sidebar-item-link menu__list-item"><a class="menu__link menu__link--active" aria-current="page" tabindex="0" href="/docs/1.1.0/user_guide/sdk_manual">JAVA SDK Manual</a></li><li class="theme-doc-sidebar-item-link menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.1.0/user_guide/linkiscli_manual">Linkis-Cli Manual</a></li><li class="theme-doc-sidebar-item-link menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.1.0/user_guide/console_manual">Console User Manual</a></li><li class="theme-doc-sidebar-item-link menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.1.0/user_guide/linkis-datasource-client">DataSource Client SDK</a></li></ul></li><li class="theme-doc-sidebar-item-category menu__list-item menu__list-item--collapsed"><a class="menu__link menu__link--sublist" href="#">Engine Usage</a></li><li class="theme-doc-sidebar-item-category menu__list-item menu__list-item--collapsed"><a class="menu__link menu__link--sublist" href="#">API Docs</a></li><li class="theme-doc-sidebar-item-category menu__list-item menu__list-item--collapsed"><a class="menu__link menu__link--sublist" href="#">Architecture</a></li><li class="theme-doc-sidebar-item-category menu__list-item menu__list-item--collapsed"><a class="menu__link menu__link--sublist" href="#">Development Doc</a></li><li class="theme-doc-sidebar-item-category menu__list-item menu__list-item--collapsed"><a class="menu__link menu__link--sublist" href="#">Upgrade Guide</a></li><li class="theme-doc-sidebar-item-category menu__list-item menu__list-item--collapsed"><a class="menu__link menu__link--sublist" href="#">Tuning And Troubleshooting</a></li></ul></nav></div></aside><main class="docMainContainer_r8cw"><div class="container padding-top--md padding-bottom--lg"><div class="row"><div class="col docItemCol_zHA2"><div class="theme-doc-version-banner alert alert--warning margin-bottom--md" role="alert"><div>This is documentation for <!-- -->Apache Linkis<!-- --> <b>1.1.0</b>, which is no longer actively maintained.</div><div class="margin-top--md">For up-to-date documentation, see the <b><a href="/docs/latest/user_guide/sdk_manual">latest version</a></b> (<!-- -->1.1.2<!-- -->).</div></div><div class="docItemContainer_oiyr"><article><span class="theme-doc-version-badge badge badge--secondary">Version: <!-- -->1.1.0</span><div class="tocCollapsible_aw-L theme-doc-toc-mobile tocMobile_Tx6Y"><button type="button" class="clean-btn tocCollapsibleButton_zr6a">On this page</button></div><div class="theme-doc-markdown markdown"><header><h1>JAVA SDK Manual</h1></header><blockquote><p>Linkis provides a convenient interface for calling JAVA and SCALA. It can be used only by introducing the linkis-computation-client module. After 1.0, the method of submitting with Label is added. The following will introduce both ways that compatible with 0.X and newly added in 1.0.</p></blockquote><h2><a aria-hidden="true" tabindex="-1" class="anchor anchor__h2 anchorWithStickyNavbar_y2LR" id="1-introduce-dependent-modules"></a>1. Introduce dependent modules<a class="hash-link" href="#1-introduce-dependent-modules" title="Direct link to heading">#</a></h2><div class="codeBlockContainer_J+bg"><div class="codeBlockContent_csEI"><pre tabindex="0" class="prism-code language-undefined codeBlock_rtdJ thin-scrollbar" style="color:#F8F8F2;background-color:#282A36"><code class="codeBlockLines_1zSZ"><span class="token-line" style="color:#F8F8F2"><span class="token plain"><dependency></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> <groupId>org.apache.linkis</groupId></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> <artifactId>linkis-computation-client</artifactId></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> <version>${linkis.version}</version></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"></dependency></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">Such as:</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"><dependency></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> <groupId>org.apache.linkis</groupId></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> <artifactId>linkis-computation-client</artifactId></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> <version>1.0.3</version></span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"></dependency></span></span></code></pre><button type="button" aria-label="Copy code to clipboard" class="copyButton_M3SB clean-btn">Copy</button></div></div><h2><a aria-hidden="true" tabindex="-1" class="anchor anchor__h2 anchorWithStickyNavbar_y2LR" id="2-java-test-code"></a>2. Java test code<a class="hash-link" href="#2-java-test-code" title="Direct link to heading">#</a></h2><p>Create the Java test class LinkisClientTest. Refer to the comments to understand the purposes of those interfaces:</p><div class="codeBlockContainer_J+bg"><div class="codeBlockContent_csEI java"><pre tabindex="0" class="prism-code language-java codeBlock_rtdJ thin-scrollbar" style="color:#F8F8F2;background-color:#282A36"><code class="codeBlockLines_1zSZ"><span class="token-line" style="color:#F8F8F2"><span class="token plain">package org.apache.linkis.client.test;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.common.utils.Utils;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.httpclient.dws.authentication.StaticAuthenticationStrategy;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.httpclient.dws.config.DWSClientConfig;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.httpclient.dws.config.DWSClientConfigBuilder;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.manager.label.constant.LabelKeyConstant;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.protocol.constants.TaskConstant;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.UJESClient;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.UJESClientImpl;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.request.JobSubmitAction;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.request.JobExecuteAction;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.request.ResultSetAction;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.response.*;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.commons.io.IOUtils;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import java.util.HashMap;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import java.util.Map;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import java.util.concurrent.TimeUnit;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">public class LinkisClientTest {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 1. build config: linkis gateway url</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> private static DWSClientConfig clientConfig = ((DWSClientConfigBuilder) (DWSClientConfigBuilder.newBuilder()</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .addServerUrl("http://127.0.0.1:9001/") //set linkis-mg-gateway url: http://{ip}:{port}</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .connectionTimeout(30000) //connectionTimeOut</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .discoveryEnabled(false) //disable discovery</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .discoveryFrequency(1, TimeUnit.MINUTES) // discovery frequency</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .loadbalancerEnabled(true) // enable loadbalance</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .maxConnectionSize(5) // set max Connection</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .retryEnabled(false) // set retry</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .readTimeout(30000) //set read timeout</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setAuthenticationStrategy(new StaticAuthenticationStrategy()) //AuthenticationStrategy Linkis authen suppory static and Token</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setAuthTokenKey("hadoop") // set submit user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setAuthTokenValue("hadoop"))) // set passwd or token (setAuthTokenValue("BML-AUTH"))</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setDWSVersion("v1") //linkis rest version v1</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .build();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 2. new Client(Linkis Client) by clientConfig</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> private static UJESClient client = new UJESClientImpl(clientConfig);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> public static void main(String[] args){</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> String user = "hadoop"; // execute user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> String executeCode = "df=spark.sql(\"show tables\")\n" +</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> "show(df)"; // code support:sql/hql/py/scala</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> try {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> System.out.println("user : " + user + ", code : [" + executeCode + "]");</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 3. build job and execute</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> JobExecuteResult jobExecuteResult = toSubmit(user, executeCode);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> //0.x:JobExecuteResult jobExecuteResult = toExecute(user, executeCode);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> System.out.println("execId: " + jobExecuteResult.getExecID() + ", taskId: " + jobExecuteResult.taskID());</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 4. get job jonfo</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> JobInfoResult jobInfoResult = client.getJobInfo(jobExecuteResult);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> int sleepTimeMills = 1000;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> int logFromLen = 0;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> int logSize = 100;</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> while(!jobInfoResult.isCompleted()) {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 5. get progress and log</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> JobProgressResult progress = client.progress(jobExecuteResult);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> System.out.println("progress: " + progress.getProgress());</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> JobLogResult logRes = client.log(jobExecuteResult, logFromLen, logSize);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> logFromLen = logRes.fromLine();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 0: info 1: warn 2: error 3: all</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> System.out.println(logRes.log().get(3));</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> Utils.sleepQuietly(sleepTimeMills);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> jobInfoResult = client.getJobInfo(jobExecuteResult);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> JobInfoResult jobInfo = client.getJobInfo(jobExecuteResult);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 6. Get the result set list (if the user submits multiple SQLs at a time,</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // multiple result sets will be generated)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> String resultSet = jobInfo.getResultSetList(client)[0];</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 7. get resultContent</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> Object fileContents = client.resultSet(ResultSetAction.builder().setPath(resultSet).setUser(jobExecuteResult.getUser()).build()).getFileContent();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> System.out.println("res: " + fileContents);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> } catch (Exception e) {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> e.printStackTrace();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> IOUtils.closeQuietly(client);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> IOUtils.closeQuietly(client);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> /**</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> * Linkis 1.0 recommends the use of Submit method</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> */</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> private static JobExecuteResult toSubmit(String user, String code) {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 1. build params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // set label map :EngineTypeLabel/UserCreatorLabel/EngineRunTypeLabel/Tenant</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> Map<String, Object> labels = new HashMap<String, Object>();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> labels.put(LabelKeyConstant.ENGINE_TYPE_KEY, "spark-2.4.3"); // required engineType Label</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> labels.put(LabelKeyConstant.USER_CREATOR_TYPE_KEY, user + "-IDE");// required execute user and creator</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> labels.put(LabelKeyConstant.CODE_TYPE_KEY, "py"); // required codeType</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // set start up map :engineConn start params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> Map<String, Object> startupMap = new HashMap<String, Object>(16);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // Support setting engine native parameters,For example: parameters of engines such as spark/hive</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> startupMap.put("spark.executor.instances", 2);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // setting linkis params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> startupMap.put("wds.linkis.rm.yarnqueue", "dws");</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 2. build jobSubmitAction</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> JobSubmitAction jobSubmitAction = JobSubmitAction.builder()</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .addExecuteCode(code)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setStartupParams(startupMap)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setUser(user) //submit user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .addExecuteUser(user) // execute user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setLabels(labels)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .build();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 3. to execute</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> return client.submit(jobSubmitAction);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> /**</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> * Compatible with 0.X execution mode</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> */</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> private static JobExecuteResult toExecute(String user, String code) {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 1. build params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // set label map :EngineTypeLabel/UserCreatorLabel/EngineRunTypeLabel/Tenant</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> Map<String, Object> labels = new HashMap<String, Object>();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // labels.put(LabelKeyConstant.TENANT_KEY, "fate");</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // set start up map :engineConn start params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> Map<String, Object> startupMap = new HashMap<String, Object>(16);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // Support setting engine native parameters,For example: parameters of engines such as spark/hive</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> startupMap.put("spark.executor.instances", 2);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // setting linkis params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> startupMap.put("wds.linkis.rm.yarnqueue", "dws");</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 2. build JobExecuteAction (0.X old way of using)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> JobExecuteAction executionAction = JobExecuteAction.builder()</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setCreator("IDE") //creator, the system name of the client requesting linkis, used for system-level isolation</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .addExecuteCode(code) //Execution Code</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setEngineTypeStr("spark") // engineConn type</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setRunTypeStr("py") // code type</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setUser(user) //execute user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setStartupParams(startupMap) // start up params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .build();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> executionAction.addRequestPayload(TaskConstant.LABELS, labels);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> String body = executionAction.getRequestPayload();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> System.out.println(body);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 3. to execute</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> return client.execute(executionAction);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">}</span></span></code></pre><button type="button" aria-label="Copy code to clipboard" class="copyButton_M3SB clean-btn">Copy</button></div></div><p>Run the above code to interact with Linkis</p><h2><a aria-hidden="true" tabindex="-1" class="anchor anchor__h2 anchorWithStickyNavbar_y2LR" id="3-scala-test-code"></a>3. Scala test code<a class="hash-link" href="#3-scala-test-code" title="Direct link to heading">#</a></h2><p>Create the Scala test class LinkisClientTest. Refer to the comments to understand the purposes of those interfaces:</p><div class="codeBlockContainer_J+bg"><div class="codeBlockContent_csEI scala"><pre tabindex="0" class="prism-code language-scala codeBlock_rtdJ thin-scrollbar" style="color:#F8F8F2;background-color:#282A36"><code class="codeBlockLines_1zSZ"><span class="token-line" style="color:#F8F8F2"><span class="token plain">package org.apache.linkis.client.test</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import java.util</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import java.util.concurrent.TimeUnit</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.common.utils.Utils</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.httpclient.dws.authentication.StaticAuthenticationStrategy</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.httpclient.dws.config.DWSClientConfigBuilder</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.manager.label.constant.LabelKeyConstant</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.protocol.constants.TaskConstant</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.UJESClient</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.request._</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.linkis.ujes.client.response._</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.commons.io.IOUtils</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.commons.lang.StringUtils</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">object LinkisClientTest {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">// 1. build config: linkis gateway url</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val clientConfig = DWSClientConfigBuilder.newBuilder()</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .addServerUrl("http://127.0.0.1:9001/") //set linkis-mg-gateway url: http://{ip}:{port}</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .connectionTimeout(30000) //connectionTimeOut</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .discoveryEnabled(false) //disable discovery</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .discoveryFrequency(1, TimeUnit.MINUTES) // discovery frequency</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .loadbalancerEnabled(true) // enable loadbalance</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .maxConnectionSize(5) // set max Connection</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .retryEnabled(false) // set retry</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .readTimeout(30000) //set read timeout</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setAuthenticationStrategy(new StaticAuthenticationStrategy()) //AuthenticationStrategy Linkis authen suppory static and Token</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setAuthTokenKey("hadoop") // set submit user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setAuthTokenValue("hadoop") // set passwd or token (setAuthTokenValue("BML-AUTH"))</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setDWSVersion("v1") //linkis rest version v1</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .build();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 2. new Client(Linkis Client) by clientConfig</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val client = UJESClient(clientConfig)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> def main(args: Array[String]): Unit = {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val user = "hadoop" // execute user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val executeCode = "df=spark.sql(\"show tables\")\n" +</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> "show(df)"; // code support:sql/hql/py/scala</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> try {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 3. build job and execute</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> println("user : " + user + ", code : [" + executeCode + "]")</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val jobExecuteResult = toSubmit(user, executeCode)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> //0.X: val jobExecuteResult = toExecute(user, executeCode)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> println("execId: " + jobExecuteResult.getExecID + ", taskId: " + jobExecuteResult.taskID)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 4. get job jonfo</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> var jobInfoResult = client.getJobInfo(jobExecuteResult)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> var logFromLen = 0</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val logSize = 100</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val sleepTimeMills : Int = 1000</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> while (!jobInfoResult.isCompleted) {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 5. get progress and log</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val progress = client.progress(jobExecuteResult)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> println("progress: " + progress.getProgress)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val logObj = client .log(jobExecuteResult, logFromLen, logSize)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> logFromLen = logObj.fromLine</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val logArray = logObj.getLog</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 0: info 1: warn 2: error 3: all</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> if (logArray != null && logArray.size >= 4 && StringUtils.isNotEmpty(logArray.get(3))) {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> println(s"log: ${logArray.get(3)}")</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> Utils.sleepQuietly(sleepTimeMills)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> jobInfoResult = client.getJobInfo(jobExecuteResult)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> if (!jobInfoResult.isSucceed) {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> println("Failed to execute job: " + jobInfoResult.getMessage)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> throw new Exception(jobInfoResult.getMessage)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 6. Get the result set list (if the user submits multiple SQLs at a time,</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // multiple result sets will be generated)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val jobInfo = client.getJobInfo(jobExecuteResult)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val resultSetList = jobInfoResult.getResultSetList(client)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> println("All result set list:")</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> resultSetList.foreach(println)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val oneResultSet = jobInfo.getResultSetList(client).head</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 7. get resultContent</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val fileContents = client.resultSet(ResultSetAction.builder().setPath(oneResultSet).setUser(jobExecuteResult.getUser).build()).getFileContent</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> println("First fileContents: ")</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> println(fileContents)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> } catch {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> case e: Exception => {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> e.printStackTrace()</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> IOUtils.closeQuietly(client)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> /**</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> * Linkis 1.0 recommends the use of Submit method</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> */</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> def toSubmit(user: String, code: String): JobExecuteResult = {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 1. build params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // set label map :EngineTypeLabel/UserCreatorLabel/EngineRunTypeLabel/Tenant</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val labels: util.Map[String, Any] = new util.HashMap[String, Any]</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> labels.put(LabelKeyConstant.ENGINE_TYPE_KEY, "spark-2.4.3"); // required engineType Label</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> labels.put(LabelKeyConstant.USER_CREATOR_TYPE_KEY, user + "-IDE");// required execute user and creator</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> labels.put(LabelKeyConstant.CODE_TYPE_KEY, "py"); // required codeType</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val startupMap = new java.util.HashMap[String, Any]()</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // Support setting engine native parameters,For example: parameters of engines such as spark/hive</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> startupMap.put("spark.executor.instances", 2);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // setting linkis params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> startupMap.put("wds.linkis.rm.yarnqueue", "dws");</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 2. build jobSubmitAction</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val jobSubmitAction = JobSubmitAction.builder</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .addExecuteCode(code)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setStartupParams(startupMap)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setUser(user) //submit user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .addExecuteUser(user) //execute user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setLabels(labels)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .build</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 3. to execute</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> client.submit(jobSubmitAction)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> /**</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> * Compatible with 0.X execution mode</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> */</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> def toExecute(user: String, code: String): JobExecuteResult = {</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 1. build params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // set label map :EngineTypeLabel/UserCreatorLabel/EngineRunTypeLabel/Tenant</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val labels = new util.HashMap[String, Any]</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // labels.put(LabelKeyConstant.TENANT_KEY, "fate");</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val startupMap = new java.util.HashMap[String, Any]()</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // Support setting engine native parameters,For example: parameters of engines such as spark/hive</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> startupMap.put("spark.executor.instances", 2)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // setting linkis params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> startupMap.put("wds.linkis.rm.yarnqueue", "dws")</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 2. build JobExecuteAction (0.X old way of using)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> val executionAction = JobExecuteAction.builder()</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setCreator("IDE") //creator, the system name of the client requesting linkis, used for system-level isolation</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .addExecuteCode(code) //Execution Code</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setEngineTypeStr("spark") // engineConn type</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setRunTypeStr("py") // code type</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setUser(user) //execute user</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .setStartupParams(startupMap) // start up params</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> .build();</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> executionAction.addRequestPayload(TaskConstant.LABELS, labels);</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> // 3. to execute</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> client.execute(executionAction)</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"> |
| </span></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">}</span></span></code></pre><button type="button" aria-label="Copy code to clipboard" class="copyButton_M3SB clean-btn">Copy</button></div></div></div><footer class="theme-doc-footer docusaurus-mt-lg"><div class="theme-doc-footer-edit-meta-row row"><div class="col"><a href="https://github.com/apache/incubator-linkis-website/edit/dev/versioned_docs/version-1.1.0/user_guide/sdk_manual.md" target="_blank" rel="noreferrer noopener" class="theme-edit-this-page"><svg fill="currentColor" height="20" width="20" viewBox="0 0 40 40" class="iconEdit_mS5F" aria-hidden="true"><g><path d="m34.5 11.7l-3 3.1-6.3-6.3 3.1-3q0.5-0.5 1.2-0.5t1.1 0.5l3.9 3.9q0.5 0.4 0.5 1.1t-0.5 1.2z m-29.5 17.1l18.4-18.5 6.3 6.3-18.4 18.4h-6.3v-6.2z"></path></g></svg>Edit this page</a></div><div class="col lastUpdated_mt2f"></div></div></footer></article><nav class="pagination-nav docusaurus-mt-lg" aria-label="Docs pages navigation"><div class="pagination-nav__item"><a class="pagination-nav__link" href="/docs/1.1.0/user_guide/how_to_use"><div class="pagination-nav__sublabel">Previous</div><div class="pagination-nav__label">« <!-- -->How to Use</div></a></div><div class="pagination-nav__item pagination-nav__item--next"><a class="pagination-nav__link" href="/docs/1.1.0/user_guide/linkiscli_manual"><div class="pagination-nav__sublabel">Next</div><div class="pagination-nav__label">Linkis-Cli Manual<!-- --> »</div></a></div></nav></div></div><div class="col col--3"><div class="tableOfContents_vrFS thin-scrollbar"><ul class="table-of-contents table-of-contents__left-border"><li><a href="#1-introduce-dependent-modules" class="table-of-contents__link">1. Introduce dependent modules</a></li><li><a href="#2-java-test-code" class="table-of-contents__link">2. Java test code</a></li><li><a href="#3-scala-test-code" class="table-of-contents__link">3. Scala test code</a></li></ul></div></div></div></div></main></div></div><footer class="footer"><div class="container"><div class="row footer__links"><div class="col footer__col"><div class="footer__title">Linkis</div><ul class="footer__items"><li class="footer__item"><a class="footer__link-item" href="/docs/latest/introduction">Document</a></li><li class="footer__item"><a class="footer__link-item" href="/faq/main">FAQ</a></li><li class="footer__item"><a href="https://github.com/apache/incubator-linkis/releases" target="_blank" rel="noopener noreferrer" class="footer__link-item"><span>Releases<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_wgqa"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></span></a></li></ul></div><div class="col footer__col"><div class="footer__title">Community</div><ul class="footer__items"><li class="footer__item"><a href="https://github.com/apache/incubator-linkis" target="_blank" rel="noopener noreferrer" class="footer__link-item"><span>GitHub<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_wgqa"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></span></a></li><li class="footer__item"><a href="https://github.com/apache/incubator-linkis/issues" target="_blank" rel="noopener noreferrer" class="footer__link-item"><span>Issue Tracker<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_wgqa"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></span></a></li><li class="footer__item"><a href="https://github.com/apache/incubator-linkis/pulls" target="_blank" rel="noopener noreferrer" class="footer__link-item"><span>Pull Requests<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_wgqa"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></span></a></li></ul></div><div class="col footer__col"><div class="footer__title">Subscribe Mailing List</div><ul class="footer__items"><li class="footer__item"><a class="footer__link-item" href="/community/how-to-subscribe">How to Subscribe</a></li><li class="footer__item"><a href="mailto:dev-subscribe@linkis.apache.org" target="_blank" rel="noopener noreferrer" class="footer__link-item"><span>Subscribe Mail<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_wgqa"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></span></a></li><li class="footer__item"><a href="https://lists.apache.org/list.html?dev@linkis.apache.org" target="_blank" rel="noopener noreferrer" class="footer__link-item"><span>Mail Archive<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_wgqa"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></span></a></li></ul></div></div><div class="footer__bottom text--center"><div class="footer__copyright"><div><img style="height:50px" alt="Apache Software Foundation" src="/img/incubator-logo.svg"><p style="color: #999999; padding: 0 20px 30px;font-weight:400;text-align:left">Apache Linkis is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p><p></p> |
| <p style="padding: 0 20px 30px;color: #999999;font-weight: 400;"> Copyright © 2022 The Apache Software Foundation. Licensed under the Apache License, Version 2.0. Apache Linkis, Apache Incubator, Apache, the Apache feather logo, the Apache Linkis logo and the Apache Incubator project logo are trademarks of The Apache Software Foundation.</p> |
| <div></div></div></div></div></div></footer></div> |
| <script src="/assets/js/runtime~main.a4c60f50.js"></script> |
| <script src="/assets/js/main.42471b13.js"></script> |
| </body> |
| </html> |
