| <!doctype html> |
| <html lang="en" dir="ltr" class="docs-wrapper docs-doc-page docs-version-1.7.0 plugin-docs plugin-id-default docs-doc-id-user-guide/udf-function"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="generator" content="Docusaurus v2.2.0"> |
| <title data-rh="true">UDF Function | Apache Linkis</title><meta data-rh="true" name="viewport" content="width=device-width,initial-scale=1"><meta data-rh="true" name="twitter:card" content="summary_large_image"><meta data-rh="true" property="og:url" content="https://linkis.apache.org/docs/latest/user-guide/udf-function"><meta data-rh="true" name="docusaurus_locale" content="en"><meta data-rh="true" name="docsearch:language" content="en"><meta data-rh="true" name="docusaurus_version" content="1.7.0"><meta data-rh="true" name="docusaurus_tag" content="docs-default-1.7.0"><meta data-rh="true" name="docsearch:version" content="1.7.0"><meta data-rh="true" name="docsearch:docusaurus_tag" content="docs-default-1.7.0"><meta data-rh="true" property="og:title" content="UDF Function | Apache Linkis"><meta data-rh="true" name="description" content="1 Introduction to UDF"><meta data-rh="true" property="og:description" content="1 Introduction to UDF"><link data-rh="true" rel="icon" href="/img/favicon.ico"><link data-rh="true" rel="canonical" href="https://linkis.apache.org/docs/latest/user-guide/udf-function"><link data-rh="true" rel="alternate" href="https://linkis.apache.org/docs/latest/user-guide/udf-function" hreflang="en"><link data-rh="true" rel="alternate" href="https://linkis.apache.org/zh-CN/docs/latest/user-guide/udf-function" hreflang="zh-CN"><link data-rh="true" rel="alternate" href="https://linkis.apache.org/docs/latest/user-guide/udf-function" hreflang="x-default"><link data-rh="true" rel="preconnect" href="https://AE29KQB3IA-dsn.algolia.net" crossorigin="anonymous"><link rel="alternate" type="application/rss+xml" href="/blog/rss.xml" title="Apache Linkis RSS Feed"> |
| <link rel="alternate" type="application/atom+xml" href="/blog/atom.xml" title="Apache Linkis Atom Feed"> |
| <link rel="alternate" type="application/json" href="/blog/feed.json" title="Apache Linkis JSON Feed"> |
| |
| |
| |
| <link rel="search" type="application/opensearchdescription+xml" title="Apache Linkis" href="/opensearch.xml"><link rel="stylesheet" href="/assets/css/styles.05b34e03.css"> |
| <link rel="preload" href="/assets/js/runtime~main.00388f66.js" as="script"> |
| <link rel="preload" href="/assets/js/main.5c2f2a32.js" as="script"> |
| </head> |
| <body class="navigation-with-keyboard"> |
| <script>!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}()</script><div id="__docusaurus"> |
| <div role="region" aria-label="Skip to main content"><a class="skipToContent_fXgn" href="#docusaurus_skipToContent_fallback">Skip to main content</a></div><nav class="navbar navbar--fixed-top"><div class="navbar__inner"><div class="navbar__items"><button aria-label="Toggle navigation bar" aria-expanded="false" class="navbar__toggle clean-btn" type="button"><svg width="30" height="30" viewBox="0 0 30 30" aria-hidden="true"><path stroke="currentColor" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M4 7h22M4 15h22M4 23h22"></path></svg></button><a class="navbar__brand" href="/"><div class="navbar__logo"><img src="/img/logo.png" alt="Apache Linkis Logo" class="themedImage_ToTc themedImage--light_HNdA"><img src="/img/logo.png" alt="Apache Linkis Logo" class="themedImage_ToTc themedImage--dark_i4oU"></div><b class="navbar__title text--truncate">Apache Linkis</b></a><a class="navbar__item navbar__link" href="/download/main">Download</a><a class="navbar__item navbar__link" href="/community/how-to-subscribe">Community</a><a class="navbar__item navbar__link" href="/blog">Blog</a><a class="navbar__item navbar__link" href="/team">Team</a><a aria-current="page" class="navbar__item navbar__link navbar__link--active" href="/user">Users</a><a href="https://docs.qq.com/doc/DUkdTTGhVSlZ0VXVt" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link">FAQ</a><div class="navbar__item dropdown dropdown--hoverable"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link">ASF</a><ul class="dropdown__menu"><li><a href="https://www.apache.org/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Foundation</a></li><li><a href="https://www.apache.org/licenses/" target="_blank" rel="noopener noreferrer" class="dropdown__link">License</a></li><li><a href="https://www.apache.org/events/current-event" target="_blank" rel="noopener noreferrer" class="dropdown__link">Events</a></li><li><a href="https://www.apache.org/security/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Security</a></li><li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Sponsorship</a></li><li><a href="https://privacy.apache.org/policies/privacy-policy-public.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Privacy</a></li><li><a href="https://www.apache.org/foundation/thanks.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Thanks</a></li></ul></div></div><div class="navbar__items navbar__items--right"><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link">Doc</a><ul class="dropdown__menu"><li><a class="dropdown__link" href="/docs/latest/about/introduction">1.7.0</a></li><li><a class="dropdown__link" href="/docs/1.6.0/about/introduction">1.6.0</a></li><li><a class="dropdown__link" href="/docs/1.5.0/about/introduction">1.5.0</a></li><li><a class="dropdown__link" href="/docs/1.4.0/about/introduction">1.4.0</a></li><li><a class="dropdown__link" href="/docs/1.8.0/about/introduction">Next(1.8.0)</a></li><li><a class="dropdown__link" href="/versions">All Version</a></li></ul></div><a href="https://github.com/apache/linkis" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link header-github-link" aria-label="GitHub"></a><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link"><svg viewBox="0 0 24 24" width="20" height="20" aria-hidden="true" class="iconLanguage_nlXk"><path fill="currentColor" d="M12.87 15.07l-2.54-2.51.03-.03c1.74-1.94 2.98-4.17 3.71-6.53H17V4h-7V2H8v2H1v1.99h11.17C11.5 7.92 10.44 9.75 9 11.35 8.07 10.32 7.3 9.19 6.69 8h-2c.73 1.63 1.73 3.17 2.98 4.56l-5.09 5.02L4 19l5-5 3.11 3.11.76-2.04zM18.5 10h-2L12 22h2l1.12-3h4.75L21 22h2l-4.5-12zm-2.62 7l1.62-4.33L19.12 17h-3.24z"></path></svg>English</a><ul class="dropdown__menu"><li><a href="/docs/latest/user-guide/udf-function" target="_self" rel="noopener noreferrer" class="dropdown__link dropdown__link--active" lang="en">English</a></li><li><a href="/zh-CN/docs/latest/user-guide/udf-function" target="_self" rel="noopener noreferrer" class="dropdown__link" lang="zh-CN">简体中文</a></li></ul></div><div class="searchBox_ZlJk"><button type="button" class="DocSearch DocSearch-Button" aria-label="Search"><span class="DocSearch-Button-Container"><svg width="20" height="20" class="DocSearch-Search-Icon" viewBox="0 0 20 20" aria-hidden="true"><path d="M14.386 14.386l4.0877 4.0877-4.0877-4.0877c-2.9418 2.9419-7.7115 2.9419-10.6533 0-2.9419-2.9418-2.9419-7.7115 0-10.6533 2.9418-2.9419 7.7115-2.9419 10.6533 0 2.9419 2.9418 2.9419 7.7115 0 10.6533z" stroke="currentColor" fill="none" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round"></path></svg><span class="DocSearch-Button-Placeholder">Search</span></span><span class="DocSearch-Button-Keys"></span></button></div></div></div><div role="presentation" class="navbar-sidebar__backdrop"></div></nav><div id="docusaurus_skipToContent_fallback" class="main-wrapper mainWrapper_z2l0 docsWrapper_BCFX"><button aria-label="Scroll back to top" class="clean-btn theme-back-to-top-button backToTopButton_sjWU" type="button"></button><div class="docPage__5DB"><aside class="theme-doc-sidebar-container docSidebarContainer_b6E3"><div class="sidebar_njMd"><nav class="menu thin-scrollbar menu_SIkG"><ul class="theme-doc-sidebar-menu menu__list"><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/about/introduction">About Linkis</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/quick/live-demo">Quick Experience</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/deployment/deploy-quick">Deployment</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret menu__link--active" aria-expanded="true" href="/docs/latest/user-guide/how-to-use">User Guide</a></div><ul style="display:block;overflow:visible;height:auto" class="menu__list"><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/user-guide/how-to-use">How to Use</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/user-guide/linkiscli-manual">Shell Scripts Manual</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/user-guide/sdk-manual">Linkis SDK Manual</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/user-guide/datasource-manual">Data Source Manual</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link menu__link--active" aria-current="page" tabindex="0" href="/docs/latest/user-guide/udf-function">UDF Function</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/user-guide/datasource-client">DataSource Client SDK</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/user-guide/dynamic-variables">built-in time variable</a></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" tabindex="0" href="/docs/latest/user-guide/control-panel/overview">Control Panel</a></div></li></ul></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/engine-usage/overview">Engine Usage</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/auth/ldap">Security Authentication</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/api/overview">API</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/architecture/overview">Architecture</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/development/directory-structure">Development</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/tuning-and-troubleshooting/overview">Tuning And Troubleshooting</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/upgrade/upgrade-from-0.X-to-1.0-guide">Upgrade Guide</a></div></li></ul></nav></div></aside><main class="docMainContainer_gTbr"><div class="container padding-top--md padding-bottom--lg"><div class="row"><div class="col docItemCol_VOVn"><div class="docItemContainer_Djhp"><article><nav class="theme-doc-breadcrumbs breadcrumbsContainer_Z_bl" aria-label="Breadcrumbs"><ul class="breadcrumbs" itemscope="" itemtype="https://schema.org/BreadcrumbList"><li class="breadcrumbs__item"><a aria-label="Home page" class="breadcrumbs__link" href="/"><svg viewBox="0 0 24 24" class="breadcrumbHomeIcon_OVgt"><path d="M10 19v-5h4v5c0 .55.45 1 1 1h3c.55 0 1-.45 1-1v-7h1.7c.46 0 .68-.57.33-.87L12.67 3.6c-.38-.34-.96-.34-1.34 0l-8.36 7.53c-.34.3-.13.87.33.87H5v7c0 .55.45 1 1 1h3c.55 0 1-.45 1-1z" fill="currentColor"></path></svg></a></li><li class="breadcrumbs__item"><span class="breadcrumbs__link">User Guide</span><meta itemprop="position" content="1"></li><li itemscope="" itemprop="itemListElement" itemtype="https://schema.org/ListItem" class="breadcrumbs__item breadcrumbs__item--active"><span class="breadcrumbs__link" itemprop="name">UDF Function</span><meta itemprop="position" content="2"></li></ul></nav><span class="theme-doc-version-badge badge badge--secondary">Version: 1.7.0</span><div class="tocCollapsible_ETCw theme-doc-toc-mobile tocMobile_ITEo"><button type="button" class="clean-btn tocCollapsibleButton_TO0P">On this page</button></div><div class="theme-doc-markdown markdown"><header><h1>UDF Function</h1></header><h2 class="anchor anchorWithStickyNavbar_LWe7" id="1-introduction-to-udf">1 Introduction to UDF<a class="hash-link" href="#1-introduction-to-udf" title="Direct link to heading"></a></h2><p>UDF: User Defined Function, user-defined function. In some scenarios, we need to use hive functions to process some data. Functions like count() and sum() are built-in. If we want to use some functions that are not built-in, we need to customize the function, which can be done by writing UDF.</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="1overview-of-the-overall-steps-for-creating-udf">1.Overview of the overall steps for creating UDF<a class="hash-link" href="#1overview-of-the-overall-steps-for-creating-udf" title="Direct link to heading"></a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="1-universal-type-udf-functions">1 Universal type UDF functions<a class="hash-link" href="#1-universal-type-udf-functions" title="Direct link to heading"></a></h3><p>Overall step description</p><ul><li>Write UDF functions in UDF format locally and package them as jar package files</li><li>【Scriptis >> Workspace】Upload to the corresponding directory in the workspace</li><li>【Management Console>>UDF Function】 Create udf (default loading)</li><li>Used in task code (only effective for newly started engines)</li></ul><p><strong>Step1 Writing jar packages locally</strong></p><p>Hive UDF Example：</p><ol><li>add hive dependency</li></ol><div class="language-xml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-xml codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token tag punctuation" style="color:rgb(248, 248, 242)"><</span><span class="token tag" style="color:rgb(255, 121, 198)">dependency</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> </span><span class="token tag punctuation" style="color:rgb(248, 248, 242)"><</span><span class="token tag" style="color:rgb(255, 121, 198)">groupId</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)">></span><span class="token plain">org.apache.hive</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)"></</span><span class="token tag" style="color:rgb(255, 121, 198)">groupId</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> </span><span class="token tag punctuation" style="color:rgb(248, 248, 242)"><</span><span class="token tag" style="color:rgb(255, 121, 198)">artifactId</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)">></span><span class="token plain">hive-exec</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)"></</span><span class="token tag" style="color:rgb(255, 121, 198)">artifactId</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> </span><span class="token tag punctuation" style="color:rgb(248, 248, 242)"><</span><span class="token tag" style="color:rgb(255, 121, 198)">version</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)">></span><span class="token plain">3.1.3</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)"></</span><span class="token tag" style="color:rgb(255, 121, 198)">version</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"></span><span class="token tag punctuation" style="color:rgb(248, 248, 242)"></</span><span class="token tag" style="color:rgb(255, 121, 198)">dependency</span><span class="token tag punctuation" style="color:rgb(248, 248, 242)">></span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><ol start="2"><li>create UDF class</li></ol><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">import org.apache.hadoop.hive.ql.exec.UDF;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">public class UDFExample extends UDF {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> public Integer evaluate(Integer value) {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> return value == null ? null : value + 1;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><ol start="3"><li>package</li></ol><div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">mvn package</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>Step2【Scriptis >> Workspace】Upload jar package</strong> |
| Select the corresponding folder and right-click to select Upload</p><p><img loading="lazy" src="/assets/images/udf_14-30dbe0ef4eaa3b3fe8f41e1160896489.png" width="382" height="731" class="img_ev3q"></p><p><strong>Step3【Management Console>>UDF Function】 Create UDF</strong></p><ul><li>Function name: Conforming to the rules is sufficient, such as test<em> Udf</em> Using jar in scripts such as SQL</li><li>Function Type: General</li><li>Script path: Select the shared directory path where the jar package is stored, such as../..// Wds<em> Functions</em> 1<em> 0</em> 0. jar</li><li>Registration format: package name+class name, such as com.webank.wedatasphere.willink.bdp.udf.ToUpperCase</li><li>Usage format: Input type and return type must be consistent with the definition in the jar package</li><li>Classification: drop-down selection; Alternatively, enter a custom directory (which will create a new target level directory under the personal function)</li></ul><p><img loading="lazy" src="/assets/images/udf_15-5d5f233b161b22630220a7a24cac303e.png" width="1221" height="512" class="img_ev3q"></p><p>Note that the newly created UDF function is loaded by default and can be viewed on the <!-- -->[Scriptis>>UDF Functions]<!-- --> page for easy viewing during Scriptis task editing. Checking the UDF function indicates that it will be loaded and used</p><p><img loading="lazy" src="/assets/images/udf_16-296864434a4259a4a227f4107083f60c.png" width="672" height="635" class="img_ev3q"></p><p><strong>Step4 Use this udf function</strong></p><p>Innovative udf function using the above steps in the task |
| Function name is <!-- -->[Create UDF]<!-- --> Function name |
| In pyspark: |
| print (sqlContext.sql("select test_udf_jar(name1) from stacyyan_ind.result_sort_1_20200226").collect())</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="2-udf-functions-of-spark-type">2 UDF functions of Spark type<a class="hash-link" href="#2-udf-functions-of-spark-type" title="Direct link to heading"></a></h3><p>Overall step description</p><ul><li>Create a new Spark script file in the desired directory in the <!-- -->[Scriptis>>workspace]</li><li>Create UDF in <!-- -->[Management Console>>UDF Functions]<!-- --> (default loading)</li><li>Used in task code (only effective for newly started engines)</li></ul><p><strong>Step1 dss-scriptis-Create a new scala script</strong></p><p><img loading="lazy" src="/assets/images/udf_17-0c9b71c0552b6539b92db244d827b134.png" width="1223" height="564" class="img_ev3q"></p><p>def helloWorld(str: String): String = "hello, " + str</p><p><strong>Step2 Create UDF</strong></p><ul><li>Function name: Conforming to the rules is sufficient, such as test<em> Udf</em> Scala</li><li>Function type: spark</li><li>Script Path:../..// B</li><li>Registration format: The input type and return type must be consistent with the definition; The function names that need to be defined in the registration format must be strictly consistent, such as helloWorld</li><li>Classification: Drop down and select the first level directory that exists under dss scriptis UDF function - Personal function; Alternatively, enter a custom directory (which will create a new target level directory under the personal function)</li></ul><p><img loading="lazy" src="/assets/images/udf_18-9564956a4fefe087a7de4678be460db1.png" width="1222" height="575" class="img_ev3q"></p><p><strong>Step3 Use this udf function</strong></p><p>Use the above steps in the task to create a new udf function |
| Function name is <!-- -->[Create UDF]<!-- --> Function name</p><ul><li>In scala |
| val s=sqlContext.sql("select test_udf_scala(name1) |
| from stacyyan_ind.result_sort_1_20200226") |
| show(s)</li><li>in pyspark |
| print(sqlContext.sql("select test_udf_scala(name1) |
| from stacyyan_ind.result_sort_1_20200226").collect());</li><li>in sql |
| select test_udf_scala(name1) from stacyyan_ind.result_sort_1_20200226;</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="3-python-functions">3 Python functions<a class="hash-link" href="#3-python-functions" title="Direct link to heading"></a></h3><p>Overall step description</p><ul><li>Create a new Python script file in the desired directory in the <!-- -->[Scriptis>>workspace]</li><li>Create UDF in <!-- -->[Management Console>>UDF Functions]<!-- --> (default loading)</li><li>Used in task code (only effective for newly started engines)</li></ul><p><strong>Step1 dss-scriptis-Create a new pyspark script</strong></p><p><img loading="lazy" src="/assets/images/udf_19-0b4e863058c0df1e8930c7deab5d86a5.png" width="1074" height="594" class="img_ev3q"></p><p>def addation(a, b): |
| return a + b |
| Step2 Create UDF</p><ul><li>Function name: Conforming to the rules is sufficient, such as test<em> Udf</em> Py</li><li>Function type: spark</li><li>Script Path:../..// A</li><li>Registration format: The function names that need to be defined must be strictly consistent, such as addition</li><li>Usage format: The input type and return type must be consistent with the definition</li><li>Classification: Drop down and select the first level directory that exists under dss scriptis UDF function - Personal function; Alternatively, enter a custom directory (which will create a new target level directory under the personal function)</li></ul><p><img loading="lazy" src="/assets/images/udf_20-ded46e8ecfa81a1f86357290217a3823.png" width="1068" height="671" class="img_ev3q"></p><p><strong>Step3 uses this udf function</strong> |
| Use the above steps in the task to create a new udf function |
| Function name is <!-- -->[Create UDF]<!-- --> Function name</p><ul><li>in pyspark |
| print(sqlContext.sql("select test_udf_py(pv,impression) from neiljianliu_ind.alias where entityid=504059 limit 50").collect());</li><li>in sql |
| select test_udf_py(pv,impression) from neiljianliu_ind.alias where entityid=504059 limit 50</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="4-scala-functions">4 scala functions<a class="hash-link" href="#4-scala-functions" title="Direct link to heading"></a></h3><p>Overall step description</p><ul><li>Create a new Spark Scala script file in the desired directory in the <!-- -->[Scriptis>>workspace]</li><li>Create UDF in <!-- -->[Management Console>>UDF Functions]<!-- --> (default loading)</li><li>Used in task code (only effective for newly started engines)</li><li><strong>Step1 dss-scriptis-Create a new scala script</strong> |
| def hellozdy(str:String):String = "hellozdy,haha " + str |
| <strong>Step2 CREATE FUNCTION</strong></li><li>Function name: Must be strictly consistent with the defined function name, such as hellozdy</li><li>Function Type: Custom Function</li><li>Script Path:../..// D</li><li>Usage format: The input type and return type must be consistent with the definition</li><li>Classification: Drop down and select the first level directory that exists under dss scriptis method function personal function; Alternatively, enter a custom directory (which will create a new target level directory under the personal function) |
| Step3 Use this function |
| Use the above steps in the task to create a new udf function |
| Function name is <!-- -->[Create UDF]<!-- --> Function name |
| val a = hellozdy("abcd"); |
| print(a)</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="5-common-usage-issues">5 Common usage issues<a class="hash-link" href="#5-common-usage-issues" title="Direct link to heading"></a></h3><h4 class="anchor anchorWithStickyNavbar_LWe7" id="51-udf-function-loading-failed">5.1 UDF function loading failed<a class="hash-link" href="#51-udf-function-loading-failed" title="Direct link to heading"></a></h4><p>"FAILED: SemanticException <!-- -->[Error 10011]<!-- -->: Invalid function xxxx"</p><p> <img loading="lazy" src="/assets/images/udf_10-33213689efc48d062472b3e5eebb4f12.png" width="1569" height="161" class="img_ev3q"></p><ul><li><p>Firstly, check if the UDF function configuration is correct:</p><p><img loading="lazy" src="/assets/images/udf_11-45ff7cae6d835b1d3b98fbef8f7e60eb.png" width="508" height="618" class="img_ev3q"></p></li><li><p>The registration format is the function path name:</p><p><img loading="lazy" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAArUAAAA9CAYAAACz3JHdAAAd+UlEQVR4nO2df4wcZ3nHv3tOfKDlHMkmtyLIu/U2fxBhyajARCjcSZeglIgFFXGbPYSpL6QtEkRWk4lV+kOHs5UokpmjnCCSiULGzZXeZha5BUOQi7zqnUKUSdwm5ZD4w5n0xk2qObBFchkRm9jTP+bXOzPv/NrbvR97z0da6e59Z9553mfeH88888z7FizLsuBw/vx53HbbbchL/ZHz+LevfRT/8pNlJvW3wFvAb3//O/zj9/8Hv3jqcO5y83L9+nUAwNDQUCCN/T/reQSxU6H+QPSSS5cueX/v27ePmxdO3464/YaF+hAx6MTZWEnzCJvX6/nmhp6UAuD2+54GAPz28v9GM39n9OoyibgKCis5zbCNO48gdhpk0BK95NKlS57BeunSpcD/g4bbZ1jjluYUYqeSxa7qx3xT6IWndqvBewpgSVIwDUDEToUMWqKX8LywcWmDaOjSfELsBNLmjbg36C697iMD0ePChivviTmsUPbHO4YgBolwW+flA9QHiP7Cem1Zwv8PAtSXCCLZHutHH9n2vS5uouYpa2hoyPtlKYMgBoHw69A8fYYguiXO+xo2bPft2zeQnlqCGDR48wfPaM1Cv+abgZ7F0uJo44xcghgUWA9sWn8giPXgxsxm8cKSEUsQ24sk50iaYbuR88u2n8nSlNXtUwRBDApsH6H+QPQDNy6W54Vl/ycIYvuR1TnCHrtZbHujNgs0kRMEQWwM4XACMmwJYvvC+94iLk6WTQ97djeKni3ptVF0+0ELLdtFEEHoI0min4RjZsOhCRSCQBDbl6TVpdz0zZhjtp1R2+1atOy5BLFTiVtehR72iLwkLcXFGq/h0ASCIAabzZxLtsUsljUombdMVxiauIlBJenLVPZ/XhpBdEPYeHVxjVgKPSCI7c1Wj6ENs21ns27XoiWIQSTrsl0E0StYg5WMV4IYXLaT4yOTpFKxiKKkphylQmoo6MeGuFzPrCphZGTE/k218Xx7FooB/PoHn/XTR0YwMjKL591zVQnFYjH0a0AxHPmZ9GB1g3ner6HAgAGlwclL1Rci8rzrXe/CyMgIphQjkOemj4xMQTEAQ2lEZTTamOKcm14ft/5ReQJ5OfXT8E709WOf4//vHxNHjG6LElRG1oZieDopFouQZClQfiDPlZuta0OBqkgB3cbpT5U4srASs+f7J6XLE4crZ6itsefFf5nq35fodZLy4mDvhwSVve8NBYahoBHbDtJ1G3tVpeH1gXdN8cYYtv1JUJx7mbVOZxtFnGgoWM16yoCwqjRwotjA2RyDdthojRq2L+PMpz6FU0+q3OP6LV8vyXr9VaWBhcx9qJ+oWCgWcaJY7Js8y1IRJ4oSlgOpTh9yrn0iw4Bil1PESXvQxckir1w+9n2J6bOq5MiRrSxbfPf6juxeGbZ8rqz+jy3b13ngx8rmlH8y+6C0rdlsp0omo1Y0O2imHiVAbNVRWrdINobSiDd6DAUNqQzNNGGaJtZEHXfeZ3ekUr2FTrOJjpNnahXMfvMFe7IXRGhyE7Lm5JkmNFnw5TdNmJ0mmp0OsBTqmDUZmmlCk2todkyYnk5KqLc6aDY7XplmJn25xWowTQ1yrYlzb76JtXOPOOL4sr755ptYW1vDhScEp45zkJs1qIyM6rwC6/j38K16iVNPDWXJNb4MKA0JZTdPm0S76uRFzhOhH3WNiCT9CBA1GU1Z83Qg6kcdw8LRT03GYcH5f84+tlVPay083ZroNF1Z7XvRqpdQqrdgOjKI03bemel5qE6b8PKEaPsxRR0T0359vPvL3HOvpqIrg9vGRLgtCKqEqi76cmLCbsNCijxJCKJ9rVYdJUYf7nnJX6Z+yLtnnIK9PMvKOgD591IzRQhOGZ58pTpanaajP7cdVD2jNdI3Y2ULXbXewptvvozvHf8ezn1Gx2JgWDCgTM0y7bmM9nSeGb2EWwQAQgWjOc4aBEbrLRyR0xqgj2vQxn3wZRu2Amo//DvsY47rNpY2r3y9Juv1R+stTG2emAwCpkwTxzL0qW45KJq4J1y8Oo9XJjUcM037lzqoAQfFDvY3O/hivQSU6vhip4n9HREHM8gwWm/hmKnhEO8ygohjJkfGJJzr75E1W3ZBxDHTl++gqOFQU8YRt35aGc94RquAKU3Gfpmpf1i2UgU3Abip0ivraOuyFXamzHHlFd9LE/LIet6rQHrYq8N6ahwPYMAz2Ah4V0r1FkS96qWz3lp1Xg8Y0EO3P4y1tQV85mbO5FwaxyRWot4dVYKkAqW6CNa2UpeAMUHAGJYYDxzPYOelGVAkBQYEiBk6NgQxath9+CEshNNcWScfYmSdhFhesnVmKJAwicnYC5UwPgmsGADUebQn5/xySnW0OoDEfYAQMCboWGFF4eqHc+aYAN07UYAo6jhqCwvlqI6xVIM2jG2MqwAEkTEkE2jKZUgcr0G4/UAQYZot2ONrK2Jo8tK48klAhzlQEDVMtuc9PcXJsy6en8XIyEikDyUNKoHQHMviHh/wqq7jDYwgch4QnfIl+2Zm0C0AYwl6eRzCeBn6YliaD6PCtmfnXuZhT7kSSQt4aNj75npyWK8OL79YxElFsT05me570Ovjeduc8hYUxqMU8FIFPWVRjxBbbgNnFSnoyVuUotfkyNM+97aXw65mwAs/eHN1Bee/VMb3yuWIR83ztPG8bUZcHW1i70lcuZ6XTGHqksOLl6QfRtZAesjTF9CjpKIbzyZXP5Lq1DlnfQKogfbp6jDQhpj6hGVdloo4MTGDN6ar3emWyzr0k0JS+8lMqY5PT+p4MWpUYKGhYBUl3C3WQw/JNeyNDjHw7KQ+veXeSLaCQQvkMWpnpqE7Xiptso15pj3Y3isNwYdax6uDJjqON6tUb0GTa6jJc6iXVEist8wUoVclPMe4ru1yRehV9nW+gRVOW4xdi1adxzQqjPGpYrpaRHFihlNJFUsYgwBAGOPOxQn6mUCxWEUuJ1EqSbICFWeCNxZ1TB4eTyxpRQcqJcBYASbHQzO+MAZBX4meZCiQ1DL8vphVPwYUSUXAThBEiPpRNBpV6BmNUg9Xt2fynASgUoeI8KtofvtZPyvQhbFQvZiHiVh5sjCDCe/hbwJ+a1AxK5VxYW0t0IfSqhfw6hYKKBRCw0DI42yKOqpdTywVlAMPlXZdqjlvprGkozxeAkrjKOuLTHkl1L9V5ofTZGS0XIukrSoNPFP2vS9HypI/0buenJkJnJDKOGKaOFJesl9TGwpOOmnHTBN36NO4WJNxJJPlXsHBjuZ5e/ZKjoEgiDgi13BxWscdnjdMxynvnpRwi9jxZL1DP8q8MjdwtiFhr+Z6kURcnmbHExUvtV15O4DkG5LLEnueiTv0Kv795WC4AS+W9tIlADPTuCw6upts4xzTfEYrIu5h6nHaj3/CQjVYx6eZATXxnqgSTuki4y109ON44d6Ybvt1CXja0ojXD0p1fJHnGXXu1x5Zsz2REDBldrAfTdwjClhVjnq6idQjEQNnj/r6OVKWcGoaOKRl83DycWWzGa23gvUJtecjZQlPM83noGjXf4/nqcwhy8yEb2Ayc1z3+kkmsf3kZLQCXGamzNenqzhRnMBF7tEV7I0OMQPFVjFogTxGLfPKs1QRoK5kaQwCDssq4wVUMT8tQKzb1pV6ZhrVwGQ9g6Xn7SP9uAz7FadWljAy8lm0DcByvEuRyjgKtSzGCJDK0AITimC/Xue9orHdkM5hOa3aZodj2K+XeFkLhYI9wberqLbLCNupnkHsxi2WD2c0JJnzqiGPZop+1Omqc70qdDHqLRNEEYLQyeaZY3F128XAIByeRHu+L1ZsV3QnD/vKngltMVbw/I/vw62ep9bpQ5x42zBJO8OoS0GPMwQRnRTPfN66aLlupoHFlf1OGy9hvBwKQSjV0Qo9HOeRdbTecowP/3ov6pP4NJM2Wp/DAX0xaAg1OzjWsj0yo3URd5cArOi4ifHSHBR9gyELr0mux6uKl0J2/x75sG8wCCLuwZLvFVvyDQTW6ICxiMuTc7Zs9omYMk3mdbmAQ3OuvBXshe7U0cBr6hm8VC0Gyr24pHJ3DYss2dXseNcYrQh4PTBfLOFpjjHjNLxgHf3GnnhPlpeAe0JtNqCfpujrINbTxiNOP8mM1kXcND3vXX9VkfC6c/9W9TO4OOHr9dT0GbzRXkwv11jE5Um/DYzWW/les3fB6qKOA3N+ex6tz+FQrwy0pv8gxhrSXesnkYx9uktukjX7QZebW8LdrRbTB4N59ZYJs4ehm5vFVjBogQ1Y/aBUFyE4sYSGIkGVfeNKYGIw3d/Dtw9FVjEwlIbjOWrh3ve8BxVBRZxNbZ/7iGcEdIQ2Im8rAe5rT3VpBjMTjJE9k3ciL6Eudtk4Cwl5HFltw76E+lwHnTneNYVA7LAb5lCqAO2wQlhjlTlPk9WAgZSmH+9+anIg3rc3lFBvBT28pcpk0Bu8okfvV6mOufISFO+pupTYfrqngrIabi8GFvXQA0dEnvXB60O5HxpyUUF5kn3zkeb5XoEeeFNiky2kw73EItoz/gNwdXom2oY9BByW0Yf723/CHqqwwRIbk6dKQQ9Uj56sd9cUTOo6Eyvox0vGrXZgG7dJpapYYDx/xzQZN/VE2mR44SX9RcCdsopnnPCDc9MC7vAMqqbvqXZ/rfDr6p3MJuinUsYBtn8ZK3g94fDVFXDCCQRM7bD76NpqW8WgBTZkSS/XW6v4XlrA8TLOxxqNtifpBcyOjKC68hDWHvyQl/fhz01CefI55mg/3hJwvJju1cUUz40qOR+kqVhC8IMkkx8OmIGgPFkooBC1awuhFE9WhpIAIY8VLRzGZNv/eAeGgsYEY9OyRdfnUJb81SEy66dUx1yZI2uvqSDg9eSGVgAo1cegS23v/6i3NP/9ilJCXQQmmNgoVaqiXR7nGHRBebJiv72wYFmW/ffNH03sQ/FlJH8YFnXC+2EnQAkVnQ2h4IVd+BiKxG9cOTAWdUxqrOGuYdINQTAUTE21A/FoKzrwB+tye5TwgXLIk2cs4nJ5PH3CqpTx+hL7ylyKvpJ0YhTDrz5XdWaiNBQ8E4o6usi++oaKZYzZnr8VlTF4VZxjY6BK49jbng/EOS5LWb7ELqGy7zn88uXoh2Eu+/a9jGfvuw+nvvqT7N4uYwWvMx/lLc9P+8aDMBZ8vR/QQfI9OTgGLIfarKsfAHhjmtWBgRf1Mj7AtBE3lrSXqwa43tqzjJcWjqzPpOifK09pHHvZLxw4bSQZN1aVE/eqrjh6V7HAeM9Hx8t4hR0r1fnIG4RknJjdHKuLZNFPflL6dKmCy2zbW9FxU+y4peJcO9h+usZdMabX31psIFvJoAUAWAwvvPCCZZpm5NcELABWTdYsU5OtmvN/s2Naptnx8v1fzZI1pgznnJqsBctmygqf12n6/6+trQV+pmlaF574ROCatiympcm1SJrZaVpA0+p0miE53Xr9wK9DTba0UDm23KF6OseZpmbJtWiZQNPqcHQZ+TkyFQoFa2hoyPrkKc1Ld9MKhUJAFls3jL5YPTY7Xpnce2GG7xkjJ3Oed6802arhDqueqJ+o/ly9uOX4MsfJxPuFdNvsRI5h5fDyOfXQ5Fqg/QXOY9tKSl6wHtH7HG032eSJ+3nlfeIJ68LaBU8fn3jigt0fLjwR24fCfQSAhUfOcesY1G+4TYfvF9t+mLxIfw7es+A1s/UP7dQnvT4QHW+aVkeTrb//5Ce9PlIoFPw+tK5fx3ocsI57v6b1nJP3ilxj0mEdR806zegnkN9sWo/XZOuVUJ8/DliznPFwlrne401Yx51zX5Fr1uPNZsw1WVlr1uPNWkDeYLn+dX057WOfazrHePJq1o8fvss6DliP3nWX9ehdd1lPKMsRHT36sGIt63pElllnvnCv/bjTj7zrANZss2nNMnmubly5TstsXeLviWlq1ulazD3pNL3rHA/JEr5nbHqafqLtANbx0Bhln8PKGdUB7zyePNG217ROy02vnunyuDpKkqdmnZabgWsHyq3J1mmvXYZ1zivbuWeBPuCfF2wj/j2L10+4DbDn8eRhdZjUfkJtj5E3Ikts3+P06Szjuzdn0q8Xv4LFBKieP38et912W2aDeKOJ22eYIAaduG1skwL02bytFMi/s1CxIAFT64wJWVUaOFfp/9JRvCW44jZX2Dbb3qoSTq4cDsVNDwarioQXx8WYeE1ia2NAaRwF5vKv1tItO2E79G1l1LqkbQVKEING0mAUZ7BSP9kkDAUnq9N4AwBQwyEt7iORjKiS/0FVTcaRPsbtsR9/sWnuEl5u3rYxbgP3ool78nydv5UZ1HoRfWUnODe2pVHrshOeOggCSB+MePlxW0gTRBJxhm04jSCI7UU3c8J2M4S3h5QxbBclE0S/4a3TvBW/TCW2PnErGxAEsf2JXdN/QKDZjiC2EHGrE3Q7EJFBSyQRv7JBdC1agiAGgzzzSdKa5luR7SMpQQw47ADDM27TBqLtNPAQW4Mkryx/tzDy3hLEIDCoHluaBQliC8DGLWUxTgdtICI2Ft6HXkmGrfs3eWwJYvsSnlsG0bAlo5YgNhleIH7cYMMavWFv7iANTET/yWrYJqUTBLG16DZEbVDmDzJqCWKLkuSxDa9ysN2+UCU2l6R1Z9M8tlkovHoRhVcje6kRBNFH3Hkgy86RLINk2N6w2QIQBNEdZMAS68Fdb5bdXIFNY49LY+hXv8TQcz/Hrl+8iKHnfo7Cb1Zx5bv/jGvv3d83+QmCCMJutAPkW47LPXe7L5VKRi1BbDLhgSjMdh9kiM0nbRWDsGGbxq7nn8XQcz/H0H//F3apzwJX3grkWyN7cE34yDqlJgiiW9h5JescMgjzDBm1BLEFGITBhNiasDuChbe8Zf/mbZELAIW1NwIG7NBL51OveX38TmB4uLcVIQgiFZ7HdSeFqJFRSxDbjJ0yOBG9JRxaEF7ZgDVsC69exI1PPo6hxQ6GXrmQ+1pvT9zdM7kJglgfYa+tmzaIDGatCGJAGfQBiegvSaEFbJ713v34/Zcfwtt/+me4fuDWfBcZfgeuj090KyJBEOsk/OFX3Dbqg/BhWBiaGQlim0AGLdELsq5yYI3swdtTn8dbP/kPvLXwI1w/9MFM5V8TPgJrZE9vhCUIIoJrkLK/MGHDlt02fZDnkMGtGUEMEGTQElmJ2/qWJW1d2gjDwxj61S8zHXrtzj/OViZBELmJ26gnyevKmzcG1bgdvBoRxIAyiAMQ0XvitreNW6orzbAt/ObXGH7g/sgKB3Fc+9jH84pMEEQOwhv1xG3IM0jrz2aFZkmC2MKwr4wIIokkg5UNK2AN3NStb69cwfAD92feSOH6oQ/CevfNXUhPEEQaeb2xOxFa/YAgtjg0WBHd4K5oAIC7XFeW0IPhv30o0xJeLtfuotADgugXaWua85bzGpRNFbIy+DUkCILYoWRd7YDHjY99B7t+/K+5rkehBwTRf7J4bHdSyAELGbUE0WN26mBCbE1yfxQGYNfPfoobZ7+W6Vjr3aMAgOsHbsX1A3+YX0CCIDLTjdE6qB+F8dgZtSSIDYYMW6LfZFnlwCWPYTv0ysvY/TcPphc6/A5c/erXcXX2UQDkpSWIjSZuntkpBiwPiqkliD6xU2KYiI0naetbl3B4QVq4AWBviTv8559DYe2NxOOuH7gVV2cfxfX3vR+4cgUYfgfF0xJEj0had9b9m91YgeYZH9IEQfSBvK+IBnV3F6J/8IzZuFUOMnHlCnY/8IXUlQ6ufeJPcKX1I9ugBYDhYVz72Mdx/dAf5a8EQRABsq5Dm5S3k+cS8tQSRJ/YaV+dEhsHu7JBL1Y5AIDdXz+OXeqzfkIBgMUcMPwOXP3Kcbw99fnIuVcf/Eou+QmCiCfslU07jucU2alzzs6sNUFsEFk9tjspkJ/oHTwPbZgsYQc3nHoMNyz8k58QMmivH7gVby38kGvQAoD13v2Z5CUIonuStsTdKdvgprFza04QfSI8oOz0JVaI9cMLI4iLmc2zygEA7Hr+Wez++nE/IWTQRsINCILoK7y5gk2jD8TiIQ0QRBfkNVDJsCXWA2uwhj8G62brW5fCqxex+8tfYBLgG7TO6gZXvvEdWCN71iU/QRB8sswJO33r2zyQUUsQOWG/Os27ViB7PkHkgd3SNmzchg3b7CsdHPZXOmAM2rRwA4Igekea0RreLp3mknjIqCWInIRf8eQxbmkwItZL2LjNYsDy2C1+CUOvXLD/YQxaCjcgiI0jyYjlpYWhuSQIrX5AEOugm/UCKe6J6AXhUIM8Bu7urx/HrqWO/Y9r0CasbkAQRP/grZSTZfUcmkuikFFLEF3AG3BYjy0NNsRGkddTe8PCk7jh1GP2P45BG9hMgSCIvhFnpKYZtu7/bhkEH5p5CaJHhBfDpoGH2GoMvfSf/koHjkFL4QYEsTGkfY+RFnZAjpN0yFNLEF0SfrJmBxt28HLTCGIzKbx6EcMP3A9cecs2aG/cjat/3aRwA4LYINi5AYj/GIznsSWyQUYtQXBI23ubTeM9PdNrImJLceUKhh+4H4XfrAIF4Pot+3H124+Td5YgNpgsDhDajbJ7eqQtFVKxiGKxiGKxAcVwkyUnrYiGk6hKznGSmrE85tdQYAQOk7xyty5OXRLru/EsSydwongCJxpnsZr3ZOMsThZP4mxI9avKSbvM4gkshKrrXa94AieKC1hej/B9Juve24bSQLHYQNvRQ5zRS4MSsdkMP/wlDP3ql7ZB+9EJXDl9lgxagtgkWKeHO0fEbXlLjpF89GC2NaA0JJQ1E6ZpwtQm0a5KUAFAEKHJTciaiVa9BAAQxppAswNTFJKLrcnQTBOaXEOzY8I0O2iGjxFEr9yNwFAk5LehBYg82TcTdQHPlI/gmHkMx8TLOK3kMWtXcfboZRyQbwqlL+OcfoddpnkEe6Ww4bof95jHnPwpHFx3JfpLeO9t3oBTqregyQIK5JUltjA3zv4Ddv3sp8DQEH7/F0fx1nfnaTMFgtgAkuaEuPhZnhOE5pbsrN+oVefRnpyDZ1uW6mh1AIlj/RlKA0WpDC3NoIUAsVVHKTbN9+QGPbV2ekOS0CgW0VBUKI1473EgPREDSqOI6vQMpqu8cw3nOjyZODpgvc5x8qgSisUGJKnh5UedvY4ewh7sFJZX9uLT9VH7H+FOHNBfzO6tVc/hsjiFD0Qq9hpQdsrEKD4wCbzWEyd68C2AJLEPFkGPfljvnq6LRRQlBYrk6ymQx+jv/566FyMjs5Ck6D0ZGhoCjDamRka881bC0kqMPJLE7QcEsVHsev5Z3PjYt2G98524+o3v4Pd/+VebLRJB7AiybNITF6YW9t7S277srFtTxgowOR7ylgpjEPTgdK9KRVR1EWbEWO0GAaJpwtRkCOF0TQZmANE0IeoT0EXbe6wvGgBUSFIZmul4lU0RuutVTqSEesv0vM72uS3fkEcJFbHjpJsQ9aMJxrJgl9Gqo5QkjyDCroro5HUAKZ/xymcVr+ns/6O4BZczGrXLWJD24k7eM0npbhzUT3khBqf0g7g7cKMv4ukuwg8MZcl/C2CKwIx/t1SJeUNgmhD1qm/4q5Ld3pw8rdzGdEyeKeqoOie+596ncO6Rr0Ita/6bh3nvRHzz1osQ19awtrYGU9Qx4RTqGryL+y/48mAmQ9siiP5x7cMfwfX3vR9XH/s+3r7nU5stDkHsGLJu0pP2/QUZtPnYAG2pmK4WIZVlNGeWNmSSF+TDjrHbxBhrgBkrUM9Mo+p59yYwgxks9UKopQnPQzcxwz/ENuzHfGM4VR4B8mG3AhWUoYc8g45x35MHhXRWlWXsnbsbo/xcYMwJaTCP4cgYAobyQfGYl3dM24tnpGxmbWm8jHbV148qu28FDKyoZxjPua33GUd56hLQYd4IlOotT0/hPAgiOmDb5iN46DM3OydWIKgrjld9CeiIuN0dhD70IDpsXElpDOX2rb48qoy5DQyPIQgeb50+i2sfTHs7RhBEv0j6NoPNpzCD9bNuo7ZUAdqLIf+hugTWmmx2TLTqdYgxYQkbiSBrvofO9aitd7xXJUhlv1xN5hQ4M4GJmWbQmOqXPImM4pYy+/8qXsPeGEM1yKp+ES9VHU/s9EW8VGU8ruo5LDOljOI1vBh3q0u3IByRG0upjpanGw2T7Xnf+BRkxsvtel3Xr7xCoQAg/wAzNPQe3PuUL4s22cY8uWqJAYHWXiaIfPA+CHP/54UcuHlE9/w/tJTWCgzOLSYAAAAASUVORK5CYII=" width="693" height="61" class="img_ev3q"></p></li><li><p>Check the scriptis udf function to see if the loaded function is checked. When the function is not checked, udf will not be loaded when the engine starts</p><p><img loading="lazy" src="/assets/images/udf_13-4e1b68c3e49633e89e640a28b565a24d.png" width="351" height="506" class="img_ev3q"></p></li><li><p>Check if the engine has loaded UDF. If not, please restart another engine or restart the current engine |
| Note: UDF will only be loaded when the engine is initialized. If UDF is added midway, the current engine will not be able to perceive and load it</p></li></ul></div><footer class="theme-doc-footer docusaurus-mt-lg"><div class="theme-doc-footer-edit-meta-row row"><div class="col"><a href="https://github.com/apache/linkis-website/edit/dev/versioned_docs/version-1.7.0/user-guide/udf-function.md" target="_blank" rel="noreferrer noopener" class="theme-edit-this-page"><svg fill="currentColor" height="20" width="20" viewBox="0 0 40 40" class="iconEdit_Z9Sw" aria-hidden="true"><g><path d="m34.5 11.7l-3 3.1-6.3-6.3 3.1-3q0.5-0.5 1.2-0.5t1.1 0.5l3.9 3.9q0.5 0.4 0.5 1.1t-0.5 1.2z m-29.5 17.1l18.4-18.5 6.3 6.3-18.4 18.4h-6.3v-6.2z"></path></g></svg>Edit this page</a></div><div class="col lastUpdated_vwxv"></div></div></footer></article><nav class="pagination-nav docusaurus-mt-lg" aria-label="Docs pages navigation"><a class="pagination-nav__link pagination-nav__link--prev" href="/docs/latest/user-guide/datasource-manual"><div class="pagination-nav__sublabel">Previous</div><div class="pagination-nav__label">Data Source Manual</div></a><a class="pagination-nav__link pagination-nav__link--next" href="/docs/latest/user-guide/datasource-client"><div class="pagination-nav__sublabel">Next</div><div class="pagination-nav__label">DataSource Client SDK</div></a></nav></div></div><div class="col col--3"><div class="tableOfContents_bqdL thin-scrollbar theme-doc-toc-desktop"><ul class="table-of-contents table-of-contents__left-border"><li><a href="#1-introduction-to-udf" class="table-of-contents__link toc-highlight">1 Introduction to UDF</a></li><li><a href="#1overview-of-the-overall-steps-for-creating-udf" class="table-of-contents__link toc-highlight">1.Overview of the overall steps for creating UDF</a><ul><li><a href="#1-universal-type-udf-functions" class="table-of-contents__link toc-highlight">1 Universal type UDF functions</a></li><li><a href="#2-udf-functions-of-spark-type" class="table-of-contents__link toc-highlight">2 UDF functions of Spark type</a></li><li><a href="#3-python-functions" class="table-of-contents__link toc-highlight">3 Python functions</a></li><li><a href="#4-scala-functions" class="table-of-contents__link toc-highlight">4 scala functions</a></li><li><a href="#5-common-usage-issues" class="table-of-contents__link toc-highlight">5 Common usage issues</a></li></ul></li></ul></div></div></div></div></main></div></div><footer class="footer"><div class="container container-fluid"><div class="row footer__links"><div class="col footer__col"><div class="footer__title">Linkis</div><ul class="footer__items clean-list"><li class="footer__item"><a class="footer__link-item" href="/docs/latest/about/introduction">Document</a></li><li class="footer__item"><a class="footer__link-item" href="/faq/main">FAQ</a></li><li class="footer__item"><a href="https://github.com/apache/linkis/releases" target="_blank" rel="noopener noreferrer" class="footer__link-item">Releases</a></li></ul></div><div class="col footer__col"><div class="footer__title">Community</div><ul class="footer__items clean-list"><li class="footer__item"><a href="https://github.com/apache/linkis" target="_blank" rel="noopener noreferrer" class="footer__link-item">GitHub</a></li><li class="footer__item"><a href="https://github.com/apache/linkis/issues" target="_blank" rel="noopener noreferrer" class="footer__link-item">Issue Tracker</a></li><li class="footer__item"><a href="https://github.com/apache/linkis/pulls" target="_blank" rel="noopener noreferrer" class="footer__link-item">Pull Requests</a></li></ul></div><div class="col footer__col"><div class="footer__title">Subscribe Mailing List</div><ul class="footer__items clean-list"><li class="footer__item"><a class="footer__link-item" href="/community/how-to-subscribe">How to Subscribe</a></li><li class="footer__item"><a href="mailto:dev-subscribe@linkis.apache.org" target="_blank" rel="noopener noreferrer" class="footer__link-item">Subscribe Mail</a></li><li class="footer__item"><a href="https://lists.apache.org/list.html?dev@linkis.apache.org" target="_blank" rel="noopener noreferrer" class="footer__link-item">Mail Archive</a></li></ul></div></div><div class="footer__bottom text--center"><div class="footer__copyright"><div><img style="height:50px" alt="Apache Software Foundation" src="/img/incubator-logo.svg"><p style="color: #999999; padding: 0 20px 30px;font-weight:400;text-align:left">Apache Linkis is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p><p></p> |
| <p style="padding: 0 20px 30px;color: #999999;font-weight: 400;"> Copyright © 2025 The Apache Software Foundation. Licensed under the Apache License, Version 2.0. Apache Linkis, Apache Incubator, Apache, the Apache feather logo, the Apache Linkis logo and the Apache Incubator project logo are trademarks of The Apache Software Foundation.</p> |
| <div></div></div></div></div></div></footer></div> |
| <script src="/assets/js/runtime~main.00388f66.js"></script> |
| <script src="/assets/js/main.5c2f2a32.js"></script> |
| </body> |
| </html> |
