| <html lang="en" dir="ltr" class="docs-wrapper docs-doc-page docs-version-current plugin-docs plugin-id-default docs-doc-id-architecture/feature/public-enhancement-services/context-service/context-service-search"> |
| <meta name="generator" content="Docusaurus v2.2.0"> |
| <title data-rh="true">CS Search Architecture | Apache Linkis</title><meta data-rh="true" name="viewport" content="width=device-width,initial-scale=1"><meta data-rh="true" name="twitter:card" content="summary_large_image"><meta data-rh="true" property="og:url" content="https://linkis.apache.org/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-search"><meta data-rh="true" name="docusaurus_locale" content="en"><meta data-rh="true" name="docsearch:language" content="en"><meta data-rh="true" name="docusaurus_version" content="current"><meta data-rh="true" name="docusaurus_tag" content="docs-default-current"><meta data-rh="true" name="docsearch:version" content="current"><meta data-rh="true" name="docsearch:docusaurus_tag" content="docs-default-current"><meta data-rh="true" property="og:title" content="CS Search Architecture | Apache Linkis"><meta data-rh="true" name="description" content="CSSearch Architecture"><meta data-rh="true" property="og:description" content="CSSearch Architecture"><link data-rh="true" rel="icon" href="/img/favicon.ico"><link data-rh="true" rel="canonical" href="https://linkis.apache.org/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-search"><link data-rh="true" rel="alternate" href="https://linkis.apache.org/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-search" hreflang="en"><link data-rh="true" rel="alternate" href="https://linkis.apache.org/zh-CN/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-search" hreflang="zh-CN"><link data-rh="true" rel="alternate" href="https://linkis.apache.org/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-search" hreflang="x-default"><link data-rh="true" rel="preconnect" href="https://AE29KQB3IA-dsn.algolia.net" crossorigin="anonymous"><link rel="alternate" type="application/rss+xml" href="/blog/rss.xml" title="Apache Linkis RSS Feed"> |
| <link rel="alternate" type="application/atom+xml" href="/blog/atom.xml" title="Apache Linkis Atom Feed"> |
| <link rel="alternate" type="application/json" href="/blog/feed.json" title="Apache Linkis JSON Feed"> |
| <link rel="search" type="application/opensearchdescription+xml" title="Apache Linkis" href="/opensearch.xml"><link rel="stylesheet" href="/assets/css/styles.fee819ac.css"> |
| <link rel="preload" href="/assets/js/runtime~main.36b81271.js" as="script"> |
| <link rel="preload" href="/assets/js/main.f09c65a0.js" as="script"> |
| <body class="navigation-with-keyboard"> |
| <script>!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}()</script><div id="__docusaurus"> |
| <div role="region" aria-label="Skip to main content"><a class="skipToContent_fXgn" href="#docusaurus_skipToContent_fallback">Skip to main content</a></div><nav class="navbar navbar--fixed-top"><div class="navbar__inner"><div class="navbar__items"><button aria-label="Toggle navigation bar" aria-expanded="false" class="navbar__toggle clean-btn" type="button"><svg width="30" height="30" viewBox="0 0 30 30" aria-hidden="true"><path stroke="currentColor" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M4 7h22M4 15h22M4 23h22"></path></svg></button><a class="navbar__brand" href="/"><div class="navbar__logo"><img src="/img/logo.png" alt="Apache Linkis Logo" class="themedImage_ToTc themedImage--light_HNdA"><img src="/img/logo.png" alt="Apache Linkis Logo" class="themedImage_ToTc themedImage--dark_i4oU"></div><b class="navbar__title text--truncate">Apache Linkis</b></a><a class="navbar__item navbar__link" href="/download/main">Download</a><a class="navbar__item navbar__link" href="/community/how-to-subscribe">Community</a><a class="navbar__item navbar__link" href="/blog">Blog</a><a class="navbar__item navbar__link" href="/team">Team</a><a class="navbar__item navbar__link" href="/user">Users</a><a href="https://docs.qq.com/doc/DUkdTTGhVSlZ0VXVt" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link">FAQ</a><div class="navbar__item dropdown dropdown--hoverable"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link">ASF</a><ul class="dropdown__menu"><li><a href="https://www.apache.org/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Foundation</a></li><li><a href="https://www.apache.org/licenses/" target="_blank" rel="noopener noreferrer" class="dropdown__link">License</a></li><li><a href="https://www.apache.org/events/current-event" target="_blank" rel="noopener noreferrer" class="dropdown__link">Events</a></li><li><a href="https://www.apache.org/security/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Security</a></li><li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Sponsorship</a></li><li><a href="https://privacy.apache.org/policies/privacy-policy-public.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Privacy</a></li><li><a href="https://www.apache.org/foundation/thanks.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Thanks</a></li></ul></div></div><div class="navbar__items navbar__items--right"><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link">Doc</a><ul class="dropdown__menu"><li><a class="dropdown__link" href="/docs/latest/about/introduction">1.5.0</a></li><li><a class="dropdown__link" href="/docs/1.4.0/about/introduction">1.4.0</a></li><li><a class="dropdown__link" href="/docs/1.3.2/about/introduction">1.3.2</a></li><li><a class="dropdown__link" href="/docs/1.6.0/about/introduction">Next(1.6.0)</a></li><li><a class="dropdown__link" href="/versions">All Version</a></li></ul></div><a href="https://github.com/apache/linkis" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link header-github-link" aria-label="GitHub"></a><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link"><svg viewBox="0 0 24 24" width="20" height="20" aria-hidden="true" class="iconLanguage_nlXk"><path fill="currentColor" d="M12.87 15.07l-2.54-2.51.03-.03c1.74-1.94 2.98-4.17 3.71-6.53H17V4h-7V2H8v2H1v1.99h11.17C11.5 7.92 10.44 9.75 9 11.35 8.07 10.32 7.3 9.19 6.69 8h-2c.73 1.63 1.73 3.17 2.98 4.56l-5.09 5.02L4 19l5-5 3.11 3.11.76-2.04zM18.5 10h-2L12 22h2l1.12-3h4.75L21 22h2l-4.5-12zm-2.62 7l1.62-4.33L19.12 17h-3.24z"></path></svg>English</a><ul class="dropdown__menu"><li><a href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-search" target="_self" rel="noopener noreferrer" class="dropdown__link dropdown__link--active" lang="en">English</a></li><li><a href="/zh-CN/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-search" target="_self" rel="noopener noreferrer" class="dropdown__link" lang="zh-CN">简体中文</a></li></ul></div><div class="searchBox_ZlJk"><button type="button" class="DocSearch DocSearch-Button" aria-label="Search"><span class="DocSearch-Button-Container"><svg width="20" height="20" class="DocSearch-Search-Icon" viewBox="0 0 20 20"><path d="M14.386 14.386l4.0877 4.0877-4.0877-4.0877c-2.9418 2.9419-7.7115 2.9419-10.6533 0-2.9419-2.9418-2.9419-7.7115 0-10.6533 2.9418-2.9419 7.7115-2.9419 10.6533 0 2.9419 2.9418 2.9419 7.7115 0 10.6533z" stroke="currentColor" fill="none" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round"></path></svg><span class="DocSearch-Button-Placeholder">Search</span></span><span class="DocSearch-Button-Keys"></span></button></div></div></div><div role="presentation" class="navbar-sidebar__backdrop"></div></nav><div id="docusaurus_skipToContent_fallback" class="main-wrapper mainWrapper_z2l0 docsWrapper_BCFX"><button aria-label="Scroll back to top" class="clean-btn theme-back-to-top-button backToTopButton_sjWU" type="button"></button><div class="docPage__5DB"><aside class="theme-doc-sidebar-container docSidebarContainer_b6E3"><div class="sidebar_njMd"><nav class="menu thin-scrollbar menu_SIkG"><ul class="theme-doc-sidebar-menu menu__list"><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/about/introduction">About Linkis</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/quick/live-demo">Quick Experience</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/deployment/deploy-quick">Deployment</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/user-guide/how-to-use">User Guide</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/engine-usage/overview">Engine Usage</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/auth/ldap">Security Authentication</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/api/overview">API</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret menu__link--active" aria-expanded="true" href="/docs/1.6.0/architecture/overview">Architecture</a></div><ul style="display:block;overflow:visible;height:auto" class="menu__list"><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/overview">Overview</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/task-flow">Task Flow Description</a></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" tabindex="0" href="/docs/1.6.0/architecture/service-architecture/overview">Service Architecture</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret menu__link--active" aria-expanded="true" tabindex="0" href="/docs/1.6.0/architecture/feature/commons/variable">Feature</a></div><ul style="display:block;overflow:visible;height:auto" class="menu__list"><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-3 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" tabindex="0" href="/docs/1.6.0/architecture/feature/commons/variable">Commons</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-3 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" tabindex="0" href="/docs/1.6.0/architecture/feature/computation-governance-services/overview">Computation Governance Services</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-3 menu__list-item"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret menu__link--active" aria-expanded="true" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/overview">Public Enhancement Services</a></div><ul style="display:block;overflow:visible;height:auto" class="menu__list"><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-4 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/overview">Overview</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-4 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/public-service">Public Service</a></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-4 menu__list-item"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--active" aria-expanded="true" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/">Context Service</a><button aria-label="Toggle the collapsible sidebar category 'Context Service'" type="button" class="clean-btn menu__caret"></button></div><ul style="display:block;overflow:visible;height:auto" class="menu__list"><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-5 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/overview">Overview</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-5 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-client">CS Client Design</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-5 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-highavailable">CS HA Design</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-5 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-listener">CS Listener Architecture</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-5 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-persistence">CS Persistence Architecture</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-5 menu__list-item"><a class="menu__link menu__link--active" aria-current="page" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-search">CS Search Architecture</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-5 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-cache">CS Cache Architecture</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-5 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/content-service-cleanup">CS Cleanup Interface Features</a></li></ul></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-4 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/bml/overview">BML</a></div></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-4 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/datasource-manager">Data Source Management Service Architecture</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-4 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/1.6.0/architecture/feature/public-enhancement-services/metadata-manager">Data Source Management Service Architecture</a></li></ul></li></ul></li></ul></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/development/directory-structure">Development</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/tuning-and-troubleshooting/overview">Tuning And Troubleshooting</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/1.6.0/upgrade/upgrade-from-0.X-to-1.0-guide">Upgrade Guide</a></div></li></ul></nav></div></aside><main class="docMainContainer_gTbr"><div class="container padding-top--md padding-bottom--lg"><div class="row"><div class="col docItemCol_VOVn"><div class="theme-doc-version-banner alert alert--warning margin-bottom--md" role="alert"><div>This is unreleased documentation for <!-- -->Apache Linkis<!-- --> <b>Next(1.6.0)</b> version.</div><div class="margin-top--md">For up-to-date documentation, see the <b><a href="/docs/latest/architecture/feature/public-enhancement-services/context-service/context-service-search">latest version</a></b> (<!-- -->1.5.0<!-- -->).</div></div><div class="docItemContainer_Djhp"><article><nav class="theme-doc-breadcrumbs breadcrumbsContainer_Z_bl" aria-label="Breadcrumbs"><ul class="breadcrumbs" itemscope="" itemtype="https://schema.org/BreadcrumbList"><li class="breadcrumbs__item"><a aria-label="Home page" class="breadcrumbs__link" href="/"><svg viewBox="0 0 24 24" class="breadcrumbHomeIcon_OVgt"><path d="M10 19v-5h4v5c0 .55.45 1 1 1h3c.55 0 1-.45 1-1v-7h1.7c.46 0 .68-.57.33-.87L12.67 3.6c-.38-.34-.96-.34-1.34 0l-8.36 7.53c-.34.3-.13.87.33.87H5v7c0 .55.45 1 1 1h3c.55 0 1-.45 1-1z" fill="currentColor"></path></svg></a></li><li class="breadcrumbs__item"><span class="breadcrumbs__link">Architecture</span><meta itemprop="position" content="1"></li><li class="breadcrumbs__item"><span class="breadcrumbs__link">Feature</span><meta itemprop="position" content="2"></li><li class="breadcrumbs__item"><span class="breadcrumbs__link">Public Enhancement Services</span><meta itemprop="position" content="3"></li><li itemscope="" itemprop="itemListElement" itemtype="https://schema.org/ListItem" class="breadcrumbs__item"><a class="breadcrumbs__link" itemprop="item" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/"><span itemprop="name">Context Service</span></a><meta itemprop="position" content="4"></li><li itemscope="" itemprop="itemListElement" itemtype="https://schema.org/ListItem" class="breadcrumbs__item breadcrumbs__item--active"><span class="breadcrumbs__link" itemprop="name">CS Search Architecture</span><meta itemprop="position" content="5"></li></ul></nav><span class="theme-doc-version-badge badge badge--secondary">Version: Next(1.6.0)</span><div class="tocCollapsible_ETCw theme-doc-toc-mobile tocMobile_ITEo"><button type="button" class="clean-btn tocCollapsibleButton_TO0P">On this page</button></div><div class="theme-doc-markdown markdown"><header><h1>CS Search Architecture</h1></header><h2 class="anchor anchorWithStickyNavbar_LWe7" id="cssearch-architecture"><strong>CSSearch Architecture</strong><a class="hash-link" href="#cssearch-architecture" title="Direct link to heading"></a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="overall-architecture"><strong>Overall architecture</strong><a class="hash-link" href="#overall-architecture" title="Direct link to heading"></a></h3><p>As shown below:</p><p><img loading="lazy" src="/assets/images/linkis-contextservice-search-01-6aaabea466a0ef38d5a625bc29bc8570.png" width="1280" height="720" class="img_ev3q"></p><ol><li><p>ContextSearch: The query entry, accepts the query conditions defined in the Map form, and returns the corresponding results according to the conditions.</p></li><li><p>Building module: Each condition type corresponds to a Parser, which is responsible for converting the condition in the form of Map into a Condition object, which is implemented by calling the logic of ConditionBuilder. Conditions with complex logical relationships will use ConditionOptimizer to optimize query plans based on cost-based algorithms.</p></li><li><p>Execution module: Filter out the results that match the conditions from the Cache. According to different query targets, there are three execution modes: Ruler, Fetcher and Match. The specific logic is described later.</p></li><li><p>Evaluation module: Responsible for calculation of conditional execution cost and statistics of historical execution status.</p></li></ol><h3 class="anchor anchorWithStickyNavbar_LWe7" id="query-condition-definition-contextsearchcondition"><strong>Query Condition Definition (ContextSearchCondition)</strong><a class="hash-link" href="#query-condition-definition-contextsearchcondition" title="Direct link to heading"></a></h3><p>A query condition specifies how to filter out the part that meets the condition from a ContextKeyValue collection. The query conditions can be used to form more complex query conditions through logical operations.</p><ol><li><p>Support ContextType, ContextScope, KeyWord matching</p><ol><li><p>Corresponding to a Condition type</p></li><li><p>In Cache, these should have corresponding indexes</p></li></ol></li><li><p>Support contains/regex matching mode for key</p><ol><li><p>ContainsContextSearchCondition: contains a string</p></li><li><p>RegexContextSearchCondition: match a regular expression</p></li></ol></li><li><p>Support logical operations of or, and and not</p><ol><li>Unary operation UnaryContextSearchCondition:</li></ol></li></ol><blockquote><p>Support logical operations of a single parameter, such as NotContextSearchCondition</p></blockquote><ol><li>Binary operation BinaryContextSearchCondition:</li></ol><blockquote><p>Support the logical operation of two parameters, defined as LeftCondition and RightCondition, such as OrContextSearchCondition and AndContextSearchCondition</p></blockquote><ol><li><p>Each logical operation corresponds to an implementation class of the above subclass</p></li><li><p>The UML class diagram of this part is as follows:</p></li></ol><p><img loading="lazy" src="/assets/images/linkis-contextservice-search-02-fb5127e38e2e4fd3f34d8f3b98a17b0e.png" width="1952" height="1841" class="img_ev3q"></p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="construction-of-query-conditions"><strong>Construction of query conditions</strong><a class="hash-link" href="#construction-of-query-conditions" title="Direct link to heading"></a></h3><ol><li><p>Support construction through ContextSearchConditionBuilder: When constructing, if multiple ContextType, ContextScope, KeyWord, contains/regex matches are declared at the same time, they will be automatically connected by And logical operation</p></li><li><p>Support logical operations between Conditions and return new Conditions: And, Or and Not (considering the form of condition1.or(condition2), the top-level interface of Condition is required to define logical operation methods)</p></li><li><p>Support to build from Map through ContextSearchParser corresponding to each underlying implementation class</p></li></ol><h3 class="anchor anchorWithStickyNavbar_LWe7" id="execution-of-query-conditions"><strong>Execution of query conditions</strong><a class="hash-link" href="#execution-of-query-conditions" title="Direct link to heading"></a></h3><ol><li><p>Three function modes of query conditions:</p><ol><li><p>Ruler: Filter out eligible ContextKeyValue sub-Arrays from an Array</p></li><li><p>Matcher: Determine whether a single ContextKeyValue meets the conditions</p></li><li><p>Fetcher: Filter out an Array of eligible ContextKeyValue from ContextCache</p></li></ol></li><li><p>Each bottom-level Condition has a corresponding Execution, responsible for maintaining the corresponding Ruler, Matcher, and Fetcher.</p></li></ol><h3 class="anchor anchorWithStickyNavbar_LWe7" id="query-entry-contextsearch"><strong>Query entry ContextSearch</strong><a class="hash-link" href="#query-entry-contextsearch" title="Direct link to heading"></a></h3><p>Provide a search interface, receive Map as a parameter, and filter out the corresponding data from the Cache.</p><ol><li><p>Use Parser to convert the condition in the form of Map into a Condition object</p></li><li><p>Obtain cost information through Optimizer, and determine the order of query according to the cost information</p></li><li><p>After executing the corresponding Ruler/Fetcher/Matcher logic through the corresponding Execution, the search result is obtained</p></li></ol><p><img loading="lazy" src="/assets/images/linkis-contextservice-search-03-be9fc6738a640d46efe8d890f0ad9632.png" width="892" height="472" class="img_ev3q"></p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="query-optimization"><strong>Query Optimization</strong><a class="hash-link" href="#query-optimization" title="Direct link to heading"></a></h3><ol><li><p>OptimizedContextSearchCondition maintains the Cost and Statistics information of the condition:</p><ol><li><p>Cost information: CostCalculator is responsible for judging whether a certain Condition can calculate Cost, and if it can be calculated, it returns the corresponding Cost object</p></li><li><p>Statistics information: start/end/execution time, number of input lines, number of output lines</p></li></ol></li><li><p>Implement a CostContextSearchOptimizer, whose optimize method is based on the cost of the Condition to optimize the Condition and convert it into an OptimizedContextSearchCondition object. The specific logic is described as follows:</p><ol><li>Disassemble a complex Condition into a tree structure based on the combination of logical operations. Each leaf node is a basic simple Condition; each non-leaf node is a logical operation.</li></ol></li></ol><blockquote><p>Tree A as shown in the figure below is a complex condition composed of five simple conditions of ABCDE through various logical operations.</p></blockquote><p><img loading="lazy" src="/assets/images/linkis-contextservice-search-04-56a888dbb72f9dc3ac464f9caf88d6f7.png" width="656" height="442" class="img_ev3q"></p><center>(Tree A)</center><ol><li>The execution of these Conditions is actually depth first, traversing the tree from left to right. Moreover, according to the exchange rules of logical operations, the left and right order of the child nodes of a node in the Condition tree can be exchanged, so all possible trees in all possible execution orders can be enumerated.</li></ol><blockquote><p>Tree B as shown in the figure below is another possible sequence of tree A above, which is exactly the same as the execution result of tree A, except that the execution order of each part has been adjusted.</p></blockquote><p><img loading="lazy" src="/assets/images/linkis-contextservice-search-05-aaf617814ecac7b890ef11722e27ed63.png" width="681" height="446" class="img_ev3q"></p><center>(Tree B)</center><ol><li>For each tree, the cost is calculated from the leaf node and collected to the root node, which is the final cost of the tree, and finally the tree with the smallest cost is obtained as the optimal execution order.</li></ol><blockquote><p>The rules for calculating node cost are as follows:</p></blockquote><ol><li><p>For leaf nodes, each node has two attributes: Cost and Weight. Cost is the cost calculated by CostCalculator. Weight is assigned according to the order of execution of the nodes. The current default is 1 on the left and 0.5 on the right. See how to adjust it later (the reason for assigning weight is that the conditions on the left have already been set in some cases. It can directly determine whether the entire combinatorial logic matches or not, so the condition on the right does not have to be executed in all cases, and the actual cost needs to be reduced by a certain percentage)</p></li><li><p>For non-leaf nodes, Cost = the sum of Cost×Weight of all child nodes; the weight assignment logic is consistent with that of leaf nodes.</p></li></ol><blockquote><p>Taking tree A and tree B as examples, calculate the costs of these two trees respectively, as shown in the figure below, the number in the node is Cost<!-- -->|<!-- -->Weight, assuming that the cost of the 5 simple conditions of ABCDE is 10, 100, 50 , 10, and 100. It can be concluded that the cost of tree B is less than that of tree A, which is a better solution.</p></blockquote><center><img loading="lazy" src="/Images/Architecture/Public_Enhancement_Service/ContextService/linkis-contextservice-search-06.png" width="300" class="img_ev3q"> <img loading="lazy" src="/Images/Architecture/Public_Enhancement_Service/ContextService/linkis-contextservice-search-07.png" width="300" class="img_ev3q"></center><ol><li><p>Use CostCalculator to measure the cost of simple conditions:</p><ol><li><p>The condition acting on the index: the cost is determined according to the distribution of the index value. For example, when the length of the Array obtained by condition A from the Cache is 100 and condition B is 200, then the cost of condition A is less than B.</p></li><li><p>Conditions that need to be traversed:</p><ol><li><p>According to the matching mode of the condition itself, an initial Cost is given: For example, Regex is 100, Contains is 10, etc. (the specific values etc. will be adjusted according to the situation when they are realized)</p></li><li><p>According to the efficiency of historical query, the real-time Cost is obtained after continuous adjustment on the basis of the initial Cost. Throughput per unit time</p></li></ol></li></ol></li></ol></div><footer class="theme-doc-footer docusaurus-mt-lg"><div class="theme-doc-footer-edit-meta-row row"><div class="col"><a href="https://github.com/apache/linkis-website/edit/dev/docs/architecture/feature/public-enhancement-services/context-service/context-service-search.md" target="_blank" rel="noreferrer noopener" class="theme-edit-this-page"><svg fill="currentColor" height="20" width="20" viewBox="0 0 40 40" class="iconEdit_Z9Sw" aria-hidden="true"><g><path d="m34.5 11.7l-3 3.1-6.3-6.3 3.1-3q0.5-0.5 1.2-0.5t1.1 0.5l3.9 3.9q0.5 0.4 0.5 1.1t-0.5 1.2z m-29.5 17.1l18.4-18.5 6.3 6.3-18.4 18.4h-6.3v-6.2z"></path></g></svg>Edit this page</a></div><div class="col lastUpdated_vwxv"></div></div></footer></article><nav class="pagination-nav docusaurus-mt-lg" aria-label="Docs pages navigation"><a class="pagination-nav__link pagination-nav__link--prev" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-persistence"><div class="pagination-nav__sublabel">Previous</div><div class="pagination-nav__label">CS Persistence Architecture</div></a><a class="pagination-nav__link pagination-nav__link--next" href="/docs/1.6.0/architecture/feature/public-enhancement-services/context-service/context-service-cache"><div class="pagination-nav__sublabel">Next</div><div class="pagination-nav__label">CS Cache Architecture</div></a></nav></div></div><div class="col col--3"><div class="tableOfContents_bqdL thin-scrollbar theme-doc-toc-desktop"><ul class="table-of-contents table-of-contents__left-border"><li><a href="#cssearch-architecture" class="table-of-contents__link toc-highlight"><strong>CSSearch Architecture</strong></a><ul><li><a href="#overall-architecture" class="table-of-contents__link toc-highlight"><strong>Overall architecture</strong></a></li><li><a href="#query-condition-definition-contextsearchcondition" class="table-of-contents__link toc-highlight"><strong>Query Condition Definition (ContextSearchCondition)</strong></a></li><li><a href="#construction-of-query-conditions" class="table-of-contents__link toc-highlight"><strong>Construction of query conditions</strong></a></li><li><a href="#execution-of-query-conditions" class="table-of-contents__link toc-highlight"><strong>Execution of query conditions</strong></a></li><li><a href="#query-entry-contextsearch" class="table-of-contents__link toc-highlight"><strong>Query entry ContextSearch</strong></a></li><li><a href="#query-optimization" class="table-of-contents__link toc-highlight"><strong>Query Optimization</strong></a></li></ul></li></ul></div></div></div></div></main></div></div><footer class="footer"><div class="container container-fluid"><div class="row footer__links"><div class="col footer__col"><div class="footer__title">Linkis</div><ul class="footer__items clean-list"><li class="footer__item"><a class="footer__link-item" href="/docs/latest/about/introduction">Document</a></li><li class="footer__item"><a class="footer__link-item" href="/faq/main">FAQ</a></li><li class="footer__item"><a href="https://github.com/apache/linkis/releases" target="_blank" rel="noopener noreferrer" class="footer__link-item">Releases</a></li></ul></div><div class="col footer__col"><div class="footer__title">Community</div><ul class="footer__items clean-list"><li class="footer__item"><a href="https://github.com/apache/linkis" target="_blank" rel="noopener noreferrer" class="footer__link-item">GitHub</a></li><li class="footer__item"><a href="https://github.com/apache/linkis/issues" target="_blank" rel="noopener noreferrer" class="footer__link-item">Issue Tracker</a></li><li class="footer__item"><a href="https://github.com/apache/linkis/pulls" target="_blank" rel="noopener noreferrer" class="footer__link-item">Pull Requests</a></li></ul></div><div class="col footer__col"><div class="footer__title">Subscribe Mailing List</div><ul class="footer__items clean-list"><li class="footer__item"><a class="footer__link-item" href="/community/how-to-subscribe">How to Subscribe</a></li><li class="footer__item"><a href="mailto:dev-subscribe@linkis.apache.org" target="_blank" rel="noopener noreferrer" class="footer__link-item">Subscribe Mail</a></li><li class="footer__item"><a href="https://lists.apache.org/list.html?dev@linkis.apache.org" target="_blank" rel="noopener noreferrer" class="footer__link-item">Mail Archive</a></li></ul></div></div><div class="footer__bottom text--center"><div class="footer__copyright"><div><img style="height:50px" alt="Apache Software Foundation" src="/img/incubator-logo.svg"><p style="color: #999999; padding: 0 20px 30px;font-weight:400;text-align:left">Apache Linkis is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p><p></p> |
| <p style="padding: 0 20px 30px;color: #999999;font-weight: 400;"> Copyright © 2024 The Apache Software Foundation. Licensed under the Apache License, Version 2.0. Apache Linkis, Apache Incubator, Apache, the Apache feather logo, the Apache Linkis logo and the Apache Incubator project logo are trademarks of The Apache Software Foundation.</p> |
| <div></div></div></div></div></div></footer></div> |
| <script src="/assets/js/runtime~main.36b81271.js"></script> |
| <script src="/assets/js/main.f09c65a0.js"></script> |
