| </a></div><button type="button" class="clean-btn close announcementBarClose_iXyO" aria-label="Close"><svg viewBox="0 0 15 15" width="14" height="14" style="color:white"><g stroke="currentColor" stroke-width="3.1"><path d="M.75.75l13.5 13.5M14.25.75L.75 14.25"></path></g></svg></button></div><nav aria-label="Main" class="navbar navbar--fixed-top"><div class="navbar__inner" style="padding:"><div class="navbar__items"><div class="navbar-left"><div class="navbar-logo-wrapper flex items-center"><a class="navbar__brand" href="/"><div class="navbar__logo"><img src="https://cdnd.selectdb.com/images/logo.svg" alt="Apache Doris" class="themedImage_ToTc themedImage--light_HNdA"><img src="https://cdnd.selectdb.com/images/logo.svg" alt="Apache Doris" class="themedImage_ToTc themedImage--dark_i4oU"></div><b class="navbar__title text--truncate"></b></a></div><a class="navbar__item navbar__link" style="text-align:center" href="/docs/get-starting/quick-start">Docs</a><a class="navbar__item navbar__link" style="text-align:center" href="/blog">Blog</a><a class="navbar__item navbar__link" style="text-align:center" href="/users">Users</a><a href="https://github.com/apache/doris/discussions" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link" style="text-align:center">Discussions</a><a class="navbar__item navbar__link" style="text-align:center" href="/ecosystem/cluster-management">Ecosystem</a><a class="navbar__item navbar__link" style="text-align:center" href="/community/join-community">Community</a></div></div><div class="navbar__items navbar__items--right"><button aria-label="Toggle navigation bar" aria-expanded="false" class="navbar__toggle clean-btn" type="button"><svg width="30" height="30" viewBox="0 0 30 30" aria-hidden="true"><path stroke="currentColor" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M4 7h22M4 15h22M4 23h22"></path></svg></button><div class="docs-search searchBox_H2mL"><div class="navbar__search searchBarContainer_PzyC"><input placeholder="Search" aria-label="Search" class="navbar__search-input navbarSearchInput_tb6T"><div class="loadingRing__K5d searchBarLoadingRing_e2f0"><div></div><div></div><div></div><div></div></div><div class="searchHintContainer_m7ml"><kbd class="searchHint_zuPL">ctrl</kbd><kbd class="searchHint_zuPL">K</kbd></div></div></div><div class="custom-navbar-item navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link"><svg class="icon-language" xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16" fill="none"><path d="M7.75756 14.3L10.5816 6.91667H11.8759L14.7 14.3H13.4057L12.7501 12.4167H9.74113L9.06873 14.3H7.75756ZM10.1109 11.35H12.3467L11.254 8.3H11.2036L10.1109 11.35ZM2.84908 12.45L1.97498 11.5833L5.11841 8.48333C4.72618 8.05 4.38439 7.60267 4.09302 7.14133C3.80165 6.68044 3.54389 6.19444 3.31976 5.68333H4.61412C4.80463 6.06111 5.00635 6.39711 5.21927 6.69133C5.43219 6.986 5.68434 7.29444 5.97571 7.61667C6.43519 7.12778 6.81621 6.62511 7.11879 6.10867C7.42137 5.59178 7.67352 5.03889 7.87523 4.45H1V3.23333H5.33694V2H6.58087V3.23333H10.9178V4.45H9.11916C8.89503 5.18333 8.59805 5.89155 8.22824 6.57467C7.85842 7.25822 7.39895 7.90555 6.84983 8.51667L8.3459 10.0167L7.87523 11.2833L5.95891 9.38333L2.84908 12.45Z" fill="#4C576C"></path></svg></a><ul class="dropdown__menu"><li><a href="/docs/2.0/faq/install-faq" target="_self" rel="noopener noreferrer" class="dropdown__link dropdown__link--active" style="text-align:center">English</a></li><li><a href="/zh-CN/docs/2.0/faq/install-faq" target="_self" rel="noopener noreferrer" class="dropdown__link" style="text-align:center">中文</a></li></ul></div><div class="custom-navbar-item navbar__item dropdown dropdown--hoverable dropdown--right"><a class="navbar__link" aria-haspopup="true" aria-expanded="false" role="button" href="/docs/2.0/get-starting/what-is-apache-doris"><span class="text-sm">Versions: <!-- -->2.0</span></a><ul class="dropdown__menu"><li><a class="dropdown__link" style="text-align:center" href="/docs/dev/faq/install-faq">Dev</a></li><li><a class="dropdown__link" style="text-align:center" href="/docs/faq/install-faq">2.1</a></li><li><a aria-current="page" class="dropdown__link dropdown__link--active" style="text-align:center" href="/docs/2.0/faq/install-faq">2.0</a></li><li><a class="dropdown__link" style="text-align:center" href="/docs/1.2/faq/install-faq">1.2</a></li></ul></div><a class="navbar__item navbar__link header-right-button-primary navbar-download-mobile" style="text-align:center" href="/download">Download</a><a href="https://github.com/apache/doris" target="_blank" rel="noopener noreferrer" class="github-btn desktop header-right-button-github"></a><a href="https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-2kl08hzc0-SPJe4VWmL_qzrFd2u2XYQA" target="_blank" rel="noopener noreferrer" class="slack-btn desktop header-right-button-slack"></a><a class="header-right-button-primary navbar-download-desktop" href="/download">Download</a></div></div><div class="navbar__bottom"></div><div role="presentation" class="navbar-sidebar__backdrop"></div></nav><div id="__docusaurus_skipToContent_fallback" class="main-wrapper mainWrapper_z2l0"><div role="region" aria-label="Skip to main content"><a class="skipToContent_fXgn" href="#__docusaurus_skipToContent_fallback">Skip to main content</a></div><div class="main-wrapper docsWrapper_BCFX"><button aria-label="Scroll back to top" class="clean-btn theme-back-to-top-button backToTopButton_sjWU" type="button"></button><div class="docPage__5DB"><aside class="theme-doc-sidebar-container docSidebarContainer_b6E3"><div class="sidebarViewport_Xe31"><div class="sidebar"><nav aria-label="Docs sidebar" class="menu custom-scrollbar menu_Y1UP menuWithAnnouncementBar_fPny"><ul class="theme-doc-sidebar-menu menu__list"><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/get-starting/what-is-apache-doris">Getting Started</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/install/source-install/compilation-with-docker">Installation and Deployment</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/db-connect/database-connect">Database Connection</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/table-design/data-type">Data Table Design</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/data-operate/import/load-manual">Data Manipulation</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/query/query-data/mysql-compatibility">Data Queries</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/lakehouse/lakehouse-overview">Lakehouse</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/admin-manual/cluster-management/upgrade">Managing Doris</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/ecosystem/spark-doris-connector">Ecosystem</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/sql-manual/sql-functions/array-functions/array">SQL Reference</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/benchmark/ssb">Benchmark</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret menu__link--active" aria-expanded="true" href="/docs/2.0/faq/install-faq">FAQ</a></div><ul style="display:block;overflow:visible;height:auto" class="menu__list menu__list_level_2"><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link menu__link--active" aria-current="page" tabindex="0" href="/docs/2.0/faq/install-faq">Install Error</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/2.0/faq/data-faq">Data Operation Error</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/2.0/faq/sql-faq">SQL Error</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/2.0/faq/lakehouse-faq">Lakehouse FAQ</a></li></ul></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a id="menu__group" class="menu__link menu__link_level_1 menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/2.0/releasenotes/release-2.0.11">Release notes</a></div></li></ul></nav></div></div></aside><main class="docMainContainer_gTbr"><div class="container padding-top--md padding-bottom--lg"><div class="row"><div class="col docItemCol_z5aJ"><div class="docItemContainer_c0TR"><article><nav class="theme-doc-breadcrumbs breadcrumbsContainer_Z_bl" aria-label="Breadcrumbs"><ul class="breadcrumbs" itemscope="" itemtype="https://schema.org/BreadcrumbList"><li class="breadcrumbs__item"><a aria-label="Home page" class="breadcrumbs__link" href="/"><svg viewBox="0 0 24 24" class="breadcrumbHomeIcon_YNFT"><path d="M10 19v-5h4v5c0 .55.45 1 1 1h3c.55 0 1-.45 1-1v-7h1.7c.46 0 .68-.57.33-.87L12.67 3.6c-.38-.34-.96-.34-1.34 0l-8.36 7.53c-.34.3-.13.87.33.87H5v7c0 .55.45 1 1 1h3c.55 0 1-.45 1-1z" fill="currentColor"></path></svg></a></li><li class="breadcrumbs__item"><span class="breadcrumbs__link">FAQ</span><meta itemprop="position" content="1"></li><li itemscope="" itemprop="itemListElement" itemtype="https://schema.org/ListItem" class="breadcrumbs__item breadcrumbs__item--active"><span class="breadcrumbs__link" itemprop="name">Install Error</span><meta itemprop="position" content="2"></li></ul></nav><div class="tocCollapsible_ETCw theme-doc-toc-mobile tocMobile_ITEo"><button type="button" class="clean-btn tocCollapsibleButton_TO0P">On this page</button></div><div class="theme-doc-markdown markdown"><header><h1>Install Error</h1></header><h1>Operation and Maintenance Error</h1><p>This document is mainly used to record the common problems of operation and maintenance during the use of Doris. It will be updated from time to time.</p><p><strong>The name of the BE binary that appears in this doc is <code>doris_be</code>, which was <code>palo_be</code> in previous versions.</strong></p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q1-why-is-there-always-some-tablet-left-when-i-log-off-the-be-node-through-decommission">Q1. Why is there always some tablet left when I log off the BE node through DECOMMISSION?<a href="#q1-why-is-there-always-some-tablet-left-when-i-log-off-the-be-node-through-decommission" class="hash-link" aria-label="Direct link to Q1. Why is there always some tablet left when I log off the BE node through DECOMMISSION?" title="Direct link to Q1. Why is there always some tablet left when I log off the BE node through DECOMMISSION?"></a></h3><p>During the offline process, use show backends to view the tabletNum of the offline node, and you will observe that the number of tabletNum is decreasing, indicating that data shards are being migrated from this node. When the number is reduced to 0, the system will automatically delete the node. But in some cases, tabletNum will not change after it drops to a certain value. This is usually possible for two reasons:</p><ol><li>The tablets belong to the table, partition, or materialized view that was just dropped. Objects that have just been deleted remain in the recycle bin. The offline logic will not process these shards. The time an object resides in the recycle bin can be modified by modifying the FE configuration parameter catalog_trash_expire_second. These tablets are disposed of when the object is removed from the recycle bin.</li><li>There is a problem with the migration task for these tablets. At this point, you need to view the errors of specific tasks through show proc <code>show proc "/cluster_balance"</code>.</li></ol><p>For the above situation, you can first check whether there are unhealthy shards in the cluster through <code>show proc "/cluster_health/tablet_health";</code>. If it is 0, you can delete the BE directly through the drop backend statement. Otherwise, you also need to check the replicas of unhealthy shards in detail.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q2-how-should-priorty_network-be-set">Q2. How should priorty_network be set?<a href="#q2-how-should-priorty_network-be-set" class="hash-link" aria-label="Direct link to Q2. How should priorty_network be set?" title="Direct link to Q2. How should priorty_network be set?"></a></h3><p>priorty_network is a configuration parameter for both FE and BE. This parameter is mainly used to help the system select the correct network card IP as its own IP. It is recommended to explicitly set this parameter in any case to prevent the problem of incorrect IP selection caused by adding new network cards to subsequent machines.</p><p>The value of priorty_network is expressed in CIDR format. Divided into two parts, the first part is the IP address in dotted decimal, and the second part is a prefix length. For example 10.168.1.0/8 will match all 10.xx.xx.xx IP addresses, and 10.168.1.0/16 will match all 10.168.xx.xx IP addresses.</p><p>The reason why the CIDR format is used instead of specifying a specific IP directly is to ensure that all nodes can use a uniform configuration value. For example, there are two nodes: 10.168.10.1 and 10.168.10.2, then we can use 10.168.10.0/24 as the value of priorty_network.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q3-what-are-the-master-follower-and-observer-of-fe">Q3. What are the Master, Follower and Observer of FE?<a href="#q3-what-are-the-master-follower-and-observer-of-fe" class="hash-link" aria-label="Direct link to Q3. What are the Master, Follower and Observer of FE?" title="Direct link to Q3. What are the Master, Follower and Observer of FE?"></a></h3><p>First of all, make it clear that FE has only two roles: Follower and Observer. The Master is just an FE selected from a group of Follower nodes. Master can be regarded as a special kind of Follower. So when we were asked how many FEs a cluster had and what roles they were, the correct answer should be the number of all FE nodes, the number of Follower roles and the number of Observer roles.</p><p>All FE nodes of the Follower role will form an optional group, similar to the group concept in the Paxos consensus protocol. A Follower will be elected as the Master in the group. When the Master hangs up, a new Follower will be automatically selected as the Master. The Observer will not participate in the election, so the Observer will not be called Master.</p><p>A metadata log needs to be successfully written in most Follower nodes to be considered successful. For example, if there are 3 FEs, only 2 can be successfully written. This is why the number of Follower roles needs to be an odd number.</p><p>The role of Observer is the same as the meaning of this word. It only acts as an observer to synchronize the metadata logs that have been successfully written, and provides metadata reading services. He will not be involved in the logic of the majority writing.</p><p>Typically, 1 Follower + 2 Observer or 3 Follower + N Observer can be deployed. The former is simple to operate and maintain, and there is almost no consistency agreement between followers to cause such complex error situations (Most companies use this method). The latter can ensure the high availability of metadata writing. If it is a high concurrent query scenario, Observer can be added appropriately.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q4-a-new-disk-is-added-to-the-node-why-is-the-data-not-balanced-to-the-new-disk">Q4. A new disk is added to the node, why is the data not balanced to the new disk?<a href="#q4-a-new-disk-is-added-to-the-node-why-is-the-data-not-balanced-to-the-new-disk" class="hash-link" aria-label="Direct link to Q4. A new disk is added to the node, why is the data not balanced to the new disk?" title="Direct link to Q4. A new disk is added to the node, why is the data not balanced to the new disk?"></a></h3><p>The current Doris balancing strategy is based on nodes. That is to say, the cluster load is judged according to the overall load index of the node (number of shards and total disk utilization). And migrate data shards from high-load nodes to low-load nodes. If each node adds a disk, from the overall point of view of the node, the load does not change, so the balancing logic cannot be triggered.</p><p>In addition, Doris currently does not support balancing operations between disks within a single node. Therefore, after adding a new disk, the data will not be balanced to the new disk.</p><p>However, when data is migrated between nodes, Doris takes the disk into account. For example, when a shard is migrated from node A to node B, the disk with low disk space utilization in node B will be preferentially selected.</p><p>Here we provide 3 ways to solve this problem:</p><ol><li><p>Rebuild the new table</p><p>Create a new table through the create table like statement, and then use the insert into select method to synchronize data from the old table to the new table. Because when a new table is created, the data shards of the new table will be distributed in the new disk, so the data will also be written to the new disk. This method is suitable for situations where the amount of data is small (within tens of GB).</p></li><li><p>Through the Decommission command</p><p>The decommission command is used to safely decommission a BE node. This command will first migrate the data shards on the node to other nodes, and then delete the node. As mentioned earlier, during data migration, the disk with low disk utilization will be prioritized, so this method can "force" the data to be migrated to the disks of other nodes. When the data migration is completed, we cancel the decommission operation, so that the data will be rebalanced back to this node. When we perform the above steps on all BE nodes, the data will be evenly distributed on all disks of all nodes.</p><p>Note that before executing the decommission command, execute the following command to avoid the node being deleted after being offline.</p><p><code>admin set frontend config("drop_backend_after_decommission" = "false");</code></p></li><li><p>Manually migrate data using the API</p><p>Doris provides <a href="https://doris.apache.org/zh-CN/docs/dev/admin-manual/http-actions/be/tablet-migration" target="_blank" rel="noopener noreferrer">HTTP API</a>, which can manually specify the migration of data shards on one disk to another disk.</p></li></ol><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q5-how-to-read-febe-logs-correctly">Q5. How to read FE/BE logs correctly?<a href="#q5-how-to-read-febe-logs-correctly" class="hash-link" aria-label="Direct link to Q5. How to read FE/BE logs correctly?" title="Direct link to Q5. How to read FE/BE logs correctly?"></a></h3><p>In many cases, we need to troubleshoot problems through logs. The format and viewing method of the FE/BE log are described here.</p><ol><li><p>FE</p><p>FE logs mainly include:</p><ul><li>fe.log: main log. Includes everything except fe.out.</li><li>fe.warn.log: A subset of the main log, only WARN and ERROR level logs are logged.</li><li>fe.out: log for standard/error output (stdout and stderr).</li><li>fe.audit.log: Audit log, which records all SQL requests received by this FE.</li></ul><p>A typical FE log is as follows:</p><div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">2021-09-16 23:13:22,502 INFO (tablet scheduler|43) [BeLoadRebalancer.selectAlternativeTabletsForCluster():85] cluster is balance: default_cluster with medium: HDD.skip</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><ul><li><code>2021-09-16 23:13:22,502</code>: log time.</li><li><code>INFO: log level, default is INFO</code>.</li><li><code>(tablet scheduler|43)</code>: thread name and thread id. Through the thread id, you can view the context information of this thread and check what happened in this thread.</li><li><code>BeLoadRebalancer.selectAlternativeTabletsForCluster():85</code>: class name, method name and code line number.</li><li><code>cluster is balance xxx</code>: log content.</li></ul><p>Usually, we mainly view the fe.log log. In special cases, some logs may be output to fe.out.</p></li><li><p>BE</p><p>BE logs mainly include:</p><ul><li>be.INFO: main log. This is actually a soft link, connected to the latest be.INFO.xxxx.</li><li>be.WARNING: A subset of the main log, only WARN and FATAL level logs are logged. This is actually a soft link, connected to the latest be.WARN.xxxx.</li><li>be.out: log for standard/error output (stdout and stderr).</li></ul><p>A typical BE log is as follows:</p><div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">I0916 23:21:22.038795 28087 task_worker_pool.cpp:1594] finish report TASK. master host: 10.10.10.10, port: 9222</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><ul><li><code>I0916 23:21:22.038795</code>: log level and datetime. The capital letter I means INFO, W means WARN, and F means FATAL.</li><li><code>28087</code>: thread id. Through the thread id, you can view the context information of this thread and check what happened in this thread.</li><li><code>task_worker_pool.cpp:1594</code>: code file and line number.</li><li><code>finish report TASK xxx</code>: log content.</li></ul><p>Usually we mainly look at the be.INFO log. In special cases, such as BE downtime, you need to check be.out.</p></li></ol><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q6-how-to-troubleshoot-the-febe-node-is-down">Q6. How to troubleshoot the FE/BE node is down?<a href="#q6-how-to-troubleshoot-the-febe-node-is-down" class="hash-link" aria-label="Direct link to Q6. How to troubleshoot the FE/BE node is down?" title="Direct link to Q6. How to troubleshoot the FE/BE node is down?"></a></h3><ol><li><p>BE</p><p>The BE process is a C/C++ process, which may hang due to some program bugs (memory out of bounds, illegal address access, etc.) or Out Of Memory (OOM). At this point, we can check the cause of the error through the following steps:</p><ol><li><p>View be.out</p><p>The BE process realizes that when the program exits due to an exception, it will print the current error stack to be.out (note that it is be.out, not be.INFO or be.WARNING). Through the error stack, you can usually get a rough idea of where the program went wrong.</p><p>Note that if there is an error stack in be.out, it is usually due to a program bug, and ordinary users may not be able to solve it by themselves. Welcome to the WeChat group, github discussion or dev mail group for help, and post the corresponding error stack, so that you can quickly Troubleshoot problems.</p></li><li><p>dmesg</p><p>If there is no stack information in be.out, the probability is that OOM was forcibly killed by the system. At this time, you can use the dmesg -T command to view the Linux system log. If a log similar to Memory cgroup out of memory: Kill process 7187 (doris_be) score 1007 or sacrifice child appears at the end, it means that it is caused by OOM.</p><p>Memory problems can have many reasons, such as large queries, imports, compactions, etc. Doris is also constantly optimizing memory usage. Welcome to the WeChat group, github discussion or dev mail group for help.</p></li><li><p>Check whether there are logs beginning with F in be.INFO.</p><p>Logs starting with F are Fatal logs. For example, F0916 , indicating the Fatal log on September 16th. Fatal logs usually indicate a program assertion error, and an assertion error will directly cause the process to exit (indicating a bug in the program). Welcome to the WeChat group, github discussion or dev mail group for help.</p></li></ol></li><li><p>FE</p><p>FE is a java process, and the robustness is better than the C/C++ program. Usually the reason for FE to hang up may be OOM (Out-of-Memory) or metadata write failure. These errors usually have an error stack in fe.log or fe.out. Further investigation is required based on the error stack information.</p></li></ol><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q7-about-the-configuration-of-data-directory-ssd-and-hdd-create-table-encounter-error-failed-to-find-enough-host-with-storage-medium-and-tag">Q7. About the configuration of data directory SSD and HDD, create table encounter error <code>Failed to find enough host with storage medium and tag</code><a href="#q7-about-the-configuration-of-data-directory-ssd-and-hdd-create-table-encounter-error-failed-to-find-enough-host-with-storage-medium-and-tag" class="hash-link" aria-label="Direct link to q7-about-the-configuration-of-data-directory-ssd-and-hdd-create-table-encounter-error-failed-to-find-enough-host-with-storage-medium-and-tag" title="Direct link to q7-about-the-configuration-of-data-directory-ssd-and-hdd-create-table-encounter-error-failed-to-find-enough-host-with-storage-medium-and-tag"></a></h3><p>Doris supports one BE node to configure multiple storage paths. Usually, one storage path can be configured for each disk. At the same time, Doris supports storage media properties that specify paths, such as SSD or HDD. SSD stands for high-speed storage device and HDD stands for low-speed storage device.</p><p>If the cluster only has one type of medium, such as all HDD or all SSD, the best practice is not to explicitly specify the medium property in be.conf. If encountering the error <code>Failed to find enough host with storage medium and tag</code> mentioned above, it is generally because be.conf only configures the SSD medium, while the table creation stage explicitly specifies <code>properties {"storage_medium" = "hdd"}</code>; similarly, if be.conf only configures the HDD medium, and the table creation stage explicitly specifies <code>properties {"storage_medium" = "ssd"}</code>, the same error will occur. The solution is to modify the properties parameter in the table creation to match the configuration; or remove the explicit configuration of SSD/HDD in be.conf.</p><p>By specifying the storage medium properties of the path, we can take advantage of Doris's hot and cold data partition storage function to store hot data in SSD at the partition level, while cold data is automatically transferred to HDD.</p><p>It should be noted that Doris does not automatically perceive the actual storage medium type of the disk where the storage path is located. This type needs to be explicitly indicated by the user in the path configuration. For example, the path "/path/to/data1.SSD" means that this path is an SSD storage medium. And "data1.SSD" is the actual directory name. Doris determines the storage media type based on the ".SSD" suffix after the directory name, not the actual storage media type. That is to say, the user can specify any path as the SSD storage medium, and Doris only recognizes the directory suffix and does not judge whether the storage medium matches. If no suffix is written, it will default to HDD.</p><p>In other words, ".HDD" and ".SSD" are only used to identify the "relative" "low speed" and "high speed" of the storage directory, not the actual storage medium type. Therefore, if the storage path on the BE node has no medium difference, the suffix does not need to be filled in.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q8-multiple-fes-cannot-log-in-when-using-nginx-to-implement-web-ui-load-balancing">Q8. Multiple FEs cannot log in when using Nginx to implement web UI load balancing<a href="#q8-multiple-fes-cannot-log-in-when-using-nginx-to-implement-web-ui-load-balancing" class="hash-link" aria-label="Direct link to Q8. Multiple FEs cannot log in when using Nginx to implement web UI load balancing" title="Direct link to Q8. Multiple FEs cannot log in when using Nginx to implement web UI load balancing"></a></h3><p>Doris can deploy multiple FEs. When accessing the Web UI, if Nginx is used for load balancing, there will be a constant prompt to log in again because of the session problem. This problem is actually a problem of session sharing. Nginx provides centralized session sharing. The solution, here we use the ip_hash technology in nginx, ip_hash can direct the request of an ip to the same backend, so that a client and a backend under this ip can establish a stable session, ip_hash is defined in the upstream configuration:</p><div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">upstream doris.com {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> server 172.22.197.238:8030 weight=3;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> server 172.22.197.239:8030 weight=4;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> server 172.22.197.240:8030 weight=4;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> ip_hash;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>The complete Nginx example configuration is as follows:</p><div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">user nginx;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">worker_processes auto;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">error_log /var/log/nginx/error.log;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">pid /run/nginx.pid;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"># Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">include /usr/share/nginx/modules/*.conf;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">events {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> worker_connections 1024;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">}</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain">http {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> log_format main '$remote_addr - $remote_user [$time_local] "$request" '</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> '$status $body_bytes_sent "$http_referer" '</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> '"$http_user_agent" "$http_x_forwarded_for"';</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> access_log /var/log/nginx/access.log main;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> sendfile on;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> tcp_nopush on;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> tcp_nodelay on;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> keepalive_timeout 65;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> types_hash_max_size 2048;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> include /etc/nginx/mime.types;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> default_type application/octet-stream;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> # Load modular configuration files from the /etc/nginx/conf.d directory.</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> # See http://nginx.org/en/docs/ngx_core_module.html#include</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> # for more information.</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> include /etc/nginx/conf.d/*.conf;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> #include /etc/nginx/custom/*.conf;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> upstream doris.com {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> server 172.22.197.238:8030 weight=3;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> server 172.22.197.239:8030 weight=4;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> server 172.22.197.240:8030 weight=4;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> ip_hash;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> server {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> listen 80;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> server_name gaia-pro-bigdata-fe02;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> if ($request_uri ~ _load) {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> return 307 http://$host$request_uri ;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> location / {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> proxy_pass http://doris.com;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> proxy_redirect default;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> error_page 500 502 503 504 /50x.html;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> location = /50x.html {</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> root html;</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span><br></span><span class="token-line" style="color:#F8F8F2"><span class="token plain"> }</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q9-fe-fails-to-start-wait-catalog-to-be-ready-fe-type-unknown-keeps-scrolling-in-felog">Q9. FE fails to start, "wait catalog to be ready. FE type UNKNOWN" keeps scrolling in fe.log<a href="#q9-fe-fails-to-start-wait-catalog-to-be-ready-fe-type-unknown-keeps-scrolling-in-felog" class="hash-link" aria-label="Direct link to Q9. FE fails to start, "wait catalog to be ready. FE type UNKNOWN" keeps scrolling in fe.log" title="Direct link to Q9. FE fails to start, "wait catalog to be ready. FE type UNKNOWN" keeps scrolling in fe.log"></a></h3><p>There are usually two reasons for this problem:</p><ol><li>The local IP obtained when FE is started this time is inconsistent with the last startup, usually because <code>priority_network</code> is not set correctly, which causes FE to match the wrong IP address when it starts. Restart FE after modifying <code>priority_network</code>.</li><li>Most Follower FE nodes in the cluster are not started. For example, there are 3 Followers, and only one is started. At this time, at least one other FE needs to be started, so that the FE electable group can elect the Master to provide services.</li></ol><p>If the above situation cannot be solved, you can restore it according to the <!-- -->[metadata operation and maintenance document]<!-- --> (../admin-manual/maint-monitor/metadata-operation.md) in the Doris official website document.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q10-lost-connection-to-mysql-server-at-reading-initial-communication-packet-system-error-0">Q10. Lost connection to MySQL server at 'reading initial communication packet', system error: 0<a href="#q10-lost-connection-to-mysql-server-at-reading-initial-communication-packet-system-error-0" class="hash-link" aria-label="Direct link to Q10. Lost connection to MySQL server at 'reading initial communication packet', system error: 0" title="Direct link to Q10. Lost connection to MySQL server at 'reading initial communication packet', system error: 0"></a></h3><p>If the following problems occur when using MySQL client to connect to Doris, this is usually caused by the different jdk version used when compiling FE and the jdk version used when running FE. Note that when using docker to compile the image, the default JDK version is openjdk 11, and you can switch to openjdk 8 through the command (see the compilation documentation for details).</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q11-recoverytracker-should-overlap-or-follow-on-disk-last-vlsn-of-4422880-recoveryfirst-4422882-unexpected_state_fatal">Q11. recoveryTracker should overlap or follow on disk last VLSN of 4,422,880 recoveryFirst= 4,422,882 UNEXPECTED_STATE_FATAL<a href="#q11-recoverytracker-should-overlap-or-follow-on-disk-last-vlsn-of-4422880-recoveryfirst-4422882-unexpected_state_fatal" class="hash-link" aria-label="Direct link to Q11. recoveryTracker should overlap or follow on disk last VLSN of 4,422,880 recoveryFirst= 4,422,882 UNEXPECTED_STATE_FATAL" title="Direct link to Q11. recoveryTracker should overlap or follow on disk last VLSN of 4,422,880 recoveryFirst= 4,422,882 UNEXPECTED_STATE_FATAL"></a></h3><p>Sometimes when FE is restarted, the above error will occur (usually only in the case of multiple Followers). And the two values in the error differ by 2. Causes FE to fail to start.</p><p>This is a bug in bdbje that has not yet been resolved. In this case, you can only restore the metadata by performing the operation of failure recovery in <a href="/docs/2.0/admin-manual/maint-monitor/metadata-operation">Metadata Operation and Maintenance Documentation</a>.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q12-doris-compile-and-install-jdk-version-incompatibility-problem">Q12. Doris compile and install JDK version incompatibility problem<a href="#q12-doris-compile-and-install-jdk-version-incompatibility-problem" class="hash-link" aria-label="Direct link to Q12. Doris compile and install JDK version incompatibility problem" title="Direct link to Q12. Doris compile and install JDK version incompatibility problem"></a></h3><p>When compiling Doris using Docker, start FE after compiling and installing, and the exception message <code>java.lang.Suchmethoderror: java.nio.ByteBuffer.limit (I)Ljava/nio/ByteBuffer;</code> appears, this is because the default in Docker It is JDK 11. If your installation environment is using JDK8, you need to switch the JDK environment to JDK8 in Docker. For the specific switching method, please refer to <a href="/docs/2.0/install/source-install/compilation-with-docker">Compile Documentation</a></p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q13-error-starting-fe-or-unit-test-locally-cannot-find-external-parser-table-action_tabledat">Q13. Error starting FE or unit test locally Cannot find external parser table action_table.dat<a href="#q13-error-starting-fe-or-unit-test-locally-cannot-find-external-parser-table-action_tabledat" class="hash-link" aria-label="Direct link to Q13. Error starting FE or unit test locally Cannot find external parser table action_table.dat" title="Direct link to Q13. Error starting FE or unit test locally Cannot find external parser table action_table.dat"></a></h3><p>Run the following command</p><div class="codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">cd fe && mvn clean install -DskipTests</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>If the same error is reported, Run the following command</p><div class="codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">cp fe-core/target/generated-sources/cup/org/apache/doris/analysis/action_table.dat fe-core/target/classes/org/apache/doris/analysis</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q14-doris-upgrades-to-version-10-or-later-and-reports-error-failed-to-set-ciphers-to-use-2026-in-mysql-appearance-via-odbc">Q14. Doris upgrades to version 1.0 or later and reports error `<code>Failed to set ciphers to use (2026)</code> in MySQL appearance via ODBC.<a href="#q14-doris-upgrades-to-version-10-or-later-and-reports-error-failed-to-set-ciphers-to-use-2026-in-mysql-appearance-via-odbc" class="hash-link" aria-label="Direct link to q14-doris-upgrades-to-version-10-or-later-and-reports-error-failed-to-set-ciphers-to-use-2026-in-mysql-appearance-via-odbc" title="Direct link to q14-doris-upgrades-to-version-10-or-later-and-reports-error-failed-to-set-ciphers-to-use-2026-in-mysql-appearance-via-odbc"></a></h3><p>This problem occurs after doris upgrades to version 1.0 and uses Connector/ODBC 8.0.x or higher. Connector/ODBC 8.0.x has multiple access methods, such as <code>/usr/lib64/libmyodbc8w.so</code> which is installed via yum and relies on <code> libssl.so.10</code> and <code>libcrypto.so.10</code>. |
| In doris 1.0 onwards, openssl has been upgraded to 1.1 and is built into the doris binary package, so this can lead to openssl conflicts and errors like the following</p><div class="codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">ERROR 1105 (HY000): errCode = 2, detailMessage = driver connect Error: HY000 [MySQL][ODBC 8.0(w) Driver]SSL connection error: Failed to set ciphers to use (2026)</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>The solution is to use the <code>Connector/ODBC 8.0.28</code> version of ODBC Connector and select <code>Linux - Generic</code> in the operating system, this version of ODBC Driver uses openssl version 1.1. Or use a lower version of ODBC connector, e.g. <a href="https://dev.mysql.com/downloads/connector/odbc/5.3.html" target="_blank" rel="noopener noreferrer">Connector/ODBC 5.3.14</a>. For details, see the <a href="https://doris.apache.org/zh-CN/docs/1.2/lakehouse/external-table/odbc" target="_blank" rel="noopener noreferrer">ODBC exterior documentation</a>.</p><div class="theme-admonition theme-admonition-tip alert alert--success admonition_LlT9"><div class="admonitionHeading_tbUL"><span class="admonitionIcon_kALy"><svg viewBox="0 0 12 16"><path fill-rule="evenodd" d="M6.5 0C3.48 0 1 2.19 1 5c0 .92.55 2.25 1 3 1.34 2.25 1.78 2.78 2 4v1h5v-1c.22-1.22.66-1.75 2-4 .45-.75 1-2.08 1-3 0-2.81-2.48-5-5.5-5zm3.64 7.48c-.25.44-.47.8-.67 1.11-.86 1.41-1.25 2.06-1.45 3.23-.02.05-.02.11-.02.17H5c0-.06 0-.13-.02-.17-.2-1.17-.59-1.83-1.45-3.23-.2-.31-.42-.67-.67-1.11C2.44 6.78 2 5.65 2 5c0-2.2 2.02-4 4.5-4 1.22 0 2.36.42 3.22 1.19C10.55 2.94 11 3.94 11 5c0 .66-.44 1.78-.86 2.48zM4 14h5c-.23 1.14-1.3 2-2.5 2s-2.27-.86-2.5-2z"></path></svg></span>提示</div><div class="admonitionContent_S0QG"><p>ODBC is no supported in version 2.0 or later, it is recommended to use <a href="/docs/2.0/lakehouse/database/jdbc">JDBC Catalog</a>.</p></div></div><p>You can verify the version of openssl used by MySQL ODBC Driver by</p><div class="codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#F8F8F2;--prism-background-color:#282A36"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#F8F8F2"><span class="token plain">ldd /path/to/libmyodbc8w.so |grep libssl.so</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>If the output contains <code>libssl.so.10</code>, there may be problems using it, if it contains <code>libssl.so.1.1</code>, it is compatible with doris 1.0</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="q15-after-upgrading-to-version-12-the-be-noclassdeffounderror-issue-failed-to-start">Q15. After upgrading to version 1.2, the BE NoClassDefFoundError issue failed to start<a href="#q15-after-upgrading-to-version-12-the-be-noclassdeffounderror-issue-failed-to-start" class="hash-link" aria-label="Direct link to Q15. After upgrading to version 1.2, the BE NoClassDefFoundError issue failed to start" title="Direct link to Q15. After upgrading to version 1.2, the BE NoClassDefFoundError issue failed to start"></a></h3><span class="version-mark paragraph"><span class="v-mark"><span class="version-tags"><span class="version-tag"><span class="version-tag-t">Since</span><span class="version-tag-n since">Version <!-- -->1.2</span></span></span></span> Java UDF dependency error </span>If the upgrade support starts be, the following Java `NoClassDefFoundError` error occurs ``` Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/doris/udf/IniUtil Caused by: java.lang.ClassNotFoundException: org.apache.doris.udf.JniUtil ``` You need to download the Java UDF function dependency package of `apache-doris-java-udf-jar-with-dependencies-1.2.0` from the official website, put it in the lib directory under the BE installation directory, and then restart BE<h3 class="anchor anchorWithStickyNavbar_LWe7" id="q16-after-upgrading-to-version-12-be-startup-shows-failed-to-initialize-jni">Q16. After upgrading to version 1.2, BE startup shows Failed to initialize JNI<a href="#q16-after-upgrading-to-version-12-be-startup-shows-failed-to-initialize-jni" class="hash-link" aria-label="Direct link to Q16. After upgrading to version 1.2, BE startup shows Failed to initialize JNI" title="Direct link to Q16. After upgrading to version 1.2, BE startup shows Failed to initialize JNI"></a></h3><span class="version-mark paragraph"><span class="v-mark"><span class="version-tags"><span class="version-tag"><span class="version-tag-t">Since</span><span class="version-tag-n since">Version <!-- -->1.2</span></span></span></span></span>If the following `Failed to initialize JNI` error occurs when starting BE after upgrading ``` Failed to initialize JNI: Failed to find the library libjvm.so. ``` You need to set the `JAVA_HOME` environment variable, or set `JAVA_HOME` variable in be.conf and restart the BE node.<h3 class="anchor anchorWithStickyNavbar_LWe7" id="q17-docker-backend-fails-to-start">Q17. Docker: backend fails to start<a href="#q17-docker-backend-fails-to-start" class="hash-link" aria-label="Direct link to Q17. Docker: backend fails to start" title="Direct link to Q17. Docker: backend fails to start"></a></h3><p>This may be due to the CPU not supporting AVX2, check the backend logs with <code>docker logs -f be</code>. |
