| <!DOCTYPE html><html lang="en-us" class="__variable_1fc36d scroll-smooth"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="preload" href="/_next/static/media/6905431624c34d00-s.p.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" as="image" href="https://www.datocms-assets.com/75153/1689174701-image1.png"/><link rel="preload" as="image" href="https://www.datocms-assets.com/75153/1689175033-image4.png"/><link rel="preload" as="image" href="https://www.datocms-assets.com/75153/1689175176-image2.png"/><link rel="preload" as="image" href="https://www.datocms-assets.com/75153/1689175271-image3.png"/><link rel="stylesheet" href="/_next/static/css/9e925a33b1acdac1.css" crossorigin="" data-precedence="next"/><link rel="stylesheet" href="/_next/static/css/c130d1629644f070.css" crossorigin="" data-precedence="next"/><link rel="preload" as="script" fetchPriority="low" href="/_next/static/chunks/webpack-dde39ac7c1b4eb4b.js" crossorigin=""/><script src="/_next/static/chunks/fd9d1056-f172993f20f9bb67.js" async="" crossorigin=""></script><script src="/_next/static/chunks/472-928e738895d89765.js" async="" crossorigin=""></script><script src="/_next/static/chunks/main-app-344a87763a0866a6.js" async="" crossorigin=""></script><script src="/_next/static/chunks/326-3a90a6443b9c824c.js" async=""></script><script src="/_next/static/chunks/980-6e243f9cd384c7d2.js" async=""></script><script src="/_next/static/chunks/702-a2bf9fe707814b79.js" async=""></script><script src="/_next/static/chunks/app/layout-776a485845c720ef.js" async=""></script><script src="/_next/static/chunks/413-f9f40b83f7bb3f22.js" async=""></script><script src="/_next/static/chunks/app/blog/%5B...slug%5D/page-502e08b6677b55da.js" async=""></script><link rel="preload" href="https://www.googletagmanager.com/gtag/js?id=G-ZXG79NJEBY" as="script"/><title>Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios | Apache Pinot™</title><meta name="description" content="The blog post discusses how implementing a star-tree index significantly improved query performance for an AdTech platform by reducing latency. This index has also been successful in cybersecurity threat detection and multiplayer game leaderboard tracking, resulting in improved query performance and cost savings. Real production use cases showed a 95% to 99% improvement in query performance using StarTree Cloud for real-time analytics."/><meta name="robots" content="index, follow"/><meta name="googlebot" content="index, follow, max-video-preview:-1, max-image-preview:large, max-snippet:-1"/><link rel="canonical" href="https://pinot.apache.org/blog/2023/07/12/star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer"/><link rel="alternate" type="application/rss+xml" href="https://pinot.apache.org/feed.xml"/><meta property="og:title" content="Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios"/><meta property="og:description" content="The blog post discusses how implementing a star-tree index significantly improved query performance for an AdTech platform by reducing latency. This index has also been successful in cybersecurity threat detection and multiplayer game leaderboard tracking, resulting in improved query performance and cost savings. Real production use cases showed a 95% to 99% improvement in query performance using StarTree Cloud for real-time analytics."/><meta property="og:url" content="https://pinot.apache.org/blog/2023/07/12/star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer"/><meta property="og:site_name" content="Apache Pinot™"/><meta property="og:locale" content="en_US"/><meta property="og:image" content="https://pinot.apache.org/static/images/twitter-card.png"/><meta property="og:type" content="article"/><meta property="article:published_time" content="2023-07-12T00:00:00.000Z"/><meta property="article:modified_time" content="2023-07-12T00:00:00.000Z"/><meta property="article:author" content="Sandeep Dabade"/><meta property="article:author" content="Kulbir Nijjer"/><meta name="twitter:card" content="summary_large_image"/><meta name="twitter:title" content="Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios"/><meta name="twitter:description" content="The blog post discusses how implementing a star-tree index significantly improved query performance for an AdTech platform by reducing latency. This index has also been successful in cybersecurity threat detection and multiplayer game leaderboard tracking, resulting in improved query performance and cost savings. Real production use cases showed a 95% to 99% improvement in query performance using StarTree Cloud for real-time analytics."/><meta name="twitter:image" content="https://pinot.apache.org/static/images/twitter-card.png"/><meta name="next-size-adjust"/><meta http-equiv="Content-Security-Policy" content="default-src 'self';script-src 'self' 'unsafe-eval' 'unsafe-inline' giscus.app analytics.umami.is www.youtube.com www.googletagmanager.com www.google-analytics.com;style-src 'self' 'unsafe-inline';img-src * blob: data:;media-src *.s3.amazonaws.com;connect-src *;font-src 'self';frame-src www.youtube.com youtube.com giscus.app youtu.be https://www.youtube.com https://youtube.com;"/><link rel="apple-touch-icon" sizes="76x76" href="/static/favicons/apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="/static/favicons/favicon-32x32.png"/><link rel="icon" type="image/png" sizes="16x16" href="/static/favicons/favicon-16x16.png"/><link rel="manifest" href="/static/favicons/site.webmanifest"/><link rel="mask-icon" href="/static/favicons/safari-pinned-tab.svg" color="#5bbad5"/><meta name="msapplication-TileColor" content="#000000"/><meta name="theme-color" media="(prefers-color-scheme: light)" content="#fff"/><meta name="theme-color" media="(prefers-color-scheme: dark)" content="#000"/><link rel="alternate" type="application/rss+xml" href="/feed.xml"/><script src="/_next/static/chunks/polyfills-c67a75d1b6f99dc8.js" crossorigin="" noModule=""></script></head><body class="bg-white text-black antialiased dark:bg-gray-950 dark:text-white"><script>!function(){try{var d=document.documentElement,c=d.classList;c.remove('light','dark');var e=localStorage.getItem('theme');if('system'===e||(!e&&false)){var t='(prefers-color-scheme: dark)',m=window.matchMedia(t);if(m.media!==t||m.matches){d.style.colorScheme = 'dark';c.add('dark')}else{d.style.colorScheme = 'light';c.add('light')}}else if(e){c.add(e|| '')}else{c.add('light')}if(e==='light'||e==='dark'||!e)d.style.colorScheme=e||'light'}catch(e){}}()</script><div class="mx-auto flex max-w-screen-customDesktop flex-col justify-between font-sans"><div class="inset-x-0 top-0 z-50 flex text-center text-base sm:text-left"><div class="flex w-full flex-col items-center justify-center bg-sky-200 pt-1 md:flex-row md:pt-0"><div class="flex flex-wrap items-center justify-center md:justify-start">🎉🎉🎉 Announcing the release of Apache Pinot 1.1.0</div><div class="flex items-center justify-center"><a href="https://github.com/apache/pinot/releases/tag/release-1.1.0" target="_blank" class="inline-flex items-center justify-center whitespace-nowrap rounded-md ring-offset-background transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:pointer-events-none disabled:opacity-50 underline-offset-4 hover:text-vine-120 h-10 px-4 py-2 mr-2 text-base font-semibold leading-tight text-vine-100">learn more<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-arrow-right mr-2 h-5 w-5"><path d="M5 12h14"></path><path d="m12 5 7 7-7 7"></path></svg></a></div></div></div><header class="border-b-1 flex items-center justify-between border-b px-5 py-3 md:px-[4rem] md:py-4"><div class="flex"><a aria-label="" href="/"><div class="flex items-center justify-between"><div class=""><svg xmlns="http://www.w3.org/2000/svg" width="120" height="48" fill="none"><g fill="#C7154A" clip-path="url(#logo_svg__a)"><path d="M42.99 18.448c1.032-.553 2.21-.831 3.535-.831 1.542 0 2.938.38 4.187 1.14 1.248.76 2.236 1.841 2.965 3.241.728 1.402 1.091 3.025 1.091 4.872s-.363 3.482-1.091 4.903c-.729 1.424-1.717 2.525-2.965 3.307-1.25.782-2.645 1.173-4.187 1.173-1.325 0-2.493-.271-3.503-.815-1.01-.543-1.83-1.226-2.46-2.053v14.612H36V17.912h4.562v2.606c.586-.825 1.395-1.515 2.426-2.068l.002-.002m6.452 5.605c-.445-.793-1.032-1.395-1.76-1.808a4.72 4.72 0 0 0-2.362-.618c-.847 0-1.602.211-2.33.635-.728.423-1.315 1.038-1.76 1.841-.445.804-.668 1.749-.668 2.835 0 1.087.221 2.032.668 2.835.445.804 1.032 1.417 1.76 1.842a4.557 4.557 0 0 0 2.33.635 4.57 4.57 0 0 0 2.362-.652c.728-.435 1.313-1.053 1.76-1.856.445-.804.668-1.76.668-2.867s-.223-2.025-.668-2.818v-.004M62.947 17.912v18.051h-4.562V17.912h4.562m.551-6.079a2.833 2.833 0 1 1-5.666 0 2.833 2.833 0 0 1 5.666 0M82.954 19.687c1.325 1.358 1.988 3.253 1.988 5.685v10.59H80.38v-9.97c0-1.434-.358-2.537-1.075-3.307-.717-.772-1.695-1.157-2.933-1.157-1.239 0-2.254.387-2.982 1.157-.728.772-1.091 1.873-1.091 3.307v9.97h-4.562V17.91h4.562v2.248a6.322 6.322 0 0 1 2.33-1.841c.944-.445 1.981-.669 3.111-.669 2.15 0 3.889.68 5.214 2.037v.002M92.892 35.098c-1.39-.77-2.482-1.861-3.275-3.275-.794-1.411-1.19-3.041-1.19-4.888s.406-3.475 1.221-4.888a8.502 8.502 0 0 1 3.34-3.275c1.412-.772 2.987-1.157 4.725-1.157 1.739 0 3.312.387 4.725 1.157a8.5 8.5 0 0 1 3.34 3.275c.815 1.411 1.222 3.041 1.222 4.888s-.418 3.475-1.255 4.888a8.708 8.708 0 0 1-3.388 3.275c-1.424.772-3.014 1.157-4.774 1.157-1.76 0-3.301-.385-4.691-1.157m7.021-3.421c.729-.402 1.309-1.005 1.744-1.809.435-.803.651-1.781.651-2.933 0-1.715-.451-3.035-1.351-3.958-.902-.924-2.004-1.385-3.307-1.385s-2.395.461-3.275 1.385c-.88.923-1.32 2.243-1.32 3.958 0 1.715.428 3.035 1.287 3.958.858.924 1.938 1.385 3.241 1.385.825 0 1.602-.2 2.33-.603v.002M115.96 21.658v8.734c0 .608.147 1.048.44 1.32.293.271.787.406 1.482.406H120v3.845h-2.867c-3.845 0-5.766-1.868-5.766-5.605v-8.7h-2.15v-3.746h2.15V13l4.595-1v5.912h4.04v3.746h-4.042M20.03 46.757l-5.538-1.385A1.97 1.97 0 0 1 13 43.46v-5.462c0-.841.349-1.601.907-2.146a12.212 12.212 0 0 0 6.975-3.644c2.602-2.731 3.627-6.578 2.882-10.251L21 9h-4V4a1 1 0 0 0-2 0v7a1 1 0 0 1-2 0v-1a1 1 0 0 0-2 0v6.758a4.489 4.489 0 0 1 2.694-.755c2.278.095 4.156 1.934 4.297 4.21a4.501 4.501 0 0 1-6.992 4.029V29a1 1 0 0 1-2 0V7a1 1 0 0 0-2 0v2h-4L.237 21.957c-.745 3.675.279 7.52 2.882 10.251a12.202 12.202 0 0 0 6.975 3.644c.558.545.907 1.305.907 2.146V43.4c0 .938-.639 1.757-1.55 1.985l-5.48 1.37c-.57.143-.97.655-.97 1.243h18c0-.588-.4-1.1-.97-1.243v.002"></path><path d="M13.5 23a2.5 2.5 0 1 0 0-5 2.5 2.5 0 0 0 0 5M8 5a1 1 0 1 0 0-2 1 1 0 0 0 0 2M12 8a1 1 0 1 0 0-2 1 1 0 0 0 0 2M16 2a1 1 0 1 0 0-2 1 1 0 0 0 0 2"></path></g><defs><clipPath id="logo_svg__a"><path fill="#fff" d="M0 0h120v48H0z"></path></clipPath></defs></svg></div><div class="hidden h-6 text-2xl font-semibold sm:block"></div></div></a><div class="ml-[4.5rem] flex items-center gap-12 text-lg leading-5"><a target="_blank" rel="noopener noreferrer" href="https://docs.pinot.apache.org" class="hidden sm:block font-medium text-gray-900 dark:text-gray-100">Docs</a><a class="hidden sm:block font-medium text-gray-900 dark:text-gray-100" href="/download">Download</a><a class="hidden sm:block font-medium text-gray-900 dark:text-gray-100" href="/powered-by">Powered by</a><a class="hidden sm:block font-medium text-gray-900 dark:text-gray-100" href="/blog">Blog</a></div></div><button aria-label="Toggle Menu" class="sm:hidden"><svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" fill="none"><mask id="menu_svg__a" width="32" height="32" x="0" y="0" maskUnits="userSpaceOnUse" style="mask-type:alpha"><path fill="#D9D9D9" d="M0 0h32v32H0z"></path></mask><g mask="url(#menu_svg__a)"><path fill="#201F1F" d="M4.667 23.513v-2h22.666v2H4.667Zm0-6.513v-2h22.666v2H4.667Zm0-6.513v-2h22.666v2H4.667Z"></path></g></svg></button><div class="hidden gap-3 sm:flex"><button aria-label="Search"><svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="h-6 w-6 text-gray-900 dark:text-gray-100"><path stroke-linecap="round" stroke-linejoin="round" d="M21 21l-5.197-5.197m0 0A7.5 7.5 0 105.196 5.196a7.5 7.5 0 0010.607 10.607z"></path></svg></button><a target="_blank" rel="noopener noreferrer" href="https://github.com/apache/pinot" class="inline-flex items-center justify-center whitespace-nowrap font-medium ring-offset-background transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:pointer-events-none disabled:opacity-50 border border-input bg-background hover:bg-accent hover:text-accent-foreground rounded-md px-3 py-2 text-base"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" fill="currentColor" class="mr-2"><g clip-path="url(#github_svg__a)"><path fill-rule="evenodd" d="M12.01 0C5.369 0 0 5.5 0 12.304c0 5.44 3.44 10.043 8.212 11.673.597.122.815-.265.815-.59 0-.286-.02-1.264-.02-2.283-3.34.734-4.036-1.466-4.036-1.466-.537-1.426-1.332-1.793-1.332-1.793-1.094-.754.08-.754.08-.754 1.212.082 1.849 1.263 1.849 1.263 1.073 1.874 2.803 1.345 3.5 1.019.098-.795.417-1.345.755-1.65-2.665-.285-5.468-1.345-5.468-6.07 0-1.345.477-2.445 1.232-3.3-.119-.306-.537-1.57.12-3.26 0 0 1.014-.326 3.3 1.263.98-.27 1.989-.407 3.003-.408 1.014 0 2.048.143 3.002.408 2.287-1.59 3.301-1.263 3.301-1.263.657 1.69.239 2.954.12 3.26.775.855 1.232 1.955 1.232 3.3 0 4.725-2.803 5.764-5.488 6.07.438.387.815 1.12.815 2.281 0 1.65-.02 2.975-.02 3.382 0 .326.22.713.816.59C20.56 22.347 24 17.744 24 12.305 24.02 5.5 18.63 0 12.01 0" clip-rule="evenodd"></path></g><defs><clipPath id="github_svg__a"><path fill="#fff" d="M0 0h24v24H0z"></path></clipPath></defs></svg>3.5k</a><button class="inline-flex items-center justify-center whitespace-nowrap font-medium ring-offset-background transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:pointer-events-none disabled:opacity-50 text-primary-foreground hover:bg-vine-120 rounded-md bg-vine-100 px-6 py-2 text-base"><a target="_blank" rel="noopener noreferrer" href="https://docs.pinot.apache.org/basics/getting-started">Get Started</a></button></div></header><main><script type="application/ld+json">{"@context":"https://schema.org","@type":"BlogPosting","headline":"Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios","datePublished":"2023-07-12T00:00:00.000Z","dateModified":"2023-07-12T00:00:00.000Z","description":"The blog post discusses how implementing a star-tree index significantly improved query performance for an AdTech platform by reducing latency. This index has also been successful in cybersecurity threat detection and multiplayer game leaderboard tracking, resulting in improved query performance and cost savings. Real production use cases showed a 95% to 99% improvement in query performance using StarTree Cloud for real-time analytics.","image":"/static/images/twitter-card.png","url":"https://pinot.apache.org/blog/2023-07-12-star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer","author":[{"@type":"Person","name":"Sandeep Dabade"},{"@type":"Person","name":"Kulbir Nijjer"}]}</script><section class=" px-5 pt-10 md:px-[13.313rem] md:py-16"><div class="fixed bottom-8 right-8 hidden flex-col gap-3 md:hidden"><button aria-label="Scroll To Top" class="rounded-full bg-gray-200 p-2 text-gray-500 transition-all hover:bg-gray-300 dark:bg-gray-700 dark:text-gray-400 dark:hover:bg-gray-600"><svg class="h-5 w-5" viewBox="0 0 20 20" fill="currentColor"><path fill-rule="evenodd" d="M3.293 9.707a1 1 0 010-1.414l6-6a1 1 0 011.414 0l6 6a1 1 0 01-1.414 1.414L11 5.414V17a1 1 0 11-2 0V5.414L4.707 9.707a1 1 0 01-1.414 0z" clip-rule="evenodd"></path></svg></button></div><article class=""><div class="mx-auto lg:flex"><div class="lg:pr-12"><header class="pt-6 md:pr-10"><h1 class="text-4xl font-semibold">Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios</h1><p class="pt-2 text-lg">By: <!-- -->Sandeep Dabade, Kulbir Nijjer</p><p class="py-2 text-sm">July 12th, 2023<!-- --> • <!-- -->8 min read</p></header><div class="flex flex-col lg:flex-row"><main class=""><div class="prose max-w-[45rem] pb-8 pt-10 dark:prose-invert"><p>In <a target="_blank" rel="noopener noreferrer" href="https://startree.ai/blog/star-tree-indexes-in-apache-pinot-part-1-understanding-the-impact-on-query-performance">part 1 of this blog series</a>, we looked at how a star-tree index brought down standalone query latency on a sizable dataset of ~633M records from 1,513ms to 4ms! — nearly 380x faster.</p><p>In <a target="_blank" rel="noopener noreferrer" href="https://startree.ai/blog/star-tree-indexes-in-apache-pinot-part-2-understanding-the-impact-during-high-concurrency">part 2 of this blog series</a>, we imitated a real production scenario by firing hundreds of concurrent queries using JMeter and showcased how using a star-tree index helped achieve a >95% drop in p90th / p95th / p99th latencies and 126x increase in Throughput.</p><p>In this part, we will cover some real customer stories that have seen 95% to 99% improvement in query performance using Star-Tree Index.</p><h2 id="adtech-use-case"><a href="#adtech-use-case" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>AdTech Use Case</h2><p>This was for a leading AdTech platform and a somewhat typical use case; users of the platform (advertisers, publishers, and influencers) wanted to see fresh metrics on how their activities (such as online content, ad, and email campaigns) were performing in real-time so they could tweak things as needed. The application team wanted to provide a rich analytical interface to these users so that not only can they see the current performance but also do custom slicing and dicing of data over a period of time. For example, compare their current campaign performance to one they ran two weeks back, do cohort analysis, and so on.</p><h3 id="why-was-the-existing-system-not-working"><a href="#why-was-the-existing-system-not-working" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Why was the existing system not working?</h3><p>Their existing tech stack was a mix of OSS and custom-built in-house code, which was both operationally difficult to manage and costly to maintain. Yet more importantly, it wasn’t able to meet the basic throughput and latency requirements required by the platform to sustain user growth as well as provide richer analytic capabilities in the product.</p><h3 id="the-problem-and-challenges"><a href="#the-problem-and-challenges" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>The Problem and Challenges?</h3><p>When the StarTree Sales Engineering team was engaged, the requirements were very clear:</p><ul><li>Throughput: Support 50+ QPS during POC and 200+ for production)</li><li>Latency: P95th latency of 2s, including query that needed aggregation of ~ 2 billion rows</li><li>Scalability: Ability to scale efficiently with future growth in QPS in a non-linear manner</li></ul><p>The biggest challenge was the size of data — 20+ TB and growing — and on top of that, a complex aggregation query driving the summary view for users so they can drill further in to get more details.</p><p>This particular query needed to aggregate close to 2 Billion records at read time and then would be fired for every active user interacting with the platform (so high concurrent QPS). In this case, despite applying all relevant indexes available in their existing system, out-of-the-box query performance was still in the 6-8 seconds range, which is expected given that bulk of the work for the query is happening in the aggregation phase and not during the filtering phase (indexing helps with this).</p><p>In other OLAP systems they explored, the only option available to handle this use case was doing ingestion time rollups. In other words, changing the data to higher granularity. However, this obviously means losing access to raw data and also potentially re-bootstrapping if new use cases come down the road that need raw data access.</p><p>This is exactly the type of scenario that the <a target="_blank" rel="noopener noreferrer" href="https://docs.pinot.apache.org/basics/indexing/star-tree-index">Star-Tree Index</a>, unique to Apache Pinot, is designed to address - handle large aggregation queries at scale that need sub-second performance. The best part is you can apply it anytime without any need to reprocess the data or plan any system downtime. (Segment reload to apply table config changes run as a background task in Apache Pinot.) In this specific case, the same query latencies with the star-tree index applied went down to 15 ms. This implicitly meant that with the same infrastructure footprint, StarTree was able to support ~70 QPS (Queries Per Second) vs < 1 QPS for this most complex query; while still keeping the raw data intact.</p><h3 id="data-size-and-infra-footprint-for-the-pilot"><a href="#data-size-and-infra-footprint-for-the-pilot" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Data Size and Infra Footprint for the Pilot: </h3><ul><li>Total # of records: ~2 Trillion</li><li>Data Size: ~20 TB</li><li>Capacity: 72 vCPUs across 9 Pinot servers (8 vCPU, 64GB per node).</li></ul><h3 id="impact-summary"><a href="#impact-summary" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Impact Summary:</h3><ul><li>99.76% reduction in latency vs. no Star-Tree Index (6.3 seconds to 15 ms)</li><li>99.99999% reduction in amount of data scanned/aggregated per query (> 1.8B docs to < 2,400)</li></ul><p><img alt="Visualization of the impact of start-tree index for an AdTech use case with Apache Pinot" src="https://www.datocms-assets.com/75153/1689174701-image1.png"/></p><h2 id="cybersecurity-use-case"><a href="#cybersecurity-use-case" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>CyberSecurity Use Case:</h2><p>A cybersecurity company that provides their customers with a real-time threat detection platform with AI, allowing them to analyze network flow logs in real-time with a sophisticated reporting/analytical UI. The initial landing page inside the customer portal is a summary view of everything the platform was monitoring in the user's environment and then provides the capability to drill down into specifics of each. For example, filter requests by a specific application or IP Address.</p><h3 id="why-was-the-existing-system-not-working-1"><a href="#why-was-the-existing-system-not-working-1" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Why was the existing system not working?</h3><p>Their existing tech stack was a mix of Athena/Presto, which couldn’t meet the throughput and latency requirements with growing data volume across their customers. Additionally, operational overhead around managing some of these systems in-house led to increased cost.</p><h3 id="the-problem-and-challenges-1"><a href="#the-problem-and-challenges-1" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>The Problem and Challenges?</h3><p>Some of the key requirements that StarTree Cloud cluster had to meet:</p><ul><li>Throughput: Up to 200 QPS (200 projected by end of year)</li><li>Latency: < 1 second P99</li><li>High ingestion rate: 300k events/sec</li><li>ROI: Provide better cost efficiencies</li></ul><p>Similar to Use case #1, the customer wanted to retain data at the lowest granularity (so no ingestion roll-ups), and given the time column granularity similar challenge with running the complex aggregation query to power off the summary view. Additionally, the requirement to get double-digit throughput(QPS) for the POC with the most efficient compute footprint made it quite challenging.</p><p>Given the overhead while doing complex aggregations, efficient filtering (indexes) wasn’t enough - in this case, with 3 * 4-core/32GB nodes query took more than 15 seconds. We immediately switched the table config to add star-tree index to the table config and do a segment reload, and the results were phenomenal — query latency was reduced to 10ms.</p><h3 id="data-size-and-infra-footprint-for-the-pilot-1"><a href="#data-size-and-infra-footprint-for-the-pilot-1" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Data Size and Infra Footprint for the Pilot: </h3><ul><li>Total # of records: ~8 Billion</li><li>Data Size: 500+ GB</li><li>Capacity: 12 vCPUs across 3 Pinot servers (4-core/32GB)</li></ul><h3 id="impact-summary-1"><a href="#impact-summary-1" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Impact Summary:</h3><ul><li>99.94% reduction in query latency (achieving 100 QPS for the same query with no extra hardware)</li><li>99.9998% reduction in data scanned/aggregated per query</li><li>Happy Customer 😃</li></ul><p><img alt="Visualization of the impact of star-tree index for a Cybersecurity use case with Apache Pinot" src="https://www.datocms-assets.com/75153/1689175033-image4.png"/></p><h2 id="multiplayer-game-leaderboard-use-case"><a href="#multiplayer-game-leaderboard-use-case" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Multiplayer Game Leaderboard Use Case</h2><p>A global leader in the interactive entertainment field has an A/B Testing / Experimentation use case that tracks players’ activities to measure the player engagement on the new features being rolled out.</p><h3 id="the-problem-and-challenges-2"><a href="#the-problem-and-challenges-2" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>The Problem and Challenges?</h3><p>Some of the key requirements that StarTree Cloud cluster had to meet:</p><ul><li>Throughput: = 200 QPS</li><li>Latencies: < 1 second P99</li><li>Ingestion rate: 50K events/sec</li></ul><p>Given the overhead while doing complex aggregations, efficient filtering (indexes) wasn’t enough - in this case, with 1 * 4-core/32GB nodes query took 163 milliseconds. After switching to a star-tree index, the query latency was reduced to 7ms (a reduction of 95.7%).</p><h3 id="data-size-and-infra-footprint-for-the-pilot-2"><a href="#data-size-and-infra-footprint-for-the-pilot-2" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Data Size and Infra Footprint for the Pilot: </h3><ul><li>Total # of records: ~34 Million</li><li>Data Size: 500+ GB</li><li>Capacity: 4 vCPUs - 1 Pinot server (4-cores / 32 GB)</li></ul><h3 id="impact-summary-2"><a href="#impact-summary-2" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Impact Summary:</h3><ul><li>95.70% improvement in query performance as a result of 99.9962% reduction in number of documents and entries scanned.</li></ul><p><img alt="Visualization of the impact of star-tree index for a Gaming use case with Apache Pinot" src="https://www.datocms-assets.com/75153/1689175176-image2.png"/></p><h2 id="quick-recap-star-tree-index-performance-improvements"><a href="#quick-recap-star-tree-index-performance-improvements" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Quick Recap: Star-Tree Index Performance Improvements</h2><p><img alt="Recap Table of the Impact that star-tree index had on three real-world customers using Apache Pinot™" src="https://www.datocms-assets.com/75153/1689175271-image3.png"/></p><ul><li>99.99% reduction in data scanned/aggregated per query</li><li>95 to 99% improvement in query performance</li></ul><p>Disk IO is the most expensive operation in query processing. The star-tree index reduces Disk IO significantly. Instead of scanning raw documents from the disk and computing aggregates on the fly, star-tree index scans pre-aggregated documents for the combination of dimensions specified in the query from the disk.</p><p>In part 1 of the series, we saw that the star-tree index reduced the disk reads by 99.999% from 584 Million entries (in case of an inverted index) to 2,045. Query latency came down 99.67% from 1,513 ms to 4 ms! This, in itself, was a HUGE benefit.</p><p>In addition to the drastic improvement in query latency, the memory and CPU usage decreased significantly, freeing up resources for taking up more concurrent workloads. The cumulative effect was the 126 x increase in QPS on this small 4 vCPU Pinot Server, as we saw in part 2 blog of this series.</p><p>And finally, in this part 3 of the blog series, we covered three real production use cases that have seen 95% to 99% improvement in query performance using Star-Tree Index.</p><h2 id="intrigued-by-what-youve-read"><a href="#intrigued-by-what-youve-read" aria-hidden="true" tabindex="-1"><span class="icon icon-link"></span></a>Intrigued by What You’ve Read?</h2><p>The next step is to load your data into an open-source <a target="_blank" rel="noopener noreferrer" href="https://docs.pinot.apache.org/basics/getting-started">Apache Pinot</a> cluster or, if you prefer, a fully-managed database-as-a-service (DBaaS). Sign up today for a <a target="_blank" rel="noopener noreferrer" href="https://startree.ai/saas-signup">StarTree Cloud account</a>, free for 30 days. If you have more questions, sign up for the <a target="_blank" rel="noopener noreferrer" href="https://communityinviter.com/apps/startreedata/startree-community">StarTree Community Slack</a>.</p><p><a target="_blank" rel="noopener noreferrer" href="https://startree.ai/saas-signup">GET STARTED ON STARTREE CLOUD</a></p></div></main></div></div><aside class="mt-10 hidden border-l-2 pl-5 lg:sticky lg:top-1 lg:block lg:h-full"><section class="sticky top-0 mb-4 w-[15.375rem]"><div class="flex flex-col space-y-1.5 pb-3"><h3 class="text-sm font-semibold leading-snug text-neutral-500 dark:text-neutral-100">Table of Contents</h3></div><nav class="flex items-center self-start" aria-label="Table of Contents"><ol class="list-none space-y-3"><li class="text-sm leading-tight font-bold"><a href="#adtech-use-case">AdTech Use Case</a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#why-was-the-existing-system-not-working">Why was the existing system not working?</a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#the-problem-and-challenges">The Problem and Challenges?</a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#data-size-and-infra-footprint-for-the-pilot">Data Size and Infra Footprint for the Pilot: </a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#impact-summary">Impact Summary:</a></li><li class="text-sm font-normal leading-tight"><a href="#cybersecurity-use-case">CyberSecurity Use Case:</a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#why-was-the-existing-system-not-working">Why was the existing system not working?</a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#the-problem-and-challenges">The Problem and Challenges?</a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#data-size-and-infra-footprint-for-the-pilot">Data Size and Infra Footprint for the Pilot: </a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#impact-summary">Impact Summary:</a></li><li class="text-sm font-normal leading-tight"><a href="#multiplayer-game-leaderboard-use-case">Multiplayer Game Leaderboard Use Case</a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#the-problem-and-challenges">The Problem and Challenges?</a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#data-size-and-infra-footprint-for-the-pilot">Data Size and Infra Footprint for the Pilot: </a></li><li class="text-sm leading-tight ml-6 font-normal"><a href="#impact-summary">Impact Summary:</a></li><li class="text-sm font-normal leading-tight"><a href="#quick-recap-star-tree-index-performance-improvements">Quick Recap: Star-Tree Index Performance Improvements</a></li><li class="text-sm font-normal leading-tight"><a href="#intrigued-by-what-youve-read">Intrigued by What You’ve Read?</a></li></ol></nav></section></aside></div></article></section></main><footer class="border-t bg-sky-100 px-5 py-10 md:px-[6.75rem] md:pb-10 md:pt-16"><div class="mx-auto flex max-w-7xl flex-wrap justify-between"><div class="flex-shrink-0"><svg xmlns="http://www.w3.org/2000/svg" width="120" height="48" fill="none"><g fill="#C7154A" clip-path="url(#logo_svg__a)"><path d="M42.99 18.448c1.032-.553 2.21-.831 3.535-.831 1.542 0 2.938.38 4.187 1.14 1.248.76 2.236 1.841 2.965 3.241.728 1.402 1.091 3.025 1.091 4.872s-.363 3.482-1.091 4.903c-.729 1.424-1.717 2.525-2.965 3.307-1.25.782-2.645 1.173-4.187 1.173-1.325 0-2.493-.271-3.503-.815-1.01-.543-1.83-1.226-2.46-2.053v14.612H36V17.912h4.562v2.606c.586-.825 1.395-1.515 2.426-2.068l.002-.002m6.452 5.605c-.445-.793-1.032-1.395-1.76-1.808a4.72 4.72 0 0 0-2.362-.618c-.847 0-1.602.211-2.33.635-.728.423-1.315 1.038-1.76 1.841-.445.804-.668 1.749-.668 2.835 0 1.087.221 2.032.668 2.835.445.804 1.032 1.417 1.76 1.842a4.557 4.557 0 0 0 2.33.635 4.57 4.57 0 0 0 2.362-.652c.728-.435 1.313-1.053 1.76-1.856.445-.804.668-1.76.668-2.867s-.223-2.025-.668-2.818v-.004M62.947 17.912v18.051h-4.562V17.912h4.562m.551-6.079a2.833 2.833 0 1 1-5.666 0 2.833 2.833 0 0 1 5.666 0M82.954 19.687c1.325 1.358 1.988 3.253 1.988 5.685v10.59H80.38v-9.97c0-1.434-.358-2.537-1.075-3.307-.717-.772-1.695-1.157-2.933-1.157-1.239 0-2.254.387-2.982 1.157-.728.772-1.091 1.873-1.091 3.307v9.97h-4.562V17.91h4.562v2.248a6.322 6.322 0 0 1 2.33-1.841c.944-.445 1.981-.669 3.111-.669 2.15 0 3.889.68 5.214 2.037v.002M92.892 35.098c-1.39-.77-2.482-1.861-3.275-3.275-.794-1.411-1.19-3.041-1.19-4.888s.406-3.475 1.221-4.888a8.502 8.502 0 0 1 3.34-3.275c1.412-.772 2.987-1.157 4.725-1.157 1.739 0 3.312.387 4.725 1.157a8.5 8.5 0 0 1 3.34 3.275c.815 1.411 1.222 3.041 1.222 4.888s-.418 3.475-1.255 4.888a8.708 8.708 0 0 1-3.388 3.275c-1.424.772-3.014 1.157-4.774 1.157-1.76 0-3.301-.385-4.691-1.157m7.021-3.421c.729-.402 1.309-1.005 1.744-1.809.435-.803.651-1.781.651-2.933 0-1.715-.451-3.035-1.351-3.958-.902-.924-2.004-1.385-3.307-1.385s-2.395.461-3.275 1.385c-.88.923-1.32 2.243-1.32 3.958 0 1.715.428 3.035 1.287 3.958.858.924 1.938 1.385 3.241 1.385.825 0 1.602-.2 2.33-.603v.002M115.96 21.658v8.734c0 .608.147 1.048.44 1.32.293.271.787.406 1.482.406H120v3.845h-2.867c-3.845 0-5.766-1.868-5.766-5.605v-8.7h-2.15v-3.746h2.15V13l4.595-1v5.912h4.04v3.746h-4.042M20.03 46.757l-5.538-1.385A1.97 1.97 0 0 1 13 43.46v-5.462c0-.841.349-1.601.907-2.146a12.212 12.212 0 0 0 6.975-3.644c2.602-2.731 3.627-6.578 2.882-10.251L21 9h-4V4a1 1 0 0 0-2 0v7a1 1 0 0 1-2 0v-1a1 1 0 0 0-2 0v6.758a4.489 4.489 0 0 1 2.694-.755c2.278.095 4.156 1.934 4.297 4.21a4.501 4.501 0 0 1-6.992 4.029V29a1 1 0 0 1-2 0V7a1 1 0 0 0-2 0v2h-4L.237 21.957c-.745 3.675.279 7.52 2.882 10.251a12.202 12.202 0 0 0 6.975 3.644c.558.545.907 1.305.907 2.146V43.4c0 .938-.639 1.757-1.55 1.985l-5.48 1.37c-.57.143-.97.655-.97 1.243h18c0-.588-.4-1.1-.97-1.243v.002"></path><path d="M13.5 23a2.5 2.5 0 1 0 0-5 2.5 2.5 0 0 0 0 5M8 5a1 1 0 1 0 0-2 1 1 0 0 0 0 2M12 8a1 1 0 1 0 0-2 1 1 0 0 0 0 2M16 2a1 1 0 1 0 0-2 1 1 0 0 0 0 2"></path></g><defs><clipPath id="logo_svg__a"><path fill="#fff" d="M0 0h120v48H0z"></path></clipPath></defs></svg></div><div class="flex flex-wrap gap-x-16 gap-y-5 py-8 md:pl-24 md:pr-[21.625rem]"> <div><h5 class="mb-4 text-lg font-semibold">Resources</h5><div class="flex justify-between gap-x-10"><div class="flex flex-col"><a target="_blank" rel="noopener noreferrer" href="https://docs.pinot.apache.org/" class="block py-1 text-gray-600 hover:text-gray-900">Docs</a><a target="_blank" rel="noopener noreferrer" href="https://docs.pinot.apache.org/getting-started" class="block py-1 text-gray-600 hover:text-gray-900">Getting Started</a><a target="_blank" rel="noopener noreferrer" href="https://docs.pinot.apache.org/integrations/thirdeye" class="block py-1 text-gray-600 hover:text-gray-900">ThirdEye</a></div><div class="flex flex-col"><a class="block py-1 text-gray-600 hover:text-gray-900" href="/powered-by">Company Stories</a><a class="block py-1 text-gray-600 hover:text-gray-900" href="/download">Download</a><a class="block py-1 text-gray-600 hover:text-gray-900" href="/blog">Blog</a></div></div></div><div><h5 class="mb-4 text-lg font-semibold">Apache</h5><div class="flex justify-between gap-x-10"><div class="flex flex-col"><a target="_blank" rel="noopener noreferrer" href="https://www.apache.org" class="block py-1 text-gray-600 hover:text-gray-900">Foundation</a><a target="_blank" rel="noopener noreferrer" href="https://www.apache.org/licenses" class="block py-1 text-gray-600 hover:text-gray-900">License</a><a target="_blank" rel="noopener noreferrer" href="https://www.apache.org/security" class="block py-1 text-gray-600 hover:text-gray-900">Security</a></div><div class="flex flex-col"><a target="_blank" rel="noopener noreferrer" href="https://www.apache.org/foundation/sponsorship.html" class="block py-1 text-gray-600 hover:text-gray-900">Sponsorship</a><a target="_blank" rel="noopener noreferrer" href="https://www.apache.org/events/current-event" class="block py-1 text-gray-600 hover:text-gray-900">Events</a><a target="_blank" rel="noopener noreferrer" href="https://www.apache.org/foundation/thanks.html" class="block py-1 text-gray-600 hover:text-gray-900">Thanks</a></div></div></div></div><div class="mt-4 flex justify-center md:mt-0"><a target="_blank" rel="noopener noreferrer" href="https://join.slack.com/t/apache-pinot/shared_invite/zt-5z7pav2f-yYtjZdVA~EDmrGkho87Vzw" class="mr-4"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-slack fill-gray-900"><rect width="3" height="8" x="13" y="2" rx="1.5"></rect><path d="M19 8.5V10h1.5A1.5 1.5 0 1 0 19 8.5"></path><rect width="3" height="8" x="8" y="14" rx="1.5"></rect><path d="M5 15.5V14H3.5A1.5 1.5 0 1 0 5 15.5"></path><rect width="8" height="3" x="14" y="13" rx="1.5"></rect><path d="M15.5 19H14v1.5a1.5 1.5 0 1 0 1.5-1.5"></path><rect width="8" height="3" x="2" y="8" rx="1.5"></rect><path d="M8.5 5H10V3.5A1.5 1.5 0 1 0 8.5 5"></path></svg></a><a target="_blank" rel="noopener noreferrer" href="https://github.com/apache/pinot"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" fill="currentColor" size="24"><g clip-path="url(#github_svg__a)"><path fill-rule="evenodd" d="M12.01 0C5.369 0 0 5.5 0 12.304c0 5.44 3.44 10.043 8.212 11.673.597.122.815-.265.815-.59 0-.286-.02-1.264-.02-2.283-3.34.734-4.036-1.466-4.036-1.466-.537-1.426-1.332-1.793-1.332-1.793-1.094-.754.08-.754.08-.754 1.212.082 1.849 1.263 1.849 1.263 1.073 1.874 2.803 1.345 3.5 1.019.098-.795.417-1.345.755-1.65-2.665-.285-5.468-1.345-5.468-6.07 0-1.345.477-2.445 1.232-3.3-.119-.306-.537-1.57.12-3.26 0 0 1.014-.326 3.3 1.263.98-.27 1.989-.407 3.003-.408 1.014 0 2.048.143 3.002.408 2.287-1.59 3.301-1.263 3.301-1.263.657 1.69.239 2.954.12 3.26.775.855 1.232 1.955 1.232 3.3 0 4.725-2.803 5.764-5.488 6.07.438.387.815 1.12.815 2.281 0 1.65-.02 2.975-.02 3.382 0 .326.22.713.816.59C20.56 22.347 24 17.744 24 12.305 24.02 5.5 18.63 0 12.01 0" clip-rule="evenodd"></path></g><defs><clipPath id="github_svg__a"><path fill="#fff" d="M0 0h24v24H0z"></path></clipPath></defs></svg></a></div></div><div class="mt-8 border-t border-neutral-300 pt-4 text-left text-sm text-gray-600">Copyright © <!-- -->2024<!-- --> The Apache Software Foundation. Apache Pinot, Pinot, Apache, the Apache feather logo, and the Apache Pinot project logo are registered trademarks of The Apache Software Foundation. This page has references to third party software - Presto, PrestoDB, ThirdEye, Trino, TrinoDB, that are not part of the Apache Software Foundation and are not covered under the Apache License.</div></footer></div><script src="/_next/static/chunks/webpack-dde39ac7c1b4eb4b.js" crossorigin="" async=""></script><script>(self.__next_f=self.__next_f||[]).push([0]);self.__next_f.push([2,null])</script><script>self.__next_f.push([1,"1:HL[\"/_next/static/media/6905431624c34d00-s.p.woff2\",\"font\",{\"crossOrigin\":\"\",\"type\":\"font/woff2\"}]\n2:HL[\"/_next/static/css/9e925a33b1acdac1.css\",\"style\",{\"crossOrigin\":\"\"}]\n0:\"$L3\"\n"])</script><script>self.__next_f.push([1,"4:HL[\"/_next/static/css/c130d1629644f070.css\",\"style\",{\"crossOrigin\":\"\"}]\n"])</script><script>self.__next_f.push([1,"5:I[3728,[],\"\"]\n7:I[9928,[],\"\"]\n8:I[7821,[\"326\",\"static/chunks/326-3a90a6443b9c824c.js\",\"980\",\"static/chunks/980-6e243f9cd384c7d2.js\",\"702\",\"static/chunks/702-a2bf9fe707814b79.js\",\"185\",\"static/chunks/app/layout-776a485845c720ef.js\"],\"ThemeProviders\"]\n9:I[3994,[\"326\",\"static/chunks/326-3a90a6443b9c824c.js\",\"980\",\"static/chunks/980-6e243f9cd384c7d2.js\",\"702\",\"static/chunks/702-a2bf9fe707814b79.js\",\"185\",\"static/chunks/app/layout-776a485845c720ef.js\"],\"\"]\na:I[9640,[\"326\",\"static/chunks/326-3a90a6443b9c824c.js"])</script><script>self.__next_f.push([1,"\",\"980\",\"static/chunks/980-6e243f9cd384c7d2.js\",\"702\",\"static/chunks/702-a2bf9fe707814b79.js\",\"185\",\"static/chunks/app/layout-776a485845c720ef.js\"],\"AlgoliaSearchProvider\"]\nb:I[7975,[\"326\",\"static/chunks/326-3a90a6443b9c824c.js\",\"980\",\"static/chunks/980-6e243f9cd384c7d2.js\",\"702\",\"static/chunks/702-a2bf9fe707814b79.js\",\"185\",\"static/chunks/app/layout-776a485845c720ef.js\"],\"\"]\nc:I[6954,[],\"\"]\nd:I[7264,[],\"\"]\ne:I[8326,[\"326\",\"static/chunks/326-3a90a6443b9c824c.js\",\"413\",\"static/chunks/413-f9f40b83f7bb3f22.js\""])</script><script>self.__next_f.push([1,",\"980\",\"static/chunks/980-6e243f9cd384c7d2.js\",\"797\",\"static/chunks/app/blog/%5B...slug%5D/page-502e08b6677b55da.js\"],\"\"]\n11:T9fe,"])</script><script>self.__next_f.push([1,"M42.99 18.448c1.032-.553 2.21-.831 3.535-.831 1.542 0 2.938.38 4.187 1.14 1.248.76 2.236 1.841 2.965 3.241.728 1.402 1.091 3.025 1.091 4.872s-.363 3.482-1.091 4.903c-.729 1.424-1.717 2.525-2.965 3.307-1.25.782-2.645 1.173-4.187 1.173-1.325 0-2.493-.271-3.503-.815-1.01-.543-1.83-1.226-2.46-2.053v14.612H36V17.912h4.562v2.606c.586-.825 1.395-1.515 2.426-2.068l.002-.002m6.452 5.605c-.445-.793-1.032-1.395-1.76-1.808a4.72 4.72 0 0 0-2.362-.618c-.847 0-1.602.211-2.33.635-.728.423-1.315 1.038-1.76 1.841-.445.804-.668 1.749-.668 2.835 0 1.087.221 2.032.668 2.835.445.804 1.032 1.417 1.76 1.842a4.557 4.557 0 0 0 2.33.635 4.57 4.57 0 0 0 2.362-.652c.728-.435 1.313-1.053 1.76-1.856.445-.804.668-1.76.668-2.867s-.223-2.025-.668-2.818v-.004M62.947 17.912v18.051h-4.562V17.912h4.562m.551-6.079a2.833 2.833 0 1 1-5.666 0 2.833 2.833 0 0 1 5.666 0M82.954 19.687c1.325 1.358 1.988 3.253 1.988 5.685v10.59H80.38v-9.97c0-1.434-.358-2.537-1.075-3.307-.717-.772-1.695-1.157-2.933-1.157-1.239 0-2.254.387-2.982 1.157-.728.772-1.091 1.873-1.091 3.307v9.97h-4.562V17.91h4.562v2.248a6.322 6.322 0 0 1 2.33-1.841c.944-.445 1.981-.669 3.111-.669 2.15 0 3.889.68 5.214 2.037v.002M92.892 35.098c-1.39-.77-2.482-1.861-3.275-3.275-.794-1.411-1.19-3.041-1.19-4.888s.406-3.475 1.221-4.888a8.502 8.502 0 0 1 3.34-3.275c1.412-.772 2.987-1.157 4.725-1.157 1.739 0 3.312.387 4.725 1.157a8.5 8.5 0 0 1 3.34 3.275c.815 1.411 1.222 3.041 1.222 4.888s-.418 3.475-1.255 4.888a8.708 8.708 0 0 1-3.388 3.275c-1.424.772-3.014 1.157-4.774 1.157-1.76 0-3.301-.385-4.691-1.157m7.021-3.421c.729-.402 1.309-1.005 1.744-1.809.435-.803.651-1.781.651-2.933 0-1.715-.451-3.035-1.351-3.958-.902-.924-2.004-1.385-3.307-1.385s-2.395.461-3.275 1.385c-.88.923-1.32 2.243-1.32 3.958 0 1.715.428 3.035 1.287 3.958.858.924 1.938 1.385 3.241 1.385.825 0 1.602-.2 2.33-.603v.002M115.96 21.658v8.734c0 .608.147 1.048.44 1.32.293.271.787.406 1.482.406H120v3.845h-2.867c-3.845 0-5.766-1.868-5.766-5.605v-8.7h-2.15v-3.746h2.15V13l4.595-1v5.912h4.04v3.746h-4.042M20.03 46.757l-5.538-1.385A1.97 1.97 0 0 1 13 43.46v-5.462c0-.841.349-1.601.907-2.146a12.212 12.212 0 0 0 6.975-3.644c2.602-2.731 3.627-6.578 2.882-10.251L21 9h-4V4a1 1 0 0 0-2 0v7a1 1 0 0 1-2 0v-1a1 1 0 0 0-2 0v6.758a4.489 4.489 0 0 1 2.694-.755c2.278.095 4.156 1.934 4.297 4.21a4.501 4.501 0 0 1-6.992 4.029V29a1 1 0 0 1-2 0V7a1 1 0 0 0-2 0v2h-4L.237 21.957c-.745 3.675.279 7.52 2.882 10.251a12.202 12.202 0 0 0 6.975 3.644c.558.545.907 1.305.907 2.146V43.4c0 .938-.639 1.757-1.55 1.985l-5.48 1.37c-.57.143-.97.655-.97 1.243h18c0-.588-.4-1.1-.97-1.243v.002"])</script><script>self.__next_f.push([1,"3:[[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/9e925a33b1acdac1.css\",\"precedence\":\"next\",\"crossOrigin\":\"\"}]],[\"$\",\"$L5\",null,{\"buildId\":\"rmcKjFZ3e9kKdH1iJwCIQ\",\"assetPrefix\":\"\",\"initialCanonicalUrl\":\"/blog/2023/07/12/star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\",\"initialTree\":[\"\",{\"children\":[\"blog\",{\"children\":[[\"slug\",\"2023/07/12/star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\",\"c\"],{\"children\":[\"__PAGE__?{\\\"slug\\\":[\\\"2023\\\",\\\"07\\\",\\\"12\\\",\\\"star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\\\"]}\",{}]}]}]},\"$undefined\",\"$undefined\",true],\"initialHead\":[false,\"$L6\"],\"globalErrorComponent\":\"$7\",\"children\":[null,[\"$\",\"html\",null,{\"lang\":\"en-us\",\"className\":\"__variable_1fc36d scroll-smooth\",\"suppressHydrationWarning\":true,\"children\":[[\"$\",\"head\",null,{\"children\":[[\"$\",\"meta\",null,{\"httpEquiv\":\"Content-Security-Policy\",\"content\":\"default-src 'self';script-src 'self' 'unsafe-eval' 'unsafe-inline' giscus.app analytics.umami.is www.youtube.com www.googletagmanager.com www.google-analytics.com;style-src 'self' 'unsafe-inline';img-src * blob: data:;media-src *.s3.amazonaws.com;connect-src *;font-src 'self';frame-src www.youtube.com youtube.com giscus.app youtu.be https://www.youtube.com https://youtube.com;\"}],[\"$\",\"link\",null,{\"rel\":\"apple-touch-icon\",\"sizes\":\"76x76\",\"href\":\"/static/favicons/apple-touch-icon.png\"}],[\"$\",\"link\",null,{\"rel\":\"icon\",\"type\":\"image/png\",\"sizes\":\"32x32\",\"href\":\"/static/favicons/favicon-32x32.png\"}],[\"$\",\"link\",null,{\"rel\":\"icon\",\"type\":\"image/png\",\"sizes\":\"16x16\",\"href\":\"/static/favicons/favicon-16x16.png\"}],[\"$\",\"link\",null,{\"rel\":\"manifest\",\"href\":\"/static/favicons/site.webmanifest\"}],[\"$\",\"link\",null,{\"rel\":\"mask-icon\",\"href\":\"/static/favicons/safari-pinned-tab.svg\",\"color\":\"#5bbad5\"}],[\"$\",\"meta\",null,{\"name\":\"msapplication-TileColor\",\"content\":\"#000000\"}],[\"$\",\"meta\",null,{\"name\":\"theme-color\",\"media\":\"(prefers-color-scheme: light)\",\"content\":\"#fff\"}],[\"$\",\"meta\",null,{\"name\":\"theme-color\",\"media\":\"(prefers-color-scheme: dark)\",\"content\":\"#000\"}],[\"$\",\"link\",null,{\"rel\":\"alternate\",\"type\":\"application/rss+xml\",\"href\":\"/feed.xml\"}]]}],[\"$\",\"body\",null,{\"className\":\"bg-white text-black antialiased dark:bg-gray-950 dark:text-white\",\"children\":[\"$\",\"$L8\",null,{\"children\":[[\"$undefined\",\"$undefined\",\"$undefined\",\"$undefined\",[[\"$\",\"$L9\",null,{\"strategy\":\"afterInteractive\",\"src\":\"https://www.googletagmanager.com/gtag/js?id=G-ZXG79NJEBY\"}],[\"$\",\"$L9\",null,{\"strategy\":\"afterInteractive\",\"id\":\"ga-script\",\"children\":\"\\n window.dataLayer = window.dataLayer || [];\\n function gtag(){dataLayer.push(arguments);}\\n gtag('js', new Date());\\n gtag('config', 'G-ZXG79NJEBY');\\n \"}]]],[\"$\",\"div\",null,{\"className\":\"mx-auto flex max-w-screen-customDesktop flex-col justify-between font-sans\",\"children\":[\"$\",\"$La\",null,{\"algoliaConfig\":{\"appId\":\"CKRA00L2X9\",\"apiKey\":\"6531f8f7783a88d76629190843f1801e\",\"indexName\":\"prod_apache_pinot_docs\"},\"children\":[[\"$\",\"$Lb\",null,{}],[\"$\",\"main\",null,{\"children\":[\"$\",\"$Lc\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\"],\"loading\":\"$undefined\",\"loadingStyles\":\"$undefined\",\"loadingScripts\":\"$undefined\",\"hasLoading\":false,\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$Ld\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":[\"$\",\"div\",null,{\"className\":\"flex flex-col items-start justify-start md:mt-24 md:flex-row md:items-center md:justify-center md:space-x-6\",\"children\":[[\"$\",\"div\",null,{\"className\":\"space-x-2 pb-8 pt-6 md:space-y-5\",\"children\":[\"$\",\"h1\",null,{\"className\":\"text-6xl font-extrabold leading-9 tracking-tight text-gray-900 dark:text-gray-100 md:border-r-2 md:px-6 md:text-8xl md:leading-14\",\"children\":\"404\"}]}],[\"$\",\"div\",null,{\"className\":\"max-w-md\",\"children\":[[\"$\",\"p\",null,{\"className\":\"mb-4 text-xl font-bold leading-normal md:text-2xl\",\"children\":\"Sorry we couldn't find this page.\"}],[\"$\",\"p\",null,{\"className\":\"mb-8\",\"children\":\"But dont worry, you can find plenty of other things on our homepage.\"}],[\"$\",\"$Le\",null,{\"href\":\"/\",\"className\":\"focus:shadow-outline-blue inline rounded-lg border border-transparent bg-blue-600 px-4 py-2 text-sm font-medium leading-5 text-white shadow transition-colors duration-150 hover:bg-blue-700 focus:outline-none dark:hover:bg-blue-500\",\"children\":\"Back to homepage\"}]]}]]}],\"notFoundStyles\":[],\"initialChildNode\":[\"$\",\"$Lc\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\",\"blog\",\"children\"],\"loading\":\"$undefined\",\"loadingStyles\":\"$undefined\",\"loadingScripts\":\"$undefined\",\"hasLoading\":false,\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$Ld\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":\"$undefined\",\"notFoundStyles\":\"$undefined\",\"initialChildNode\":[\"$\",\"$Lc\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\",\"blog\",\"children\",[\"slug\",\"2023/07/12/star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\",\"c\"],\"children\"],\"loading\":\"$undefined\",\"loadingStyles\":\"$undefined\",\"loadingScripts\":\"$undefined\",\"hasLoading\":false,\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$Ld\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":\"$undefined\",\"notFoundStyles\":\"$undefined\",\"initialChildNode\":[\"$Lf\",\"$L10\",null],\"childPropSegment\":\"__PAGE__?{\\\"slug\\\":[\\\"2023\\\",\\\"07\\\",\\\"12\\\",\\\"star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\\\"]}\",\"styles\":[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/c130d1629644f070.css\",\"precedence\":\"next\",\"crossOrigin\":\"\"}]]}],\"childPropSegment\":[\"slug\",\"2023/07/12/star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\",\"c\"],\"styles\":null}],\"childPropSegment\":\"blog\",\"styles\":null}]}],[\"$\",\"footer\",null,{\"className\":\"border-t bg-sky-100 px-5 py-10 md:px-[6.75rem] md:pb-10 md:pt-16\",\"children\":[[\"$\",\"div\",null,{\"className\":\"mx-auto flex max-w-7xl flex-wrap justify-between\",\"children\":[[\"$\",\"div\",null,{\"className\":\"flex-shrink-0\",\"children\":[\"$\",\"svg\",null,{\"xmlns\":\"http://www.w3.org/2000/svg\",\"width\":120,\"height\":48,\"fill\":\"none\",\"children\":[[\"$\",\"g\",null,{\"fill\":\"#C7154A\",\"clipPath\":\"url(#logo_svg__a)\",\"children\":[[\"$\",\"path\",null,{\"d\":\"$11\"}],[\"$\",\"path\",null,{\"d\":\"M13.5 23a2.5 2.5 0 1 0 0-5 2.5 2.5 0 0 0 0 5M8 5a1 1 0 1 0 0-2 1 1 0 0 0 0 2M12 8a1 1 0 1 0 0-2 1 1 0 0 0 0 2M16 2a1 1 0 1 0 0-2 1 1 0 0 0 0 2\"}]]}],[\"$\",\"defs\",null,{\"children\":[\"$\",\"clipPath\",null,{\"id\":\"logo_svg__a\",\"children\":[\"$\",\"path\",null,{\"fill\":\"#fff\",\"d\":\"M0 0h120v48H0z\"}]}]}]]}]}],[\"$\",\"div\",null,{\"className\":\"flex flex-wrap gap-x-16 gap-y-5 py-8 md:pl-24 md:pr-[21.625rem]\",\"children\":[\" \",[[\"$\",\"div\",\"Resources\",{\"children\":[[\"$\",\"h5\",null,{\"className\":\"mb-4 text-lg font-semibold\",\"children\":\"Resources\"}],[\"$\",\"div\",null,{\"className\":\"flex justify-between gap-x-10\",\"children\":[[\"$\",\"div\",null,{\"className\":\"flex flex-col\",\"children\":[[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://docs.pinot.apache.org/\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Docs\"}],[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://docs.pinot.apache.org/getting-started\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Getting Started\"}],[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://docs.pinot.apache.org/integrations/thirdeye\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"ThirdEye\"}]]}],[\"$\",\"div\",null,{\"className\":\"flex flex-col\",\"children\":[[\"$\",\"$Le\",null,{\"href\":\"/powered-by\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Company Stories\"}],[\"$\",\"$Le\",null,{\"href\":\"/download\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Download\"}],[\"$\",\"$Le\",null,{\"href\":\"/blog\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Blog\"}]]}]]}]]}],[\"$\",\"div\",\"Apache\",{\"children\":[[\"$\",\"h5\",null,{\"className\":\"mb-4 text-lg font-semibold\",\"children\":\"Apache\"}],[\"$\",\"div\",null,{\"className\":\"flex justify-between gap-x-10\",\"children\":[[\"$\",\"div\",null,{\"className\":\"flex flex-col\",\"children\":[[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://www.apache.org\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Foundation\"}],[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://www.apache.org/licenses\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"License\"}],[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://www.apache.org/security\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Security\"}]]}],[\"$\",\"div\",null,{\"className\":\"flex flex-col\",\"children\":[[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://www.apache.org/foundation/sponsorship.html\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Sponsorship\"}],[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://www.apache.org/events/current-event\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Events\"}],[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://www.apache.org/foundation/thanks.html\",\"className\":\"block py-1 text-gray-600 hover:text-gray-900\",\"children\":\"Thanks\"}]]}]]}]]}]]]}],[\"$\",\"div\",null,{\"className\":\"mt-4 flex justify-center md:mt-0\",\"children\":[[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://join.slack.com/t/apache-pinot/shared_invite/zt-5z7pav2f-yYtjZdVA~EDmrGkho87Vzw\",\"className\":\"mr-4\",\"children\":[\"$\",\"svg\",null,{\"xmlns\":\"http://www.w3.org/2000/svg\",\"width\":24,\"height\":24,\"viewBox\":\"0 0 24 24\",\"fill\":\"none\",\"stroke\":\"currentColor\",\"strokeWidth\":2,\"strokeLinecap\":\"round\",\"strokeLinejoin\":\"round\",\"className\":\"lucide lucide-slack fill-gray-900\",\"children\":[[\"$\",\"rect\",\"diqz80\",{\"width\":\"3\",\"height\":\"8\",\"x\":\"13\",\"y\":\"2\",\"rx\":\"1.5\"}],[\"$\",\"path\",\"183iwg\",{\"d\":\"M19 8.5V10h1.5A1.5 1.5 0 1 0 19 8.5\"}],[\"$\",\"rect\",\"hqg7r1\",{\"width\":\"3\",\"height\":\"8\",\"x\":\"8\",\"y\":\"14\",\"rx\":\"1.5\"}],[\"$\",\"path\",\"76g71w\",{\"d\":\"M5 15.5V14H3.5A1.5 1.5 0 1 0 5 15.5\"}],[\"$\",\"rect\",\"1kmz0a\",{\"width\":\"8\",\"height\":\"3\",\"x\":\"14\",\"y\":\"13\",\"rx\":\"1.5\"}],[\"$\",\"path\",\"jc4sz0\",{\"d\":\"M15.5 19H14v1.5a1.5 1.5 0 1 0 1.5-1.5\"}],[\"$\",\"rect\",\"1omvl4\",{\"width\":\"8\",\"height\":\"3\",\"x\":\"2\",\"y\":\"8\",\"rx\":\"1.5\"}],[\"$\",\"path\",\"16f3cl\",{\"d\":\"M8.5 5H10V3.5A1.5 1.5 0 1 0 8.5 5\"}],\"$undefined\"]}]}],[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://github.com/apache/pinot\",\"children\":[\"$\",\"svg\",null,{\"xmlns\":\"http://www.w3.org/2000/svg\",\"width\":24,\"height\":24,\"fill\":\"currentColor\",\"size\":24,\"children\":[[\"$\",\"g\",null,{\"clipPath\":\"url(#github_svg__a)\",\"children\":[\"$\",\"path\",null,{\"fillRule\":\"evenodd\",\"d\":\"M12.01 0C5.369 0 0 5.5 0 12.304c0 5.44 3.44 10.043 8.212 11.673.597.122.815-.265.815-.59 0-.286-.02-1.264-.02-2.283-3.34.734-4.036-1.466-4.036-1.466-.537-1.426-1.332-1.793-1.332-1.793-1.094-.754.08-.754.08-.754 1.212.082 1.849 1.263 1.849 1.263 1.073 1.874 2.803 1.345 3.5 1.019.098-.795.417-1.345.755-1.65-2.665-.285-5.468-1.345-5.468-6.07 0-1.345.477-2.445 1.232-3.3-.119-.306-.537-1.57.12-3.26 0 0 1.014-.326 3.3 1.263.98-.27 1.989-.407 3.003-.408 1.014 0 2.048.143 3.002.408 2.287-1.59 3.301-1.263 3.301-1.263.657 1.69.239 2.954.12 3.26.775.855 1.232 1.955 1.232 3.3 0 4.725-2.803 5.764-5.488 6.07.438.387.815 1.12.815 2.281 0 1.65-.02 2.975-.02 3.382 0 .326.22.713.816.59C20.56 22.347 24 17.744 24 12.305 24.02 5.5 18.63 0 12.01 0\",\"clipRule\":\"evenodd\"}]}],[\"$\",\"defs\",null,{\"children\":[\"$\",\"clipPath\",null,{\"id\":\"github_svg__a\",\"children\":[\"$\",\"path\",null,{\"fill\":\"#fff\",\"d\":\"M0 0h24v24H0z\"}]}]}]]}]}]]}]]}],[\"$\",\"div\",null,{\"className\":\"mt-8 border-t border-neutral-300 pt-4 text-left text-sm text-gray-600\",\"children\":[\"Copyright © \",2024,\" The Apache Software Foundation. Apache Pinot, Pinot, Apache, the Apache feather logo, and the Apache Pinot project logo are registered trademarks of The Apache Software Foundation. This page has references to third party software - Presto, PrestoDB, ThirdEye, Trino, TrinoDB, that are not part of the Apache Software Foundation and are not covered under the Apache License.\"]}]]}]]}]}]]}]}]]}],null]}]]\n"])</script><script>self.__next_f.push([1,"12:I[1514,[\"326\",\"static/chunks/326-3a90a6443b9c824c.js\",\"413\",\"static/chunks/413-f9f40b83f7bb3f22.js\",\"980\",\"static/chunks/980-6e243f9cd384c7d2.js\",\"797\",\"static/chunks/app/blog/%5B...slug%5D/page-502e08b6677b55da.js\"],\"\"]\n13:I[5185,[\"326\",\"static/chunks/326-3a90a6443b9c824c.js\",\"413\",\"static/chunks/413-f9f40b83f7bb3f22.js\",\"980\",\"static/chunks/980-6e243f9cd384c7d2.js\",\"797\",\"static/chunks/app/blog/%5B...slug%5D/page-502e08b6677b55da.js\"],\"\"]\n"])</script><script>self.__next_f.push([1,"10:[[\"$\",\"script\",null,{\"type\":\"application/ld+json\",\"dangerouslySetInnerHTML\":{\"__html\":\"{\\\"@context\\\":\\\"https://schema.org\\\",\\\"@type\\\":\\\"BlogPosting\\\",\\\"headline\\\":\\\"Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios\\\",\\\"datePublished\\\":\\\"2023-07-12T00:00:00.000Z\\\",\\\"dateModified\\\":\\\"2023-07-12T00:00:00.000Z\\\",\\\"description\\\":\\\"The blog post discusses how implementing a star-tree index significantly improved query performance for an AdTech platform by reducing latency. This index has also been successful in cybersecurity threat detection and multiplayer game leaderboard tracking, resulting in improved query performance and cost savings. Real production use cases showed a 95% to 99% improvement in query performance using StarTree Cloud for real-time analytics.\\\",\\\"image\\\":\\\"/static/images/twitter-card.png\\\",\\\"url\\\":\\\"https://pinot.apache.org/blog/2023-07-12-star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\\\",\\\"author\\\":[{\\\"@type\\\":\\\"Person\\\",\\\"name\\\":\\\"Sandeep Dabade\\\"},{\\\"@type\\\":\\\"Person\\\",\\\"name\\\":\\\"Kulbir Nijjer\\\"}]}\"}}],[\"$\",\"section\",null,{\"className\":\" px-5 pt-10 md:px-[13.313rem] md:py-16\",\"children\":[[\"$\",\"$L12\",null,{}],[\"$\",\"article\",null,{\"className\":\"\",\"children\":[\"$\",\"div\",null,{\"className\":\"mx-auto lg:flex\",\"children\":[[\"$\",\"div\",null,{\"className\":\"lg:pr-12\",\"children\":[[\"$\",\"header\",null,{\"className\":\"pt-6 md:pr-10\",\"children\":[[\"$\",\"h1\",null,{\"className\":\"text-4xl font-semibold\",\"children\":\"Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios\"}],[\"$\",\"p\",null,{\"className\":\"pt-2 text-lg\",\"children\":[\"By: \",\"Sandeep Dabade, Kulbir Nijjer\"]}],[\"$\",\"p\",null,{\"className\":\"py-2 text-sm\",\"children\":[\"July 12th, 2023\",\" • \",\"8 min read\"]}]]}],[\"$\",\"div\",null,{\"className\":\"flex flex-col lg:flex-row\",\"children\":[\"$\",\"main\",null,{\"className\":\"\",\"children\":[\"$\",\"div\",null,{\"className\":\"prose max-w-[45rem] pb-8 pt-10 dark:prose-invert\",\"children\":[[\"$\",\"p\",null,{\"children\":[\"In \",[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://startree.ai/blog/star-tree-indexes-in-apache-pinot-part-1-understanding-the-impact-on-query-performance\",\"children\":\"part 1 of this blog series\"}],\", we looked at how a star-tree index brought down standalone query latency on a sizable dataset of ~633M records from 1,513ms to 4ms! — nearly 380x faster.\"]}],[\"$\",\"p\",null,{\"children\":[\"In \",[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://startree.ai/blog/star-tree-indexes-in-apache-pinot-part-2-understanding-the-impact-during-high-concurrency\",\"children\":\"part 2 of this blog series\"}],\", we imitated a real production scenario by firing hundreds of concurrent queries using JMeter and showcased how using a star-tree index helped achieve a \u003e95% drop in p90th / p95th / p99th latencies and 126x increase in Throughput.\"]}],[\"$\",\"p\",null,{\"children\":\"In this part, we will cover some real customer stories that have seen 95% to 99% improvement in query performance using Star-Tree Index.\"}],[\"$\",\"h2\",null,{\"id\":\"adtech-use-case\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#adtech-use-case\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"AdTech Use Case\"]}],[\"$\",\"p\",null,{\"children\":\"This was for a leading AdTech platform and a somewhat typical use case; users of the platform (advertisers, publishers, and influencers) wanted to see fresh metrics on how their activities (such as online content, ad, and email campaigns) were performing in real-time so they could tweak things as needed. The application team wanted to provide a rich analytical interface to these users so that not only can they see the current performance but also do custom slicing and dicing of data over a period of time. For example, compare their current campaign performance to one they ran two weeks back, do cohort analysis, and so on.\"}],[\"$\",\"h3\",null,{\"id\":\"why-was-the-existing-system-not-working\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#why-was-the-existing-system-not-working\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Why was the existing system not working?\"]}],[\"$\",\"p\",null,{\"children\":\"Their existing tech stack was a mix of OSS and custom-built in-house code, which was both operationally difficult to manage and costly to maintain. Yet more importantly, it wasn’t able to meet the basic throughput and latency requirements required by the platform to sustain user growth as well as provide richer analytic capabilities in the product.\"}],[\"$\",\"h3\",null,{\"id\":\"the-problem-and-challenges\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#the-problem-and-challenges\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"The Problem and Challenges?\"]}],[\"$\",\"p\",null,{\"children\":\"When the StarTree Sales Engineering team was engaged, the requirements were very clear:\"}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"Throughput: Support 50+ QPS during POC and 200+ for production)\"}],[\"$\",\"li\",null,{\"children\":\"Latency: P95th latency of 2s, including query that needed aggregation of ~ 2 billion rows\"}],[\"$\",\"li\",null,{\"children\":\"Scalability: Ability to scale efficiently with future growth in QPS in a non-linear manner\"}]]}],[\"$\",\"p\",null,{\"children\":\"The biggest challenge was the size of data — 20+ TB and growing — and on top of that, a complex aggregation query driving the summary view for users so they can drill further in to get more details.\"}],[\"$\",\"p\",null,{\"children\":\"This particular query needed to aggregate close to 2 Billion records at read time and then would be fired for every active user interacting with the platform (so high concurrent QPS). In this case, despite applying all relevant indexes available in their existing system, out-of-the-box query performance was still in the 6-8 seconds range, which is expected given that bulk of the work for the query is happening in the aggregation phase and not during the filtering phase (indexing helps with this).\"}],[\"$\",\"p\",null,{\"children\":\"In other OLAP systems they explored, the only option available to handle this use case was doing ingestion time rollups. In other words, changing the data to higher granularity. However, this obviously means losing access to raw data and also potentially re-bootstrapping if new use cases come down the road that need raw data access.\"}],[\"$\",\"p\",null,{\"children\":[\"This is exactly the type of scenario that the \",[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://docs.pinot.apache.org/basics/indexing/star-tree-index\",\"children\":\"Star-Tree Index\"}],\", unique to Apache Pinot, is designed to address - handle large aggregation queries at scale that need sub-second performance. The best part is you can apply it anytime without any need to reprocess the data or plan any system downtime. (Segment reload to apply table config changes run as a background task in Apache Pinot.) In this specific case, the same query latencies with the star-tree index applied went down to 15 ms. This implicitly meant that with the same infrastructure footprint, StarTree was able to support ~70 QPS (Queries Per Second) vs \u003c 1 QPS for this most complex query; while still keeping the raw data intact.\"]}],[\"$\",\"h3\",null,{\"id\":\"data-size-and-infra-footprint-for-the-pilot\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#data-size-and-infra-footprint-for-the-pilot\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Data Size and Infra Footprint for the Pilot: \"]}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"Total # of records: ~2 Trillion\"}],[\"$\",\"li\",null,{\"children\":\"Data Size: ~20 TB\"}],[\"$\",\"li\",null,{\"children\":\"Capacity: 72 vCPUs across 9 Pinot servers (8 vCPU, 64GB per node).\"}]]}],[\"$\",\"h3\",null,{\"id\":\"impact-summary\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#impact-summary\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Impact Summary:\"]}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"99.76% reduction in latency vs. no Star-Tree Index (6.3 seconds to 15 ms)\"}],[\"$\",\"li\",null,{\"children\":\"99.99999% reduction in amount of data scanned/aggregated per query (\u003e 1.8B docs to \u003c 2,400)\"}]]}],[\"$\",\"p\",null,{\"children\":[\"$\",\"img\",null,{\"alt\":\"Visualization of the impact of start-tree index for an AdTech use case with Apache Pinot\",\"src\":\"https://www.datocms-assets.com/75153/1689174701-image1.png\"}]}],[\"$\",\"h2\",null,{\"id\":\"cybersecurity-use-case\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#cybersecurity-use-case\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"CyberSecurity Use Case:\"]}],[\"$\",\"p\",null,{\"children\":\"A cybersecurity company that provides their customers with a real-time threat detection platform with AI, allowing them to analyze network flow logs in real-time with a sophisticated reporting/analytical UI. The initial landing page inside the customer portal is a summary view of everything the platform was monitoring in the user's environment and then provides the capability to drill down into specifics of each. For example, filter requests by a specific application or IP Address.\"}],[\"$\",\"h3\",null,{\"id\":\"why-was-the-existing-system-not-working-1\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#why-was-the-existing-system-not-working-1\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Why was the existing system not working?\"]}],[\"$\",\"p\",null,{\"children\":\"Their existing tech stack was a mix of Athena/Presto, which couldn’t meet the throughput and latency requirements with growing data volume across their customers. Additionally, operational overhead around managing some of these systems in-house led to increased cost.\"}],[\"$\",\"h3\",null,{\"id\":\"the-problem-and-challenges-1\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#the-problem-and-challenges-1\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"The Problem and Challenges?\"]}],[\"$\",\"p\",null,{\"children\":\"Some of the key requirements that StarTree Cloud cluster had to meet:\"}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"Throughput: Up to 200 QPS (200 projected by end of year)\"}],[\"$\",\"li\",null,{\"children\":\"Latency: \u003c 1 second P99\"}],[\"$\",\"li\",null,{\"children\":\"High ingestion rate: 300k events/sec\"}],[\"$\",\"li\",null,{\"children\":\"ROI: Provide better cost efficiencies\"}]]}],[\"$\",\"p\",null,{\"children\":\"Similar to Use case #1, the customer wanted to retain data at the lowest granularity (so no ingestion roll-ups), and given the time column granularity similar challenge with running the complex aggregation query to power off the summary view. Additionally, the requirement to get double-digit throughput(QPS) for the POC with the most efficient compute footprint made it quite challenging.\"}],[\"$\",\"p\",null,{\"children\":\"Given the overhead while doing complex aggregations, efficient filtering (indexes) wasn’t enough - in this case, with 3 * 4-core/32GB nodes query took more than 15 seconds. We immediately switched the table config to add star-tree index to the table config and do a segment reload, and the results were phenomenal — query latency was reduced to 10ms.\"}],[\"$\",\"h3\",null,{\"id\":\"data-size-and-infra-footprint-for-the-pilot-1\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#data-size-and-infra-footprint-for-the-pilot-1\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Data Size and Infra Footprint for the Pilot: \"]}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"Total # of records: ~8 Billion\"}],[\"$\",\"li\",null,{\"children\":\"Data Size: 500+ GB\"}],[\"$\",\"li\",null,{\"children\":\"Capacity: 12 vCPUs across 3 Pinot servers (4-core/32GB)\"}]]}],[\"$\",\"h3\",null,{\"id\":\"impact-summary-1\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#impact-summary-1\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Impact Summary:\"]}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"99.94% reduction in query latency (achieving 100 QPS for the same query with no extra hardware)\"}],[\"$\",\"li\",null,{\"children\":\"99.9998% reduction in data scanned/aggregated per query\"}],[\"$\",\"li\",null,{\"children\":\"Happy Customer 😃\"}]]}],[\"$\",\"p\",null,{\"children\":[\"$\",\"img\",null,{\"alt\":\"Visualization of the impact of star-tree index for a Cybersecurity use case with Apache Pinot\",\"src\":\"https://www.datocms-assets.com/75153/1689175033-image4.png\"}]}],[\"$\",\"h2\",null,{\"id\":\"multiplayer-game-leaderboard-use-case\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#multiplayer-game-leaderboard-use-case\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Multiplayer Game Leaderboard Use Case\"]}],[\"$\",\"p\",null,{\"children\":\"A global leader in the interactive entertainment field has an A/B Testing / Experimentation use case that tracks players’ activities to measure the player engagement on the new features being rolled out.\"}],[\"$\",\"h3\",null,{\"id\":\"the-problem-and-challenges-2\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#the-problem-and-challenges-2\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"The Problem and Challenges?\"]}],[\"$\",\"p\",null,{\"children\":\"Some of the key requirements that StarTree Cloud cluster had to meet:\"}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"Throughput: = 200 QPS\"}],[\"$\",\"li\",null,{\"children\":\"Latencies: \u003c 1 second P99\"}],[\"$\",\"li\",null,{\"children\":\"Ingestion rate: 50K events/sec\"}]]}],[\"$\",\"p\",null,{\"children\":\"Given the overhead while doing complex aggregations, efficient filtering (indexes) wasn’t enough - in this case, with 1 * 4-core/32GB nodes query took 163 milliseconds. After switching to a star-tree index, the query latency was reduced to 7ms (a reduction of 95.7%).\"}],[\"$\",\"h3\",null,{\"id\":\"data-size-and-infra-footprint-for-the-pilot-2\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#data-size-and-infra-footprint-for-the-pilot-2\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Data Size and Infra Footprint for the Pilot: \"]}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"Total # of records: ~34 Million\"}],[\"$\",\"li\",null,{\"children\":\"Data Size: 500+ GB\"}],[\"$\",\"li\",null,{\"children\":\"Capacity: 4 vCPUs - 1 Pinot server (4-cores / 32 GB)\"}]]}],[\"$\",\"h3\",null,{\"id\":\"impact-summary-2\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#impact-summary-2\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Impact Summary:\"]}],[\"$\",\"ul\",null,{\"children\":[\"$\",\"li\",null,{\"children\":\"95.70% improvement in query performance as a result of 99.9962% reduction in number of documents and entries scanned.\"}]}],[\"$\",\"p\",null,{\"children\":[\"$\",\"img\",null,{\"alt\":\"Visualization of the impact of star-tree index for a Gaming use case with Apache Pinot\",\"src\":\"https://www.datocms-assets.com/75153/1689175176-image2.png\"}]}],[\"$\",\"h2\",null,{\"id\":\"quick-recap-star-tree-index-performance-improvements\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#quick-recap-star-tree-index-performance-improvements\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Quick Recap: Star-Tree Index Performance Improvements\"]}],[\"$\",\"p\",null,{\"children\":[\"$\",\"img\",null,{\"alt\":\"Recap Table of the Impact that star-tree index had on three real-world customers using Apache Pinot™\",\"src\":\"https://www.datocms-assets.com/75153/1689175271-image3.png\"}]}],[\"$\",\"ul\",null,{\"children\":[[\"$\",\"li\",null,{\"children\":\"99.99% reduction in data scanned/aggregated per query\"}],[\"$\",\"li\",null,{\"children\":\"95 to 99% improvement in query performance\"}]]}],[\"$\",\"p\",null,{\"children\":\"Disk IO is the most expensive operation in query processing. The star-tree index reduces Disk IO significantly. Instead of scanning raw documents from the disk and computing aggregates on the fly, star-tree index scans pre-aggregated documents for the combination of dimensions specified in the query from the disk.\"}],[\"$\",\"p\",null,{\"children\":\"In part 1 of the series, we saw that the star-tree index reduced the disk reads by 99.999% from 584 Million entries (in case of an inverted index) to 2,045. Query latency came down 99.67% from 1,513 ms to 4 ms! This, in itself, was a HUGE benefit.\"}],[\"$\",\"p\",null,{\"children\":\"In addition to the drastic improvement in query latency, the memory and CPU usage decreased significantly, freeing up resources for taking up more concurrent workloads. The cumulative effect was the 126 x increase in QPS on this small 4 vCPU Pinot Server, as we saw in part 2 blog of this series.\"}],[\"$\",\"p\",null,{\"children\":\"And finally, in this part 3 of the blog series, we covered three real production use cases that have seen 95% to 99% improvement in query performance using Star-Tree Index.\"}],[\"$\",\"h2\",null,{\"id\":\"intrigued-by-what-youve-read\",\"children\":[[\"$\",\"a\",null,{\"href\":\"#intrigued-by-what-youve-read\",\"aria-hidden\":\"true\",\"tabIndex\":\"-1\",\"children\":[\"$\",\"span\",null,{\"className\":\"icon icon-link\"}]}],\"Intrigued by What You’ve Read?\"]}],[\"$\",\"p\",null,{\"children\":[\"The next step is to load your data into an open-source \",[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://docs.pinot.apache.org/basics/getting-started\",\"children\":\"Apache Pinot\"}],\" cluster or, if you prefer, a fully-managed database-as-a-service (DBaaS). Sign up today for a \",[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://startree.ai/saas-signup\",\"children\":\"StarTree Cloud account\"}],\", free for 30 days. If you have more questions, sign up for the \",[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://communityinviter.com/apps/startreedata/startree-community\",\"children\":\"StarTree Community Slack\"}],\".\"]}],[\"$\",\"p\",null,{\"children\":[\"$\",\"a\",null,{\"target\":\"_blank\",\"rel\":\"noopener noreferrer\",\"href\":\"https://startree.ai/saas-signup\",\"children\":\"GET STARTED ON STARTREE CLOUD\"}]}]]}]}]}]]}],[\"$\",\"aside\",null,{\"className\":\"mt-10 hidden border-l-2 pl-5 lg:sticky lg:top-1 lg:block lg:h-full\",\"children\":[\"$\",\"section\",null,{\"className\":\"sticky top-0 mb-4 w-[15.375rem]\",\"children\":[[\"$\",\"div\",null,{\"className\":\"flex flex-col space-y-1.5 pb-3\",\"children\":[\"$\",\"h3\",null,{\"className\":\"text-sm font-semibold leading-snug text-neutral-500 dark:text-neutral-100\",\"children\":\"Table of Contents\"}]}],[\"$\",\"$L13\",null,{\"chapters\":[{\"value\":\"AdTech Use Case\",\"url\":\"#adtech-use-case\",\"depth\":2},{\"value\":\"Why was the existing system not working?\",\"url\":\"#why-was-the-existing-system-not-working\",\"depth\":3},{\"value\":\"The Problem and Challenges?\",\"url\":\"#the-problem-and-challenges\",\"depth\":3},{\"value\":\"Data Size and Infra Footprint for the Pilot: \",\"url\":\"#data-size-and-infra-footprint-for-the-pilot\",\"depth\":3},{\"value\":\"Impact Summary:\",\"url\":\"#impact-summary\",\"depth\":3},{\"value\":\"CyberSecurity Use Case:\",\"url\":\"#cybersecurity-use-case\",\"depth\":2},{\"value\":\"Why was the existing system not working?\",\"url\":\"#why-was-the-existing-system-not-working\",\"depth\":3},{\"value\":\"The Problem and Challenges?\",\"url\":\"#the-problem-and-challenges\",\"depth\":3},{\"value\":\"Data Size and Infra Footprint for the Pilot: \",\"url\":\"#data-size-and-infra-footprint-for-the-pilot\",\"depth\":3},{\"value\":\"Impact Summary:\",\"url\":\"#impact-summary\",\"depth\":3},{\"value\":\"Multiplayer Game Leaderboard Use Case\",\"url\":\"#multiplayer-game-leaderboard-use-case\",\"depth\":2},{\"value\":\"The Problem and Challenges?\",\"url\":\"#the-problem-and-challenges\",\"depth\":3},{\"value\":\"Data Size and Infra Footprint for the Pilot: \",\"url\":\"#data-size-and-infra-footprint-for-the-pilot\",\"depth\":3},{\"value\":\"Impact Summary:\",\"url\":\"#impact-summary\",\"depth\":3},{\"value\":\"Quick Recap: Star-Tree Index Performance Improvements\",\"url\":\"#quick-recap-star-tree-index-performance-improvements\",\"depth\":2},{\"value\":\"Intrigued by What You’ve Read?\",\"url\":\"#intrigued-by-what-youve-read\",\"depth\":2}]}]]}]}]]}]}]]}]]\n"])</script><script>self.__next_f.push([1,"6:[[\"$\",\"meta\",\"0\",{\"name\":\"viewport\",\"content\":\"width=device-width, initial-scale=1\"}],[\"$\",\"meta\",\"1\",{\"charSet\":\"utf-8\"}],[\"$\",\"title\",\"2\",{\"children\":\"Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios | Apache Pinot™\"}],[\"$\",\"meta\",\"3\",{\"name\":\"description\",\"content\":\"The blog post discusses how implementing a star-tree index significantly improved query performance for an AdTech platform by reducing latency. This index has also been successful in cybersecurity threat detection and multiplayer game leaderboard tracking, resulting in improved query performance and cost savings. Real production use cases showed a 95% to 99% improvement in query performance using StarTree Cloud for real-time analytics.\"}],[\"$\",\"meta\",\"4\",{\"name\":\"robots\",\"content\":\"index, follow\"}],[\"$\",\"meta\",\"5\",{\"name\":\"googlebot\",\"content\":\"index, follow, max-video-preview:-1, max-image-preview:large, max-snippet:-1\"}],[\"$\",\"link\",\"6\",{\"rel\":\"canonical\",\"href\":\"https://pinot.apache.org/blog/2023/07/12/star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\"}],[\"$\",\"link\",\"7\",{\"rel\":\"alternate\",\"type\":\"application/rss+xml\",\"href\":\"https://pinot.apache.org/feed.xml\"}],[\"$\",\"meta\",\"8\",{\"property\":\"og:title\",\"content\":\"Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios\"}],[\"$\",\"meta\",\"9\",{\"property\":\"og:description\",\"content\":\"The blog post discusses how implementing a star-tree index significantly improved query performance for an AdTech platform by reducing latency. This index has also been successful in cybersecurity threat detection and multiplayer game leaderboard tracking, resulting in improved query performance and cost savings. Real production use cases showed a 95% to 99% improvement in query performance using StarTree Cloud for real-time analytics.\"}],[\"$\",\"meta\",\"10\",{\"property\":\"og:url\",\"content\":\"https://pinot.apache.org/blog/2023/07/12/star-tree-index-in-apache-pinot-part-3-understanding-the-impact-in-real-customer\"}],[\"$\",\"meta\",\"11\",{\"property\":\"og:site_name\",\"content\":\"Apache Pinot™\"}],[\"$\",\"meta\",\"12\",{\"property\":\"og:locale\",\"content\":\"en_US\"}],[\"$\",\"meta\",\"13\",{\"property\":\"og:image\",\"content\":\"https://pinot.apache.org/static/images/twitter-card.png\"}],[\"$\",\"meta\",\"14\",{\"property\":\"og:type\",\"content\":\"article\"}],[\"$\",\"meta\",\"15\",{\"property\":\"article:published_time\",\"content\":\"2023-07-12T00:00:00.000Z\"}],[\"$\",\"meta\",\"16\",{\"property\":\"article:modified_time\",\"content\":\"2023-07-12T00:00:00.000Z\"}],[\"$\",\"meta\",\"17\",{\"property\":\"article:author\",\"content\":\"Sandeep Dabade\"}],[\"$\",\"meta\",\"18\",{\"property\":\"article:author\",\"content\":\"Kulbir Nijjer\"}],[\"$\",\"meta\",\"19\",{\"name\":\"twitter:card\",\"content\":\"summary_large_image\"}],[\"$\",\"meta\",\"20\",{\"name\":\"twitter:title\",\"content\":\"Star-Tree Index in Apache Pinot - Part 3 - Understanding the Impact in Real Customer Scenarios\"}],[\"$\",\"meta\",\"21\",{\"name\":\"twitter:description\",\"content\":\"The blog post discusses how implementing a star-tree index significantly improved query performance for an AdTech platform by reducing latency. This index has also been successful in cybersecurity threat detection and multiplayer game leaderboard tracking, resulting in improved query performance and cost savings. Real production use cases showed a 95% to 99% improvement in query performance using StarTree Cloud for real-time analytics.\"}],[\"$\",\"meta\",\"22\",{\"name\":\"twitter:image\",\"content\":\"https://pinot.apache.org/static/images/twitter-card.png\"}],[\"$\",\"meta\",\"23\",{\"name\":\"next-size-adjust\"}]]\n"])</script><script>self.__next_f.push([1,"f:null\n"])</script><script>self.__next_f.push([1,""])</script></body></html> |