| <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="generator" content="rustdoc"><meta name="description" content="Tantivy can (if instructed to do so in the schema) store the term positions in a given field. This position is expressed as token ordinal. For instance, In “The beauty and the beast”, the term “the” appears in position 0 and position 3. This information is useful to run phrase queries."><meta name="keywords" content="rust, rustlang, rust-lang, positions"><title>tantivy::positions - Rust</title><link rel="preload" as="font" type="font/woff2" crossorigin href="../../SourceSerif4-Regular.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../FiraSans-Regular.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../FiraSans-Medium.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../SourceCodePro-Regular.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../SourceSerif4-Bold.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../SourceCodePro-Semibold.ttf.woff2"><link rel="stylesheet" href="../../normalize.css"><link rel="stylesheet" href="../../rustdoc.css" id="mainThemeStyle"><link rel="stylesheet" href="../../ayu.css" disabled><link rel="stylesheet" href="../../dark.css" disabled><link rel="stylesheet" href="../../light.css" id="themeStyle"><script id="default-settings" ></script><script src="../../storage.js"></script><script defer src="../../main.js"></script><noscript><link rel="stylesheet" href="../../noscript.css"></noscript><link rel="alternate icon" type="image/png" href="../../favicon-16x16.png"><link rel="alternate icon" type="image/png" href="../../favicon-32x32.png"><link rel="icon" type="image/svg+xml" href="../../favicon.svg"></head><body class="rustdoc mod"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><nav class="mobile-topbar"><button class="sidebar-menu-toggle">☰</button><a class="sidebar-logo" href="../../tantivy/index.html"><div class="logo-container"><img src="http://fulmicoton.com/tantivy-logo/tantivy-logo.png" alt="logo"></div></a><h2></h2></nav><nav class="sidebar"><a class="sidebar-logo" href="../../tantivy/index.html"><div class="logo-container"> |
| <img src="http://fulmicoton.com/tantivy-logo/tantivy-logo.png" alt="logo"></div></a><h2 class="location"><a href="#">Module positions</a></h2><div class="sidebar-elems"><section><ul class="block"><li><a href="#structs">Structs</a></li></ul></section></div></nav><main><div class="width-limiter"><nav class="sub"><form class="search-form"><div class="search-container"><span></span><input class="search-input" name="search" autocomplete="off" spellcheck="false" placeholder="Click or press ‘S’ to search, ‘?’ for more options…" type="search"><div id="help-button" title="help" tabindex="-1"><a href="../../help.html">?</a></div><div id="settings-menu" tabindex="-1"><a href="../../settings.html" title="settings"><img width="22" height="22" alt="Change settings" src="../../wheel.svg"></a></div></div></form></nav><section id="main-content" class="content"><div class="main-heading"><h1 class="fqn">Module <a href="../index.html">tantivy</a>::<wbr><a class="mod" href="#">positions</a><button id="copy-path" onclick="copy_path(this)" title="Copy item path to clipboard"><img src="../../clipboard.svg" width="19" height="18" alt="Copy item path"></button></h1><span class="out-of-band"><a class="srclink" href="../../src/tantivy/positions/mod.rs.html#1-237">source</a> · <a id="toggle-all-docs" href="javascript:void(0)" title="collapse all docs">[<span class="inner">−</span>]</a></span></div><details class="rustdoc-toggle top-doc" open><summary class="hideme"><span>Expand description</span></summary><div class="docblock"><p>Tantivy can (if instructed to do so in the schema) store the term positions in a given field. |
| This position is expressed as token ordinal. For instance, |
| In “The beauty and the beast”, the term “the” appears in position 0 and position 3. |
| This information is useful to run phrase queries.</p> |
| <p>The <a href="../enum.SegmentComponent.html#variant.Positions">position</a> file contains all of the |
| bitpacked positions delta, for all terms of a given field, one term after the other.</p> |
| <p>Each term is encoded independently. |
| Like for posting lists, tantivy relies on simd bitpacking to encode the positions delta in |
| blocks of 128 deltas. Because we rarely have a multiple of 128, the final block encodes |
| the remaining values with variable int encoding.</p> |
| <p>In order to make reading possible, the term delta positions first encode the number of |
| bitpacked blocks, then the bitwidth for each block, then the actual bitpacked blocks and finally |
| the final variable int encoded block.</p> |
| <p>Contrary to postings list, the reader does not have access on the number of positions that is |
| encoded, and instead stops decoding the last block when its byte slice has been entirely read.</p> |
| <p>More formally:</p> |
| <ul> |
| <li><em>Positions</em> := <em>NumBitPackedBlocks</em> <em>BitPackedPositionBlock</em>^(P/128) |
| <em>BitPackedPositionsDeltaBitWidth</em> <em>VIntPosDeltas</em>?</li> |
| <li><em>NumBitPackedBlocks</em>*: := <em>P</em> / 128 encoded as a variable byte integer.</li> |
| <li><em>BitPackedPositionBlock</em> := bit width encoded block of 128 positions delta</li> |
| <li><em>BitPackedPositionsDeltaBitWidth</em> := (<em>BitWidth</em>: u8)^<em>NumBitPackedBlocks</em></li> |
| <li><em>VIntPosDeltas</em> := <em>VIntPosDelta</em>^(<em>P</em> % 128).</li> |
| </ul> |
| <p>The skip widths encoded separately makes it easy and fast to rapidly skip over n positions.</p> |
| </div></details><h2 id="structs" class="small-section-header"><a href="#structs">Structs</a></h2><div class="item-table"><div class="item-row"><div class="item-left module-item"><a class="struct" href="struct.PositionReader.html" title="tantivy::positions::PositionReader struct">PositionReader</a></div><div class="item-right docblock-short">When accessing the positions of a term, we get a positions_idx from the <code>Terminfo</code>. |
| This means we need to skip to the <code>nth</code> position efficiently.</div></div><div class="item-row"><div class="item-left module-item"><a class="struct" href="struct.PositionSerializer.html" title="tantivy::positions::PositionSerializer struct">PositionSerializer</a></div><div class="item-right docblock-short">The PositionSerializer is in charge of serializing all of the positions |
| of all of the terms of a given field.</div></div></div></section></div></main><div id="rustdoc-vars" data-root-path="../../" data-current-crate="tantivy" data-themes="ayu,dark,light" data-resource-suffix="" data-rustdoc-version="1.66.0-nightly (5c8bff74b 2022-10-21)" ></div></body></html> |
