| <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="generator" content="rustdoc"><meta name="description" content="Middleware that allows balancing load among multiple services."><meta name="keywords" content="rust, rustlang, rust-lang, balance"><title>tower::balance - Rust</title><link rel="preload" as="font" type="font/woff2" crossorigin href="../../SourceSerif4-Regular.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../FiraSans-Regular.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../FiraSans-Medium.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../SourceCodePro-Regular.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../SourceSerif4-Bold.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../SourceCodePro-Semibold.ttf.woff2"><link rel="stylesheet" href="../../normalize.css"><link rel="stylesheet" href="../../rustdoc.css" id="mainThemeStyle"><link rel="stylesheet" href="../../ayu.css" disabled><link rel="stylesheet" href="../../dark.css" disabled><link rel="stylesheet" href="../../light.css" id="themeStyle"><script id="default-settings" ></script><script src="../../storage.js"></script><script defer src="../../main.js"></script><noscript><link rel="stylesheet" href="../../noscript.css"></noscript><link rel="alternate icon" type="image/png" href="../../favicon-16x16.png"><link rel="alternate icon" type="image/png" href="../../favicon-32x32.png"><link rel="icon" type="image/svg+xml" href="../../favicon.svg"></head><body class="rustdoc mod"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><nav class="mobile-topbar"><button class="sidebar-menu-toggle">☰</button><a class="sidebar-logo" href="../../tower/index.html"><div class="logo-container"><img class="rust-logo" src="../../rust-logo.svg" alt="logo"></div></a><h2></h2></nav><nav class="sidebar"><a class="sidebar-logo" href="../../tower/index.html"><div class="logo-container"><img class="rust-logo" src="../../rust-logo.svg" alt="logo"></div></a><h2 class="location"><a href="#">Module balance</a></h2><div class="sidebar-elems"><section><ul class="block"><li><a href="#modules">Modules</a></li></ul></section></div></nav><main><div class="width-limiter"><nav class="sub"><form class="search-form"><div class="search-container"><span></span><input class="search-input" name="search" autocomplete="off" spellcheck="false" placeholder="Click or press ‘S’ to search, ‘?’ for more options…" type="search"><div id="help-button" title="help" tabindex="-1"><a href="../../help.html">?</a></div><div id="settings-menu" tabindex="-1"><a href="../../settings.html" title="settings"><img width="22" height="22" alt="Change settings" src="../../wheel.svg"></a></div></div></form></nav><section id="main-content" class="content"><div class="main-heading"><h1 class="fqn">Module <a href="../index.html">tower</a>::<wbr><a class="mod" href="#">balance</a><button id="copy-path" onclick="copy_path(this)" title="Copy item path to clipboard"><img src="../../clipboard.svg" width="19" height="18" alt="Copy item path"></button></h1><span class="out-of-band"><a class="srclink" href="../../src/tower/balance/mod.rs.html#1-61">source</a> · <a id="toggle-all-docs" href="javascript:void(0)" title="collapse all docs">[<span class="inner">−</span>]</a></span></div><details class="rustdoc-toggle top-doc" open><summary class="hideme"><span>Expand description</span></summary><div class="docblock"><p>Middleware that allows balancing load among multiple services.</p> |
| <p>In larger systems, multiple endpoints are often available for a given service. As load |
| increases, you want to ensure that that load is spread evenly across the available services. |
| Otherwise, clients could see spikes in latency if their request goes to a particularly loaded |
| service, even when spare capacity is available to handle that request elsewhere.</p> |
| <p>This module provides two pieces of middleware that helps with this type of load balancing:</p> |
| <p>First, <a href="p2c/index.html" title="p2c"><code>p2c</code></a> implements the “<a href="http://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf">Power of Two Random Choices</a>” algorithm, a simple but robust |
| technique for spreading load across services with only inexact load measurements. Use this if |
| the set of available services is not within your control, and you simply want to spread load |
| among that set of services.</p> |
| <p>Second, <a href="pool/index.html" title="pool"><code>pool</code></a> implements a dynamically sized pool of services. It estimates the overall |
| current load by tracking successful and unsuccessful calls to <a href="../trait.Service.html#tymethod.poll_ready"><code>poll_ready</code></a>, and uses an |
| exponentially weighted moving average to add (using <a href="../make/trait.MakeService.html"><code>MakeService</code></a>) or remove (by dropping) |
| services in response to increases or decreases in load. Use this if you are able to |
| dynamically add more service endpoints to the system to handle added load.</p> |
| <h2 id="examples"><a href="#examples">Examples</a></h2> |
| <div class="example-wrap"><pre class="rust rust-example-rendered"><code><span class="kw">use </span>tower::balance::p2c::Balance; |
| <span class="kw">use </span>tower::load::Load; |
| <span class="kw">use </span>tower::{Service, ServiceExt}; |
| <span class="kw">use </span>futures_util::pin_mut; |
| |
| <span class="kw">async fn </span>spread<Req, S: Service<Req> + Load>(svc1: S, svc2: S, reqs: <span class="kw">impl </span>Stream<Item = Req>) |
| <span class="kw">where |
| </span>S::Error: Into<tower::BoxError>, |
| S::Metric: std::fmt::Debug, |
| { |
| <span class="comment">// Spread load evenly across the two services |
| </span><span class="kw">let </span>p2c = Balance::new(tower::discover::ServiceList::new(<span class="macro">vec!</span>[svc1, svc2])); |
| |
| <span class="comment">// Issue all the requests that come in. |
| // Some will go to svc1, some will go to svc2. |
| </span><span class="macro">pin_mut!</span>(reqs); |
| <span class="kw">let </span><span class="kw-2">mut </span>responses = p2c.call_all(reqs); |
| <span class="kw">while let </span><span class="prelude-val">Some</span>(rsp) = responses.next().<span class="kw">await </span>{ |
| <span class="comment">// ... |
| </span>} |
| }</code></pre></div> |
| </div></details><h2 id="modules" class="small-section-header"><a href="#modules">Modules</a></h2><div class="item-table"><div class="item-row"><div class="item-left module-item"><a class="mod" href="error/index.html" title="tower::balance::error mod">error</a></div><div class="item-right docblock-short">Error types for the <a href="index.html"><code>tower::balance</code></a> middleware.</div></div><div class="item-row"><div class="item-left module-item"><a class="mod" href="p2c/index.html" title="tower::balance::p2c mod">p2c</a></div><div class="item-right docblock-short">This module implements the “<a href="http://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf">Power of Two Random Choices</a>” load balancing algorithm.</div></div><div class="item-row"><div class="item-left module-item"><a class="mod" href="pool/index.html" title="tower::balance::pool mod">pool</a></div><div class="item-right docblock-short">This module defines a load-balanced pool of services that adds new services when load is high.</div></div></div></section></div></main><div id="rustdoc-vars" data-root-path="../../" data-current-crate="tower" data-themes="ayu,dark,light" data-resource-suffix="" data-rustdoc-version="1.66.0-nightly (5c8bff74b 2022-10-21)" ></div></body></html> |
