| <!DOCTYPE html>
|
| <!--[if IE]><![endif]-->
|
| <html>
|
|
|
| <head> |
| <link rel="canonical" href="https://ignite.apache.org/releases/2.4.0/dotnetdoc/api/Apache.Ignite.Core.Datastream.html" /> |
| <META NAME="ROBOTS" CONTENT="NOINDEX"> |
|
|
| <meta charset="utf-8">
|
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
|
| <title>Namespace Apache.Ignite.Core.Datastream
|
| | Apache Ignite.NET </title>
|
| <meta name="viewport" content="width=device-width">
|
| <meta name="title" content="Namespace Apache.Ignite.Core.Datastream
|
| | Apache Ignite.NET ">
|
| <meta name="generator" content="docfx 2.24.0.0">
|
|
|
| <link rel="shortcut icon" href="../images/favicon.ico">
|
| <link rel="stylesheet" href="../styles/docfx.vendor.css">
|
| <link rel="stylesheet" href="../styles/docfx.css">
|
| <link rel="stylesheet" href="../styles/main.css">
|
| <meta property="docfx:navrel" content="../toc.html">
|
| <meta property="docfx:tocrel" content="toc.html">
|
|
|
| <meta property="docfx:rel" content="../">
|
|
|
| |
| <script> |
| (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ |
| (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), |
| m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) |
| })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); |
| |
| ga('create', 'UA-61232409-1', 'auto'); |
| ga('send', 'pageview'); |
| |
| </script></head>
|
| <body data-spy="scroll" data-target="#affix">
|
| <div id="wrapper">
|
| <header>
|
|
|
| <nav id="autocollapse" class="navbar navbar-inverse ng-scope" role="navigation">
|
| <div class="container">
|
| <div class="navbar-header">
|
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar">
|
| <span class="sr-only">Toggle navigation</span>
|
| <span class="icon-bar"></span>
|
| <span class="icon-bar"></span>
|
| <span class="icon-bar"></span>
|
| </button>
|
|
|
| <a class="navbar-brand" href="../index.html">
|
| <img id="logo" class="svg" src="../images/logo_ignite_32_32.png" alt="">
|
| </a>
|
| </div>
|
| <div class="collapse navbar-collapse" id="navbar">
|
| <form class="navbar-form navbar-right" role="search" id="search">
|
| <div class="form-group">
|
| <input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off">
|
| </div>
|
| </form>
|
| </div>
|
| </div>
|
| </nav>
|
|
|
| <div class="subnav navbar navbar-default">
|
| <div class="container hide-when-search" id="breadcrumb">
|
| <ul class="breadcrumb">
|
| <li></li>
|
| </ul>
|
| </div>
|
| </div>
|
| </header>
|
| <div class="container body-content">
|
|
|
| <div id="search-results">
|
| <div class="search-list"></div>
|
| <div class="sr-items"></div>
|
| <ul id="pagination"></ul>
|
| </div>
|
| </div>
|
| <div role="main" class="container body-content hide-when-search">
|
|
|
| <div class="sidenav hide-when-search">
|
| <a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a>
|
| <div class="sidetoggle collapse" id="sidetoggle">
|
| <div id="sidetoc"></div>
|
| </div>
|
| </div>
|
| <div class="article row grid-right">
|
| <div class="col-md-10">
|
| <article class="content wrap" id="_content" data-uid="Apache.Ignite.Core.Datastream">
|
|
|
| <h1 id="Apache_Ignite_Core_Datastream" data-uid="Apache.Ignite.Core.Datastream" class="text-break">Namespace Apache.Ignite.Core.Datastream
|
| </h1>
|
| <div class="markdown level0 summary"></div>
|
| <div class="markdown level0 conceptual"></div>
|
| <div class="markdown level0 remarks"></div>
|
| <h3 id="classes">Classes
|
| </h3>
|
| <h4><a class="xref" href="Apache.Ignite.Core.Datastream.DataStreamerDefaults.html">DataStreamerDefaults</a></h4>
|
| <section><p>Data streamer configuration defaults.</p> |
| </section>
|
| <h4><a class="xref" href="Apache.Ignite.Core.Datastream.StreamTransformer-4.html">StreamTransformer<TK, TV, TArg, TRes></a></h4>
|
| <section><p>Convenience adapter to transform update existing values in streaming cache |
| based on the previously cached value.</p> |
| </section>
|
| <h4><a class="xref" href="Apache.Ignite.Core.Datastream.StreamVisitor-2.html">StreamVisitor<TK, TV></a></h4>
|
| <section><p>Convenience adapter to visit every key-value tuple in the stream. |
| Note that the visitor does not update the cache.</p> |
| </section>
|
| <h3 id="interfaces">Interfaces
|
| </h3>
|
| <h4><a class="xref" href="Apache.Ignite.Core.Datastream.IDataStreamer-2.html">IDataStreamer<TK, TV></a></h4>
|
| <section><p>Data streamer is responsible for loading external data into cache. It achieves it by |
| properly buffering updates and properly mapping keys to nodes responsible for the data |
| to make sure that there is the least amount of data movement possible and optimal |
| network and memory utilization. |
| <p> |
| Note that streamer will load data concurrently by multiple internal threads, so the |
| data may get to remote nodes in different order from which it was added to |
| the streamer. |
| <p> |
| Also note that <code>IDataStreamer</code> is not the only way to load data into cache. |
| Alternatively you can use |
| <a class="xref" href="Apache.Ignite.Core.Cache.Store.ICacheStore-2.html#Apache_Ignite_Core_Cache_Store_ICacheStore_2_LoadCache_System_Action__0__1__System_Object___">LoadCache(Action<TK, TV>, Object[])</a> |
| method to load data from underlying data store. You can also use standard cache |
| <code>put</code> and <code>putAll</code> operations as well, but they most likely will not perform |
| as well as this class for loading data. And finally, data can be loaded from underlying |
| data store on demand, whenever it is accessed - for this no explicit data loading step |
| is needed. |
| <p> |
| <code>IDataStreamer</code> supports the following configuration properties: |
| <ul><li>PerNodeBufferSizeWhen entries are added to data streamer they are not sent to Ignite |
| right away and are buffered internally for better performance and network utilization. |
| This setting controls the size of internal per-node buffer before buffered data is sent to |
| remote node. Default value is 1024.</li><li>PerNodeParallelOperationsSometimes data may be added to the data streamer faster than it can be put |
| in cache. In this case, new buffered load messages are sent to remote nodes before |
| responses from previous ones are received. This could cause unlimited heap memory |
| utilization growth on local and remote nodes. To control memory utilization, this |
| setting limits maximum allowed number of parallel buffered load messages that are |
| being processed on remote nodes. If this number is exceeded, then data streamer add/remove |
| methods will block to control memory utilization. Default value is 16.</li><li>AutoFlushFrequencyAutomatic flush frequency in milliseconds. Essentially, this is the time |
| after which the streamer will make an attempt to submit all data added so far to remote |
| nodes. Note that there is no guarantee that data will be delivered after this concrete |
| attempt (e.g., it can fail when topology is changing), but it won't be lost anyway. |
| Disabled by default (default value is <code>0</code>).</li><li>IsolatedDefines if data streamer will assume that there are no other concurrent |
| updates and allow data streamer choose most optimal concurrent implementation. Default value |
| is <code>false</code>.</li></ul> |
| <p> |
| All members are thread-safe and may be used concurrently from multiple threads.</p> |
| </section>
|
| <h4><a class="xref" href="Apache.Ignite.Core.Datastream.IStreamReceiver-2.html">IStreamReceiver<TK, TV></a></h4>
|
| <section><p>Updates cache with batch of entries. |
| Usually it is enough to configure <a class="xref" href="Apache.Ignite.Core.Datastream.IDataStreamer-2.html#Apache_Ignite_Core_Datastream_IDataStreamer_2_AllowOverwrite">AllowOverwrite</a> property and appropriate |
| internal cache receiver will be chosen automatically. But in some cases custom implementation may help |
| to achieve better performance.</p> |
| </section>
|
| </article>
|
| </div>
|
|
|
| <div class="hidden-sm col-md-2" role="complementary">
|
| <div class="sideaffix">
|
| <nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix">
|
| <!-- <p><a class="back-to-top" href="#top">Back to top</a><p> -->
|
| </nav>
|
| </div>
|
| </div>
|
| </div>
|
| </div>
|
|
|
| <footer>
|
| <div class="grad-bottom"></div>
|
| <div class="footer">
|
| <div class="container">
|
| <span class="pull-right">
|
| <a href="#top">Back to top</a>
|
| </span>
|
| © 2015 - 2018 The Apache Software Foundation
|
|
|
| </div>
|
| </div>
|
| </footer>
|
| </div>
|
|
|
| <script type="text/javascript" src="../styles/docfx.vendor.js"></script>
|
| <script type="text/javascript" src="../styles/docfx.js"></script>
|
| <script type="text/javascript" src="../styles/main.js"></script>
|
| </body>
|
| </html>
|
