| <!DOCTYPE html PUBLIC "" |
| ""> |
| <html><head><meta charset="UTF-8" /><title>org.apache.clojure-mxnet.kvstore documentation</title><link rel="stylesheet" type="text/css" href="css/default.css" /><link rel="stylesheet" type="text/css" href="css/highlight.css" /><script type="text/javascript" src="js/highlight.min.js"></script><script type="text/javascript" src="js/jquery.min.js"></script><script type="text/javascript" src="js/page_effects.js"></script><script>hljs.initHighlightingOnLoad();</script></head><body><div id="header"><h2>Generated by <a href="https://github.com/weavejester/codox">Codox</a></h2><h1><a href="index.html"><span class="project-title"><span class="project-name">Clojure-mxnet</span> <span class="project-version">1.9.1-SNAPSHOT</span></span></a></h1></div><div class="sidebar primary"><h3 class="no-link"><span class="inner">Project</span></h3><ul class="index-link"><li class="depth-1 "><a href="index.html"><div class="inner">Index</div></a></li></ul><h3 class="no-link"><span class="inner">Namespaces</span></h3><ul><li class="depth-1"><div class="no-link"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>org</span></div></div></li><li class="depth-2"><div class="no-link"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>apache</span></div></div></li><li class="depth-3"><div class="no-link"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>clojure-mxnet</span></div></div></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.base.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>base</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.callback.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>callback</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.context.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>context</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.dtype.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>dtype</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.eval-metric.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>eval-metric</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.executor.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>executor</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.image.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>image</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.infer.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>infer</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.initializer.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>initializer</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.io.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>io</span></div></a></li><li class="depth-4 branch current"><a href="org.apache.clojure-mxnet.kvstore.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>kvstore</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.kvstore-server.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>kvstore-server</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.layout.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>layout</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.lr-scheduler.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>lr-scheduler</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.module.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>module</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.monitor.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>monitor</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.ndarray.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>ndarray</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.ndarray-api.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>ndarray-api</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.ndarray-random-api.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>ndarray-random-api</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.optimizer.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>optimizer</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.primitives.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>primitives</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.profiler.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>profiler</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.random.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>random</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.resource-scope.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>resource-scope</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.shape.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>shape</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.symbol.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>symbol</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.symbol-api.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>symbol-api</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.symbol-random-api.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>symbol-random-api</span></div></a></li><li class="depth-4 branch"><a href="org.apache.clojure-mxnet.util.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>util</span></div></a></li><li class="depth-4"><a href="org.apache.clojure-mxnet.visualization.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>visualization</span></div></a></li></ul></div><div class="sidebar secondary"><h3><a href="#top"><span class="inner">Public Vars</span></a></h3><ul><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-barrier"><div class="inner"><span>barrier</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-create"><div class="inner"><span>create</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-dispose"><div class="inner"><span>dispose</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-init"><div class="inner"><span>init</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-load-optimizer-states"><div class="inner"><span>load-optimizer-states</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-num-dead-node"><div class="inner"><span>num-dead-node</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-num-workers"><div class="inner"><span>num-workers</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-pull"><div class="inner"><span>pull</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-push"><div class="inner"><span>push</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-rank"><div class="inner"><span>rank</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-save-optimizer-states"><div class="inner"><span>save-optimizer-states</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-send-command-to-servers"><div class="inner"><span>send-command-to-servers</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-set-barrier-before-exit"><div class="inner"><span>set-barrier-before-exit</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-set-optimizer"><div class="inner"><span>set-optimizer</span></div></a></li><li class="depth-1"><a href="org.apache.clojure-mxnet.kvstore.html#var-type"><div class="inner"><span>type</span></div></a></li></ul></div><div class="namespace-docs" id="content"><h1 class="anchor" id="top">org.apache.clojure-mxnet.kvstore</h1><div class="doc"><pre class="plaintext"></pre></div><div class="public anchor" id="var-barrier"><h3>barrier</h3><div class="usage"><code>(barrier kvstore)</code></div><div class="doc"><pre class="plaintext">Global barrier among all worker nodes |
| For example, assume there are n machines, we want to let machine 0 first |
| init the values, and then pull the inited value to all machines. Before |
| pulling, we can place a barrier to guarantee that the initialization is |
| finished.</pre></div></div><div class="public anchor" id="var-create"><h3>create</h3><div class="usage"><code>(create name)</code><code>(create)</code></div><div class="doc"><pre class="plaintext"> Create a new KVStore |
| WARNING: it is your responsibility to clear this object through dispose. |
| - name : #{local, dist} (default is local) |
| The type of KVStore |
| - local works for multiple devices on a single machine (single process) |
| - dist works for multi-machines (multiple processes)</pre></div></div><div class="public anchor" id="var-dispose"><h3>dispose</h3><div class="usage"><code>(dispose kvstore)</code></div><div class="doc"><pre class="plaintext">Release the native memory. |
| The object shall never be used after it is disposed.</pre></div></div><div class="public anchor" id="var-init"><h3>init</h3><div class="usage"><code>(init kvstore ks vs)</code></div><div class="doc"><pre class="plaintext">Initialize a single or a sequence of key-value pairs into the store. |
| For each key, one must init it before push and pull. |
| Only worker 0's (rank == 0) data are used. |
| This function returns after data have been initialized successfully |
| kvstore - KVstore |
| ks - keys (vec or strings or single string) |
| vs - values (vec or NDArrays or single ndarry)</pre></div></div><div class="public anchor" id="var-load-optimizer-states"><h3>load-optimizer-states</h3><div class="usage"><code>(load-optimizer-states kvstore fname)</code></div><div class="doc"><pre class="plaintext">Load optimizer (updater) state from file |
| - kvstore |
| -fname Path to input states file.</pre></div></div><div class="public anchor" id="var-num-dead-node"><h3>num-dead-node</h3><div class="usage"><code>(num-dead-node kvstore node-id)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-num-workers"><h3>num-workers</h3><div class="usage"><code>(num-workers kvstore)</code></div><div class="doc"><pre class="plaintext">Get the number of worker nodes |
| </pre></div></div><div class="public anchor" id="var-pull"><h3>pull</h3><div class="usage"><code>(pull kvstore ks outs priority)</code><code>(pull kvstore ks outs)</code></div><div class="doc"><pre class="plaintext"> Pull a single value or a sequence of values from the store. |
| Data consistency: |
| 1. this function returns after adding an operator to the engine. But any |
| further read on out will be blocked until it is finished. |
| 2. pull is always called after all previous push and pull on the same key are finished |
| 3. It pulls the newest value from the store. |
| - kvstore |
| - ks single or vector of (strings) |
| - outs single or vector of outs (NDArrays) |
| - priority |
| The priority of the push operation. |
| The higher the priority, the faster this action is likely |
| to be executed before other push actions.</pre></div></div><div class="public anchor" id="var-push"><h3>push</h3><div class="usage"><code>(push kvstore ks vs priority)</code><code>(push kvstore ks vs)</code></div><div class="doc"><pre class="plaintext"> Push a single or a sequence of key-value pairs into the store. |
| Data consistency: |
| 1. this function returns after adding an operator to the engine. |
| 2. push is always called after all previous push and pull on the same key are finished |
| 3. there is no synchronization between workers. One can use _barrier() to sync all workers |
| |
| -ks Keys |
| -vs values According values |
| - priority |
| The priority of the push operation. |
| The higher the priority, the faster this action is likely |
| to be executed before other push actions.</pre></div></div><div class="public anchor" id="var-rank"><h3>rank</h3><div class="usage"><code>(rank kvstore)</code></div><div class="doc"><pre class="plaintext">Get the rank of this worker node |
| returns The rank of this node, which is in [0, get_num_workers()) </pre></div></div><div class="public anchor" id="var-save-optimizer-states"><h3>save-optimizer-states</h3><div class="usage"><code>(save-optimizer-states kvstore fname)</code></div><div class="doc"><pre class="plaintext">Save optimizer (updater) state to file |
| - kvstore |
| - fname Path to output states file.</pre></div></div><div class="public anchor" id="var-send-command-to-servers"><h3>send-command-to-servers</h3><div class="usage"><code>(send-command-to-servers kvstore head body)</code></div><div class="doc"><pre class="plaintext">Send a command to all server nodes |
| Send a command to all server nodes, which will make each server node run |
| KVStoreServer.controller |
| This function returns after the command has been executed in all server nodes |
| -kvstore |
| -head the head of the command |
| - body the body of the command</pre></div></div><div class="public anchor" id="var-set-barrier-before-exit"><h3>set-barrier-before-exit</h3><div class="usage"><code>(set-barrier-before-exit kvstore barrier-before-exit)</code></div><div class="doc"><pre class="plaintext"> Whether to do barrier when the kvstore finalizes |
| - kvstore |
| - barrier-before-exit boolean</pre></div></div><div class="public anchor" id="var-set-optimizer"><h3>set-optimizer</h3><div class="usage"><code>(set-optimizer kvstore optimizer)</code></div><div class="doc"><pre class="plaintext">Register an optimizer to the store |
| If there are multiple machines, this process (should be a worker node) |
| will pack this optimizer and send it to all servers. It returns after |
| this action is done</pre></div></div><div class="public anchor" id="var-type"><h3>type</h3><div class="usage"><code>(type kvstore)</code></div><div class="doc"><pre class="plaintext">Get the type of the kvstore |
| </pre></div></div></div></body></html> |
