IGNITE-14609 Document old and new async continuation behavior
Java:
* Update "Asynchronous Execution" section on "Basic Cache Operations" page
* Fix IgniteFuture javadoc (listen methods)
.NET:
* Add dedicated async page to ".NET Specific" section
* Document Cache and Compute async continuation behavior in different versions
diff --git a/docs/_data/toc.yaml b/docs/_data/toc.yaml
index 929eb8a..7ede037 100644
--- a/docs/_data/toc.yaml
+++ b/docs/_data/toc.yaml
@@ -380,6 +380,8 @@
url: net-specific/net-deployment-options
- title: Standalone Nodes
url: net-specific/net-standalone-nodes
+ - title: Asynchronous APIs
+ url: net-specific/net-async
- title: Logging
url: net-specific/net-logging
- title: LINQ
diff --git a/docs/_docs/key-value-api/basic-cache-operations.adoc b/docs/_docs/key-value-api/basic-cache-operations.adoc
index 5c6d66a..7205cd0 100644
--- a/docs/_docs/key-value-api/basic-cache-operations.adoc
+++ b/docs/_docs/key-value-api/basic-cache-operations.adoc
@@ -218,9 +218,13 @@
*ALSO, do we need to explain what a "closure" is?*
Blocking and closure are basic notions a java developer should know. We also expect that users know/can learn themselves how to use the Feature class. We can elaborate on this if we get relevant feedback.
+
+NOTE: Closure is not a technically correct term here. Closure is something that captures (encloses) context: https://en.wikipedia.org/wiki/Closure_(computer_programming).
+We don't know if the user code which is passed as a listener is a closure or not.
+Callback seems to be a better fit.
////
-To wait for the results in a non-blocking fashion, register a closure using the `IgniteFuture.listen()` or `IgniteFuture.chain()` method. The closure is called when the operation is completed.
+To wait for the results in a non-blocking fashion, register a callback using the `IgniteFuture.listen()` or `IgniteFuture.chain()` method. The callback is called when the operation is completed.
[tabs]
--
@@ -245,18 +249,29 @@
[NOTE]
====
[discrete]
-=== Closures Execution and Thread Pools
+=== Callbacks Execution and Thread Pools
////////////////////////////////////////////////////////////////////////////////
This is java specific
////////////////////////////////////////////////////////////////////////////////
-If an asynchronous operation is completed by the time the closure is passed to either the `IgniteFuture.listen()` or `IgniteFuture.chain()` method, then the closure is executed synchronously by the calling thread. Otherwise, the closure is executed asynchronously when the operation is completed.
+If an asynchronous operation is completed by the time the callback is passed to either the `IgniteFuture.listen()` or `IgniteFuture.chain()` method, then the callback is executed synchronously by the calling thread.
+Otherwise, the callback is executed asynchronously when the operation is completed.
-Depending on the type of operation, the closure might be called by a thread from the system pool (asynchronous cache operations) or by a thread from the public pool (asynchronous compute operations). Therefore, you should avoid calling synchronous cache and compute operations from inside the closure, because it may lead to a deadlock due to pools starvation.
+Callbacks for asynchronous compute operations are invoked by threads from the link:perf-and-troubleshooting/thread-pools-tuning[Ignite public pool].
+Calling synchronous cache and compute operations from inside the callback may lead to a deadlock due to pools starvation.
+To achieve nested execution of asynchronous compute operations, you can take advantage of link:perf-and-troubleshooting/thread-pools-tuning#creating-custom-thread-pool[custom thread pools].
-To achieve nested execution of asynchronous compute operations, you can take advantage of link:perf-troubleshooting-guide/thread-pools-tuning#creating-custom-thread-pool[custom thread pools].
+Callbacks for asynchronous cache operations are invoked by using `ForkJoinPool#commonPool`, unless a different executor is configured with `IgniteConfiguration#asyncContinuationExecutor`.
+
+* This default executor is safe for any operations inside the callback.
+* Default behavior was changed in Ignite 2.11. Before that, async cache operation callbacks were called from an Ignite system pool (so-called "striped pool").
+* To restore previous behavior, use `IgniteConfiguration.setAsyncContinuationExecutor(Runnable::run)`.
+** Previous behavior can provide a small performance improvement, because callbacks are executed without any indirection or scheduling.
+** UNSAFE: cache operations cannot proceed while system threads execute callbacks, and deadlocks are possible if other cache operations are invoked from the callback.
+
+
====
@@ -268,7 +283,7 @@
== Resource Injection
-Ignite allows dependency injection of pre-defined Ignite resources, and supports field-based as well as method-based injection. Resources with proper annotations will be injected into the corresponding task, job, closure, or SPI before it is initialized.
+Ignite allows dependency injection of pre-defined Ignite resources, and supports field-based as well as method-based injection. Resources with proper annotations will be injected into the corresponding task, job, callback, or SPI before it is initialized.
You can inject resources by annotating either a field or a method. When you annotate a field, Ignite simply sets the value of the field at injection time (disregarding an access modifier of the field). If you annotate a method with a resource annotation, it should accept an input parameter of the type corresponding to the injected resource. If it does, then the method is invoked at injection time with the appropriate resource passed as an input argument.
diff --git a/docs/_docs/net-specific/net-async.adoc b/docs/_docs/net-specific/net-async.adoc
new file mode 100644
index 0000000..87ee02c
--- /dev/null
+++ b/docs/_docs/net-specific/net-async.adoc
@@ -0,0 +1,121 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+= Asynchronous APIs
+
+== Overview
+
+Many Ignite APIs have asynchronous variants, for example, `void ICache.Put` and `Task ICache.PutAsync`.
+Async APIs allow us to write link:https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/async/[efficient non-blocking code]:
+
+[source,csharp]
+----
+ICache<int, string> cache = ignite.GetOrCreateCache<int, string>("name");
+
+// Sync, blocks thread on every call.
+cache.Put(1, "Hello");
+string hello = cache.Get(1);
+
+// Async, does not block threads.
+await cache.PutAsync(1, "Hello");
+string hello = await cache.GetAsync(1);
+----
+
+With async APIs, current thread is not blocked while we wait for the cache operation to complete;
+the thread is returned to the thread pool and can perform other work.
+
+When the async operation completes, our method resumes execution - either on the same thread, or on a different one -
+depending on the environment and the configuration. This is called "async continuation".
+
+
+== Async Continuations
+
+Unless specified otherwise, Ignite executes async continuations on the link:https://docs.microsoft.com/en-us/dotnet/standard/threading/the-managed-thread-pool[.NET Thread Pool], which is safe and does not require any special care.
+
+
+=== Thin Client
+
+All thin client async APIs use link:https://docs.microsoft.com/en-us/dotnet/standard/threading/the-managed-thread-pool[.NET Thread Pool.] for async continuations.
+
+=== Thick Cache
+
+Callbacks for asynchronous cache operations on server and thick client nodes are invoked by using Java `ForkJoinPool#commonPool`, unless a different executor is configured with `IgniteConfiguration.AsyncContinuationExecutor`.
+
+* This default executor is safe for any operations inside the callback.
+* Default behavior was changed in Ignite 2.11. Before that, async cache operation callbacks were called from an Ignite system pool (so-called "striped pool").
+* To restore the previous behavior, use `IgniteConfiguration.AsyncContinuationExecutor = AsyncContinuationExecutor.UnsafeSynchronous`.
+** Previous behavior can provide a small performance improvement, because callbacks are executed without any indirection or scheduling.
+** UNSAFE: cache operations cannot proceed while system threads execute callbacks, and deadlocks are possible if other cache operations are invoked from the callback.
+
+[IMPORTANT]
+====
+[discrete]
+=== *Ignite 2.10 and before*: possibility of deadlocks and system pool starvation
+
+In Ignite versions 2.10 and before, system pool is used to run async continuations,
+which means that `GetAsync` call in the code above is executed by the system thread.
+
+This can lead to deadlocks if user code blocks the thread, or cause starvation because system thread is busy
+running user code instead of performing cache operations.
+
+To enable safe behavior, move continuations to .NET Thread Pool manually:
+
+[source,csharp]
+----
+await cache.PutAsync(1, "Hello").ContinueWith(
+ t => {},
+ CancellationToken.None,
+ TaskContinuationOptions.None,
+ TaskScheduler.Default);
+
+string hello = await cache.GetAsync(1).ContinueWith(
+ t => t.Result,
+ CancellationToken.None,
+ TaskContinuationOptions.None,
+ TaskScheduler.Default);
+----
+
+Tip: use an extension method to reduce verbosity.
+
+====
+
+
+=== Compute
+
+*Ignite 2.11 and later*: all `ICompute` async APIs use .NET Thread Pool to run async continuations.
+
+*Ignite 2.10 and before*: Compute async continuations are executed on link:perf-and-troubleshooting/thread-pools-tuning[Ignite public pool].
+To reduce the load on the public pool, it is recommended to use the same `ContinueWith` approach as above:
+
+[source,csharp]
+----
+await compute.CallAsync(new MyAction()).ContinueWith(
+ t => t.Result,
+ CancellationToken.None,
+ TaskContinuationOptions.None,
+ TaskScheduler.Default);
+----
+
+This will move the continuation from Ignite public pool (reserved for Compute functionality) to the .NET thread pool (`TaskScheduler.Default`).
+
+
+== ConfigureAwait
+
+`Task.ConfigureAwait` method can be used as usual with all Ignite async APIs.
+
+See link:https://devblogs.microsoft.com/dotnet/configureawait-faq/[ConfigureAwait FAQ] for more details.
+
+
+
+
diff --git a/modules/core/src/main/java/org/apache/ignite/lang/IgniteFuture.java b/modules/core/src/main/java/org/apache/ignite/lang/IgniteFuture.java
index 93c6b33..d97aa85 100644
--- a/modules/core/src/main/java/org/apache/ignite/lang/IgniteFuture.java
+++ b/modules/core/src/main/java/org/apache/ignite/lang/IgniteFuture.java
@@ -96,20 +96,18 @@
public boolean isDone();
/**
- * Registers listener closure to be asynchronously notified whenever future completes.
- * Closure will be processed in thread that completes this future or (if future already
- * completed) immediately in current thread.
+ * Registers a callback to be invoked when the future completes.
+ * If the future is already completed, callback will be invoked immediately in the current thread.
*
* @param lsnr Listener closure to register. Cannot be {@code null}.
*/
public void listen(IgniteInClosure<? super IgniteFuture<V>> lsnr);
/**
- * Registers listener closure to be asynchronously notified whenever future completes.
- * Closure will be processed in specified executor.
+ * Registers a callback to be invoked with the specified executor when the future completes.
*
* @param lsnr Listener closure to register. Cannot be {@code null}.
- * @param exec Executor to run listener. Cannot be {@code null}.
+ * @param exec Executor to invoke the listener. Cannot be {@code null}.
*/
public void listenAsync(IgniteInClosure<? super IgniteFuture<V>> lsnr, Executor exec);
