blob: 9e68d832519ce64522679abc0b105c148c029745 [file] [log] [blame]
// Licensed to the Apache Software Foundation (ASF) under one or more
// contributor license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// The ASF licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
= Ignite Clients
Apache Ignite 3 clients connect to the cluster via a standard socket connection. Unlike Ignite 2.x, there is no separate Thin and Thick clients in Apache Ignite 3. All clients are 'thin'.
Clients do not become a part of the cluster topology, never hold any data, and are not used as a destination for compute calculations.
== Getting Started
=== Java Client
==== Prerequisites
To use Java thin client, Java 11 or newer is required.
==== Installation
Java client can be added to your project by using maven:
[source, xml]
----
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-client</artifactId>
<version>3.0.0-beta1</version>
</dependency>
----
=== C# Client
==== Prerequisites
To use C# thin client, .NET 6.0 or newer is required.
==== Installation
C# client is available via NuGet. To add it, use the `add package` command:
----
dotnet add package Apache.Ignite --version 3.0.0-beta1
----
=== C++ Client
==== Prerequisites
To run C\++ client, you need a C++ build environment to run the `cmake` command:
- C\++ compiler supporting C++ 17;
- CMake 3.10+;
- One of build systems: make, ninja, MS Visual Studio, or other;
- Conan C/C++ package manager 1.X (optional).
==== Installation
The source code of the C++ client comes with the Apache Ignite 3 distribution. To build it, use the following commands:
[tabs]
--
tab:Windows[]
[source,bat]
----
mkdir cmake-build-release
cd cmake-build-release
conan install .. --build=missing -s build_type=Release
cmake ..
cmake --build . -j8
----
tab:Linux[]
[source,bash,subs="attributes,specialchars"]
----
mkdir cmake-build-release
cd cmake-build-release
conan install .. --build=missing -s build_type=Release -s compiler.libcxx=libstdc++11
cmake .. -DCMAKE_BUILD_TYPE=Release
cmake --build . -j8
----
tab:MacOS[]
[source,bash,subs="attributes,specialchars"]
----
mkdir cmake-build-release
cd cmake-build-release
conan install .. --build=missing -s build_type=Release -s compiler.libcxx=libc++
cmake .. -DCMAKE_BUILD_TYPE=Release
cmake --build . -j8
----
--
== Connecting to Cluster
To initialize a client, use the IgniteClient class, and provide it with the configuration:
[tabs]
--
tab:Java[]
[source, java]
----
try (IgniteClient client = IgniteClient.builder()
.addresses("127.0.0.1:10800")
.build()
) {
// Your code goes here
}
----
tab:.NET[]
[source, csharp]
----
var clientCfg = new IgniteClientConfiguration
{
Endpoints = { "127.0.0.1" }
};
using var client = await IgniteClient.StartAsync(clientCfg);
----
tab:C++[]
[source, cpp]
----
using namespace ignite;
ignite_client_configuration cfg{"127.0.0.1"};
auto client = ignite_client::start(cfg, std::chrono::seconds(5));
----
--
== User Object Serialization
Apache Ignite 3 supports mapping user objects to table tuples. This ensures that objects created in any programming language can be used for key-value operations directly.
=== Limitations
There are limitations to user types that can be used for such a mapping. Some limitations are common, and others are platform-specific due to the programming language used.
- Only flat field structure is supported, meaning no nesting user objects. This is because Ignite tables, and therefore tuples have flat structure themselves;
- Fields should be mapped to Ignite types;
- All fields in user type should either be mapped to Table column or explicitly excluded;
- All columns from Table should be mapped to some field in the user type;
- *Java only*: Users should implement Mapper classes for user types for more flexibility;
- *.NET only*: Any type (class, struct, record) is supported as long as all fields can be mapped to Ignite types;
- *C++ only*: User has to provide marshaling functions explicitly as there is no reflection to generate them based on user type structure.
=== Usage Examples
[tabs]
--
tab:Java[]
[source, java]
----
public static class Account {
public long id;
public long balance;
public Account() {}
public Account(long balance) {
this.balance = balance;
}
}
----
tab:.NET[]
[source, csharp]
----
public class Account
{
public long Id { get; set; }
public long Balance { get; set; }
[NotMapped]
public Guid UnmappedId { get; set; }
}
----
tab:C++[]
[source, cpp]
----
struct account {
account() = default;
account(std::int64_t id) : id(id) {}
account(std::int64_t id, std::int64_t balance) : id(id), balance(balance) {}
std::int64_t id{0};
std::int64_t balance{0};
};
namespace ignite {
template<>
ignite_tuple convert_to_tuple(account &&value) {
ignite_tuple tuple;
tuple.set("id", value.id);
tuple.set("balance", value.balance);
return tuple;
}
template<>
account convert_from_tuple(ignite_tuple&& value) {
account res;
res.id = value.get<std::int64_t>("id");
// Sometimes only key columns are returned, i.e. "id",
// so we have to check whether there are any other columns.
if (value.column_count() > 1)
res.balance = value.get<std::int64_t>("balance");
return res;
}
} // namespace ignite
----
--
== SQL API
Apache Ignite 3 is focused on SQL, and SQL API is the primary way to work with the data. You can read more about supported SQL statements in the link:sql-reference/ddl[SQL Reference] section. Here is how you can send SQL requests:
[tabs]
--
tab:Java[]
[source, java]
----
try (Session session = client.sql().createSession()) {
ResultSet resultSet = session.execute(null, "SELECT name from POJO where id = 42");
SqlRow row = resultSet.next();
assert row.stringValue(0).equals("John Doe");
}
----
tab:.NET[]
[source, csharp]
----
IResultSet<IIgniteTuple> resultSet = await client.Sql.ExecuteAsync(transaction: null, "select name from tbl where id = ?", 42);
List<IIgniteTuple> rows = await resultSet.ToListAsync();
IIgniteTuple row = rows.Single();
Debug.Assert(row["name"] as string == "John Doe");
----
tab:C++[]
[source, cpp]
----
result_set result = client.get_sql().execute(nullptr, {"select name from tbl where id = ?"}, {std::int64_t{42});
std::vector<ignite_tuple> page = result_set.current_page();
ignite_tuple& row = page.front();
assert(row->get<std::int64_t>("id") == 42);
assert(row->get<std::string>("name") == "John Doe");
----
--
== Transactions
All table operations in Apache Ignite 3 are transactional. You can provide an explicit transaction as a first argument of any Table and SQL API call. If you do not provide an explicit transaction, an implicit one will be created for every call.
Here is how you can provide a transaction explicitly:
[tabs]
--
tab:Java[]
[source, java]
----
KeyValueView<Long, Account> accounts =
table.keyValueView(Mapper.of(Long.class), Mapper.of(Account.class));
accounts.put(null, 42, new Account(16_000));
var tx = client.transactions().begin();
Account account = accounts.get(tx, 42);
account.balance += 500;
accounts.put(tx, 42, account);
assert accounts.get(tx, 42).balance == 16_500;
tx.rollback();
assert accounts.get(tx, 42).balance == 16_000;
----
tab:.NET[]
[source, csharp]
----
var accounts = table.GetKeyValueView<long, Account>();
await accounts.PutAsync(transaction: null, 42, new Account(16_000));
await using ITransaction tx = await client.Transactions.BeginAsync();
(Account account, bool hasValue) = await accounts.GetAsync(tx, 42);
account = account with { Balance = account.Balance + 500 };
await accounts.PutAsync(tx, 42, account);
Debug.Assert((await accounts.GetAsync(tx, 42)).Value.Balance == 16_500);
await tx.RollbackAsync();
Debug.Assert((await accounts.GetAsync(null, 42)).Value.Balance == 16_000);
public record Account(decimal Balance);
----
tab:C++[]
[source, cpp]
----
auto accounts = table.get_key_value_view<account, account>();
account init_value(42, 16'000);
accounts.put(nullptr, {42}, init_value);
auto tx = client.get_transactions().begin();
std::optional<account> res_account = accounts.get(&tx, {42});
res_account->balance += 500;
accounts.put(&tx, {42}, res_account);
assert(accounts.get(&tx, {42})->balance == 16'500);
tx.rollback();
assert(accounts.get(&tx, {42})->balance == 16'000);
----
--
== Table API
To execute table operations on a specific table, you need to get a specific view of the table and use one of its methods. You can only create new tables by using SQL API.
When working with tables, you can use built-in Tuple type, which is a set of key-value pairs underneath, or map the data to your own types for a strongly-typed access. Here is how you can work with tables:
=== Getting a Table Instance
First, get an instance of the table. To obtain an instance of table, use the `IgniteTables.table(String)` method. You can also use `IgniteTables.tables()` method to list all existing tables.
[tabs]
--
tab:Java[]
[source, java]
----
IgniteTables tableApi = client.tables();
List<Table> existingTables = tableApi.tables();
Table firstTable = existingTables.get(0);
Table myTable = tableApi.table("MY_TABLE");
----
tab:.NET[]
[source, csharp]
----
var existingTables = await Client.Tables.GetTablesAsync();
var firstTable = existingTables[0];
var myTable = await Client.Tables.GetTableAsync("MY_TABLE");
----
tab:C++[]
[source, cpp]
----
using namespace ignite;
auto table_api = client.get_tables();
std::vector<table> existing_tables = table_api.get_tables();
table first_table = existing_tables.front();
std::optional<table> my_table = table_api.get_table("MY_TABLE);
----
--
=== Basic Table Operations
Once you've got a table you need to get a specific view to choose how you want to operate table records.
==== Binary Record View
A binary record view. It can be used to operate table tuples directly.
[tabs]
--
tab:Java[]
[source, java]
----
RecordView<Tuple> view = table.recordView();
Tuple fullRecord = Tuple.create()
.set("id", 42)
.set("name", "John Doe");
view.upsert(null, fullRecord);
Tuple keyRecord = Tuple.create().set("id", 42);
Tuple resRecord = view.get(null, keyRecord);
assert resRecord.columnCount() == 2;
assert resRecord.intValue("id") == 42;
assert resRecord.stringValue("name").equals("John Doe");
----
tab:.NET[]
[source, csharp]
----
IRecordView<IIgniteTuple> view = table.RecordBinaryView;
IIgniteTuple fullRecord = new IgniteTuple
{
["id"] = 42,
["name"] = "John Doe"
};
await view.UpsertAsync(transaction: null, fullRecord);
IIgniteTuple keyRecord = new IgniteTuple { ["id"] = 42 };
(IIgniteTuple value, bool hasValue) = await view.GetAsync(transaction: null, keyRecord);
Debug.Assert(hasValue);
Debug.Assert(value.FieldCount == 2);
Debug.Assert(value["id"] as int? == 42);
Debug.Assert(value["name"] as string == "John Doe");
----
tab:C++[]
[source, cpp]
----
record_view<ignite_tuple> view = table.get_record_binary_view();
ignite_tuple record{
{"id", 42},
{"name", "John Doe"}
};
view.upsert(nullptr, record);
std::optional<ignite_tuple> res_record = view.get(nullptr, {"id", 42});
assert(res_record.has_value());
assert(res_record->column_count() == 2);
assert(res_record->get<std::int64_t>("id") == 42);
assert(res_record->get<std::string>("name") == "John Doe");
----
--
==== Record View
A record view mapped to a user type. It can be used to operate table using user objects which are mapped to table tuples.
[tabs]
--
tab:Java[]
[source, java]
----
RecordView<Pojo> pojoView = table.recordView(Mapper.of(Pojo.class));
pojoView.upsert(null, new Pojo(42, "John Doe"));
Pojo resRecord = pojoView.get(null, new Pojo(42));
assert resRecord.id == 42;
assert resRecord.name.equals("John Doe");
----
tab:.NET[]
[source, csharp]
----
var pocoView = table.GetRecordView<Poco>();
await pocoView.UpsertAsync(transaction: null, new Poco(42, "John Doe"));
var (value, hasValue) = await pocoView.GetAsync(transaction: null, new Poco(42));
Debug.Assert(hasValue);
Debug.Assert(value.Name == "John Doe");
public record Poco(long Id, string? Name = null);
----
tab:C++[]
[source, cpp]
----
record_view<person> view = table.get_record_view<person>();
person record(42, "John Doe");
view.upsert(nullptr, record);
std::optional<person> res_record = view.get(nullptr, person{42});
assert(res.has_value());
assert(res->id == 42);
assert(res->name == "John Doe");
----
--
==== Key-Value Binary View
A binary key-value view. It can be used to operate table using key and value tuples separately.
[tabs]
--
tab:Java[]
[source, java]
----
KeyValueView<Tuple, Tuple> kvView = table.keyValueView();
Tuple key = Tuple.create().set("id", 42)
Tuple val = Tuple.create().set("name", "John Doe");
kvView.put(null, key, val);
Tuple res = kvView.get(null, key);
assert res.columnCount() == 1;
assert res.stringValue("name").equals("John Doe");
----
tab:.NET[]
[source, csharp]
----
IKeyValueView<IIgniteTuple, IIgniteTuple> kvView = table.KeyValueBinaryView;
IIgniteTuple key = new IgniteTuple { ["id"] = 42 };
IIgniteTuple val = new IgniteTuple { ["name"] = "John Doe" };
await kvView.PutAsync(transaction: null, key, val);
(IIgniteTuple? value, bool hasValue) = await kvView.GetAsync(transaction: null, key);
Debug.Assert(hasValue);
Debug.Assert(value.FieldCount == 1);
Debug.Assert(value["name"] as string == "John Doe");
----
tab:C++[]
[source, cpp]
----
key_value_view<ignite_tuple, ignite_tuple> kv_view = table.get_key_value_binary_view();
ignite_tuple key_tuple{{"id", 42}};
ignite_tuple val_tuple{{"name", "John Doe"}};
kv_view.put(nullptr, key_tuple, val_tuple);
std::optional<ignite_tuple> res_tuple = kv_view.get(nullptr, key_tuple);
assert(res_tuple.has_value());
assert(res_tuple->column_count() == 2);
assert(res_tuple->get<std::int64_t>("id") == 42);
assert(res_tuple->get<std::string>("name") == "John Doe");
----
--
==== Key-Value View
A key-value view with user objects. It can be used to operate table using key and value user objects mapped to table tuples.
[tabs]
--
tab:Java[]
[source, java]
----
KeyValueView<Long, Pojo> pojoView =
table.keyValueView(Mapper.of(Long.class), Mapper.of(Pojo.class));
pojoView.put(null, 42, new Pojo("John Doe"));
Pojo val = pojoView.get(null, 42);
assert val.name.equals("John Doe");
----
tab:.NET[]
[source, csharp]
----
IKeyValueView<long, Poco> kvView = table.GetKeyValueView<long, Poco>();
await kvView.PutAsync(transaction: null, 42, new Poco(Id: 0, Name: "John Doe"));
(Poco? value, bool hasValue) = await kvView.GetAsync(transaction: null, 42);
Debug.Assert(hasValue);
Debug.Assert(value.Name == "John Doe");
public record Poco(long Id, string? Name = null);
----
tab:C++[]
[source, cpp]
----
key_value_view<person, person> kv_view = table.get_key_value_view<person, person>();
kv_view.put(nullptr, {42}, {"John Doe"});
std::optional<person> res = kv_view.get(nullptr, {42});
assert(res.has_value());
assert(res->id == 42);
assert(res->name == "John Doe");
----
--
== Executing Compute Tasks
Apache Ignite 3 clients support basic compute capabilities. You can execute compute tasks that are already deployed in the cluster.
You can run a task across all cluster nodes or a specific cluster group. The deployment assumes that you create a JAR file with the compute tasks and add the JAR to the cluster nodes' classpath.
The example below shows how to get access to the compute APIs and execute the compute task named `MyTask`:
[tabs]
--
tab:Java[]
[source, java]
----
String result = client.compute().<String>execute(
client.clusterNodes(), "MyTask", "Lorem", "ipsum", 42);
----
tab:.NET[]
[source, csharp]
----
IList<IClusterNode> nodes = await client.GetClusterNodesAsync();
string res = await client.Compute.ExecuteAsync<string>(nodes, "org.foo.bar.MyTask", 42)
----
tab:C++[]
[source, cpp]
----
std::vector<cluster_node> nodes = client.get_cluster_nodes();
std::optional<primitive> res = client.get_compute().execute(nodes, "org.foo.bar.MyTask", {42})
std::cout << res->get<std::string>() << std::endl;
----
--
== Partition Awareness
In Apache Ignite 3, partition awareness is enabled automatically for all clients.
Data in the cluster is distributed between the nodes in a balanced manner for scalability and performance reasons. Each cluster node maintains a subset of the data, and the partition distribution map, which is used to determine the node that keeps the primary/backup copy of requested entries.
Partition awareness allows the client to send query requests directly to the node that owns the queried data.
Without partition awareness, an application that is connected to the cluster via a client would execute all queries and operations via a single server node that acts as a proxy for the incoming requests.
These operations would then be re-routed to the node that stores the data that is being requested.
This would result in a bottleneck that could prevent the application from scaling linearly.
image::images/partitionawareness01.png[Without Partition Awareness]
Notice how queries must pass through the proxy server node, where they are routed to the correct node.
With partition awareness in place, the client can directly route queries and operations to the primary nodes that own the data required for the queries.
This eliminates the bottleneck, allowing the application to scale more easily.
image::images/partitionawareness02.png[With Partition Awareness]