| <!-- |
| |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| |
| --> |
| |
| # Cluster Quick Start |
| |
| The purpose of this article is to show how to start, expand, and shrink an IoTDB cluster in an easy way. |
| |
| See also: |
| [FAQ](../FAQ/Frequently-asked-questions.md) |
| |
| |
| ## 1. Installation and deployment |
| |
| As an example, we'd like to start an IoTDB cluster with 3 ConfigNodes and 3 DataNodes(3C3D) with minimum modifications. Thus, |
| - the cluster name is defaultCluster |
| - data/schema replica is 1 |
| - the max heap size of ConfigNode take the 1/4 of the computer |
| - the max heap size of DataNode take the 1/4 of the computer |
| |
| Suppose there are 3 computers(3 nodes we called here) with Linux OS and JDK installed(detail see [Prerequisites](../QuickStart/QuickStart.md)) and IoTDB working directory is `/data/iotdb`. |
| IP address and configurations is like below: |
| |
| | Node IP | 192.168.132.10 | 192.168.132.11 | 192.168.132.12 | |
| |--------|:---------------|:---------------|:---------------| |
| | service | ConfigNode | ConfigNode | ConfigNode | |
| | service | DataNode | DataNode | DataNode | |
| |
| Port: |
| |
| | Service | ConfigNode | DataNode | |
| |---|---|---| |
| |port| 10710, 10720 | 6667, 10730, 10740, 10750, 10760 | |
| |
| **illustration：** |
| - We could use IP address or hostname/domain to set up an IoTDB cluster, then we would take IP address. If using hostname/domain, `/etc/hosts` must be set well. |
| - JVM memory configuration: `MAX_HEAP_SIZE` in `confignode-env.sh` and `datanode-env.sh`, equal to or greater than 1G is recommended. It's enough for ConfigNode taking 1~2G. The memory taking of DataNode is decided by the inputing and querying data. |
| |
| ### 1.1 download |
| In every computer, [Download](https://iotdb.apache.org/Download/) the IoTDB install package and extract it to working directory of `/data/iotdb`. |
| Then get the directory tree: |
| ```shell |
| /data/iotdb/ |
| ├── conf # configuration files |
| ├── lib # jar library |
| ├── sbin # start/stop shell etc. |
| └── tools # other tools |
| ``` |
| |
| ### 1.2. configuration |
| |
| Configuration files are in `/data/iotdb/conf`. |
| Modify the specified configuration file according to the table below: |
| |
| | Configuration| Configuration Option | IP:192.168.132.10 | IP:192.168.132.11 | IP:192.168.132.12 | |
| |------------|:-------------------------------|----------------------|----------------------|:---------------------| |
| | iotdb-confignode.properties | cn\_internal\_address | 192.168.132.10 | 192.168.132.11 | 192.168.132.12 | |
| | | cn\_target\_config\_node\_list | 192.168.132.10:10710 | 192.168.132.10:10710 | 192.168.132.10:10710 | |
| | iotdb-datanode.properties | dn\_rpc\_address | 192.168.132.10 | 192.168.132.11 | 192.168.132.12 | |
| | | dn\_internal\_address | 192.168.132.10 | 192.168.132.11 | 192.168.132.12 | |
| | | dn\_target\_config\_node\_list | 192.168.132.10:10710 | 192.168.132.10:10710 | 192.168.132.10:10710 | |
| |
| **Notice:** |
| It's recommended that the configurations of iotdb-common.properties and the heap size of JVM in all nodes are the same. |
| |
| ### 1.3. start IoTDB cluster |
| Before starting the IoTDB cluster, make sure the configurations are correct and there is no any data in the working directory. |
| |
| #### 1.3.1. start the first node |
| That is `cn_target_config_node_list` in above configuration table. |
| Execute these commands below in node of `192.168.132.10`. |
| ```shell |
| cd /data/iotdb |
| # start ConfigNode and DataNode services |
| sbin/start-standalone.sh |
| |
| # check DataNode logs to see whether starting successfully or not |
| tail -f logs/log_datanode_all.log |
| # expecting statements like below |
| # 2023-07-21 20:26:01,881 [main] INFO o.a.i.db.service.DataNode:192 - Congratulation, IoTDB DataNode is set up successfully. Now, enjoy yourself! |
| ``` |
| If there is no such logs mentioned abolve or there are some `Exception`s in log files, it's failed. Then please check `log_confignode_all.log` and `log_datanode_all.log` in directory of `/data/iotdb/logs`. |
| |
| **Notice**： |
| - Make sure the first node, especially the first ConfigNode that `cn_target_config_node_list` specified, starting successfully, and then start the other services. |
| - If starting failed，it's necessary to do [cleanup](#【reference】cleanup) before starting again. |
| - How to start service ConfigNode or DataNode alone: |
| ```shell |
| # start ConfigNode alone in daemon |
| sbin/start-confignode.sh -d |
| # start DataNode alone in daemon |
| sbin/start-datanode.sh -d |
| ``` |
| |
| #### 1.3.2. start service ConfigNode and DataNode in other nodes |
| Execute commands below in both 192.168.132.11 and 192.168.132.12: |
| ```shell |
| cd /data/iotdb |
| # start service ConfigNode and DataNode |
| sbin/start-standalone.sh |
| ``` |
| If starting failed, it's necessary to do [cleanup](#【reference】cleanup) in all nodes, and then doging all again from starting the first node. |
| |
| #### 1.3.3. check the cluster status |
| If everything goes well, the cluster will start successfully. Then, we can start the Cli for verification. |
| ```shell |
| /data/iotdb/sbin/start-cli.sh -h 192.168.132.10 |
| IoTDB>show cluster; |
| # example result: |
| +------+----------+-------+---------------+------------+-------+---------+ |
| |NodeID| NodeType| Status|InternalAddress|InternalPort|Version|BuildInfo| |
| +------+----------+-------+---------------+------------+-------+---------+ |
| | 0|ConfigNode|Running| 192.168.132.10| 10710|1.x.x | xxxxxxx| |
| | 1| DataNode|Running| 192.168.132.10| 10730|1.x.x | xxxxxxx| |
| | 2|ConfigNode|Running| 192.168.132.11| 10710|1.x.x | xxxxxxx| |
| | 3| DataNode|Running| 192.168.132.11| 10730|1.x.x | xxxxxxx| |
| | 4|ConfigNode|Running| 192.168.132.12| 10710|1.x.x | xxxxxxx| |
| | 5| DataNode|Running| 192.168.132.12| 10730|1.x.x | xxxxxxx| |
| +------+----------+-------+---------------+------------+-------+---------+ |
| ``` |
| **illustration:** |
| The IP address of `start-cli.sh -h` could be any IP address of DataNode service. |
| |
| |
| ### 【reference】Cleanup |
| Execute commands in every node: |
| 1. End processes of ConfigNode and DataNode |
| ```shell |
| # 1. Stop services ConfigNode and DataNode |
| sbin/stop-standalone.sh |
| |
| # 2. Check whether there are IoTDB processes left or not |
| jps |
| # 或者 |
| ps -ef|gerp iotdb |
| |
| # 3. If there is any IoTDB process left, kill it |
| kill -9 <pid> |
| # If there is only 1 IoTDB instance, execue command below to remove all IoTDB process |
| ps -ef|grep iotdb|grep -v grep|tr -s ' ' ' ' |cut -d ' ' -f2|xargs kill -9 |
| ``` |
| |
| 2. Remove directories of data and logs |
| ```shell |
| cd /data/iotdb |
| rm -rf data logs |
| ``` |
| **illustration:** |
| It's necessary to remove directory of `data` but it's not necessary to remove directory of `logs`, only for convenience. |
| |
| |
| ## 2. Expand |
| `Expand` means add services of ConfigNode or DataNode into an existing IoTDB cluster. |
| |
| It's the same as starting the other nodes mentioned above. That is downloading IoTDB install package, extracting, configurating and then starting. The new node here is `192.168.132.13`. |
| **Notice** |
| - It's must be cleaned up, in other words doing [cleanup](#cleanup) in it. |
| - `cluster_name` of `iotdb-common.properties` must be the same to the cluster. |
| - `cn_target_config_node_list` and `dn_target_config_node_list` must be the same to the cluster. |
| - The old data wouldn't be moved to the new node but the new data would be. |
| |
| ### 2.1. configuration |
| Modify the specified configuration file according to the table below: |
| |
| | Configuration | Configuration Option| IP:192.168.132.13 | |
| |------------|:-------------------------------|:---------------------| |
| | iotdb-confignode.properties | cn\_internal\_address | 192.168.132.13 | |
| | | cn\_target\_config\_node\_list | 192.168.132.10:10710 | |
| | iotdb-datanode.properties | dn\_rpc\_address | 192.168.132.13 | |
| | | dn\_internal\_address | 192.168.132.13 | |
| | | dn\_target\_config\_node\_list | 192.168.132.10:10710 | |
| |
| ### 2.2. expand |
| Execute commands below in new node of `192.168.132.13`: |
| ```shell |
| cd /data/iotdb |
| # start service ConfigNode and DataNode |
| sbin/start-standalone.sh |
| ``` |
| |
| ### 2.3. check the result |
| Execute `show cluster` through Cli and the result like below: |
| ```shell |
| /data/iotdb/sbin/start-cli.sh -h 192.168.132.10 |
| IoTDB>show cluster; |
| # example result： |
| +------+----------+-------+---------------+------------+-------+---------+ |
| |NodeID| NodeType| Status|InternalAddress|InternalPort|Version|BuildInfo| |
| +------+----------+-------+---------------+------------+-------+---------+ |
| | 0|ConfigNode|Running| 192.168.132.10| 10710|1.x.x | xxxxxxx| |
| | 1| DataNode|Running| 192.168.132.10| 10730|1.x.x | xxxxxxx| |
| | 2|ConfigNode|Running| 192.168.132.11| 10710|1.x.x | xxxxxxx| |
| | 3| DataNode|Running| 192.168.132.11| 10730|1.x.x | xxxxxxx| |
| | 4|ConfigNode|Running| 192.168.132.12| 10710|1.x.x | xxxxxxx| |
| | 5| DataNode|Running| 192.168.132.12| 10730|1.x.x | xxxxxxx| |
| | 6|ConfigNode|Running| 192.168.132.13| 10710|1.x.x | xxxxxxx| |
| | 7| DataNode|Running| 192.168.132.13| 10730|1.x.x | xxxxxxx| |
| +------+----------+-------+---------------+------------+-------+---------+ |
| ``` |
| |
| |
| ## 3. Remove service |
| `Shrink` means removing a service from the IoTDB cluster. |
| **Notice:** |
| - `Shrink` could be done in any node within the cluster |
| - Any service could be shrinked within cluster. But the DataNode service of the cluster must greater than the data replica of iotdb-common.properties. |
| - Be patient to wait for the end of shrinking, and then read the guide in logs carefully. |
| |
| ### 3.1 shrink service ConfigNode |
| ```shell |
| cd /data/iotdb |
| # way 1: shrink with ip:port |
| sbin/remove-confignode.sh 192.168.132.13:10710 |
| |
| # way 2: shrink with NodeID of `show cluster` |
| sbin/remove-confignode.sh 6 |
| ``` |
| |
| ### 3.2 shrink service DataNode |
| ```shell |
| cd /data/iotdb |
| # way 1: shrink with ip:port |
| sbin/remove-datanode.sh 192.168.132.13:6667 |
| |
| # way 2: shrink with NodeID of `show cluster` |
| sbin/remove-confignode.sh 7 |
| ``` |
| |
| ### 3.3 check the result |
| |
| Execute `show cluster` through Cli, the result is like below: |
| ```shell |
| +------+----------+-------+---------------+------------+-------+---------+ |
| |NodeID| NodeType| Status|InternalAddress|InternalPort|Version|BuildInfo| |
| +------+----------+-------+---------------+------------+-------+---------+ |
| | 0|ConfigNode|Running| 192.168.132.10| 10710|1.x.x | xxxxxxx| |
| | 1| DataNode|Running| 192.168.132.10| 10730|1.x.x | xxxxxxx| |
| | 2|ConfigNode|Running| 192.168.132.11| 10710|1.x.x | xxxxxxx| |
| | 3| DataNode|Running| 192.168.132.11| 10730|1.x.x | xxxxxxx| |
| | 4|ConfigNode|Running| 192.168.132.12| 10710|1.x.x | xxxxxxx| |
| | 5| DataNode|Running| 192.168.132.12| 10730|1.x.x | xxxxxxx| |
| +------+----------+-------+---------------+------------+-------+---------+ |
| ``` |
