| <!-- |
| |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| |
| --> |
| |
| ## Cluster Setup |
| |
| This article is the setup process of IoTDB Cluster (0.14.0-preview1). |
| |
| ## Environments |
| |
| 1. JDK>=1.8. |
| |
| 2. Max open file 65535. |
| |
| 3. Disable the swap memory. |
| |
| ## Get the binary distribution |
| |
| ### Download |
| |
| Download the binary distribution from website [Download Page](https://iotdb.apache.org/Download/). |
| |
| ### Compiled from source code |
| |
| Download the source code: |
| |
| ``` |
| git clone https://github.com/apache/iotdb.git |
| ``` |
| |
| The default branch is master, you should checkout to the release tag: |
| |
| ``` |
| git checkout v0.14.0-preview1 |
| ``` |
| |
| Under the source root folder: |
| |
| ``` |
| mvn clean package -pl distribution -am -DskipTests |
| ``` |
| |
| Then you will get the binary distribution under **distribution/target**, in which the **all-bin** contains ConfigNode and DataNode, and DataNode contains the Cli. |
| |
| ## Binary Distribution Content |
| |
| | **Folder** | **Description** | |
| |-------------------------|-------------------------------------------------------------------------------------| |
| | confignode | Contains start/stop/remove shell, configurations, logs, data of ConfigNode | |
| | datanode | Contains start/stop/remove shell, configurations, logs, data of DataNode, cli shell | |
| | grafana-metrics-example | Grafana metric page module | |
| | lib | Jar files folder | |
| | tools | System tools | |
| |
| ## Start the Cluster |
| |
| Users could start a cluster which contains multiple ConfigNode and DataNode. |
| A cluster need at least one ConfigNode and no less than the number of data/schema_replication_factor DataNodes. |
| |
| The total process are three steps: |
| |
| * Start the first ConfigNode |
| * Add ConfigNode (Optional) |
| * Add DataNode |
| |
| ### Start the first ConfigNode |
| |
| Please set the important parameters in iotdb-confignode.properties: |
| |
| | **Configuration** | **Description** | |
| |--------------------------------------------|----------------------------------------------------------------------------------------------| |
| | internal\_address | Internal rpc service address of ConfigNode | |
| | internal\_port | Internal rpc service address of ConfigNode | |
| | consensus\_port | ConfigNode replication consensus protocol communication port | |
| | target\_config\_nodes | Target ConfigNode address, if the current is the first ConfigNode, then set its address:port | |
| | data\_replication\_factor | Data replication factor, no more than DataNode number | |
| | data\_region\_consensus\_protocol\_class | Consensus protocol of data replicas | |
| | schema\_replication\_factor | Schema replication factor, no more than DataNode number | |
| | schema\_region\_consensus\_protocol\_class | Consensus protocol of schema replicas | |
| |
| Start on Linux: |
| ``` |
| # Foreground |
| ./confignode/sbin/start-confignode.sh |
| |
| # Background |
| nohup ./confignode/sbin/start-confignode.sh >/dev/null 2>&1 & |
| ``` |
| |
| Start on Windows: |
| ``` |
| confignode\sbin\start-confignode.bat |
| ``` |
| |
| More details [ConfigNode Configurations](https://iotdb.apache.org/UserGuide/Master/Reference/ConfigNode-Config-Manual.html). |
| |
| ### Add ConfigNode (Optional) |
| |
| This will add the replication factor of ConfigNode, except for the port couldn't conflict, make sure other configurations are the same with existing ConfigNode in Cluster. |
| |
| The adding ConfigNode also use the start-confignode.sh/bat. |
| |
| ### Start DataNode |
| |
| You could add any number of DataNode. |
| |
| Please set the important parameters in iotdb-datanode.properties. |
| |
| | **Configuration** | **Description** | |
| |---------------------------------|--------------------------------------------------| |
| | rpc\_address | Client RPC Service address | |
| | rpc\_port | Client RPC Service port | |
| | internal\_address | Control flow address of DataNode inside cluster | |
| | internal\_port | Control flow port of DataNode inside cluster | |
| | mpp\_data\_exchange\_port | Data flow port of DataNode inside cluster | |
| | data\_region\_consensus\_port | Data replicas communication port for consensus | |
| | schema\_region\_consensus\_port | Schema replicas communication port for consensus | |
| | target\_config\_nodes | Running ConfigNode of the Cluster | |
| |
| Start on Linux: |
| ``` |
| # Foreground |
| ./datanode/sbin/start-datanode.sh |
| |
| # Background |
| nohup ./datanode/sbin/start-datanode.sh >/dev/null 2>&1 & |
| ``` |
| |
| Start on Windows: |
| ``` |
| datanode\sbin\start-datanode.bat |
| ``` |
| |
| More details [DataNode Configurations](https://iotdb.apache.org/UserGuide/Master/Reference/DataNode-Config-Manual.html). |
| |
| ### Start Cli |
| |
| Cli is in datanode/sbin folder. |
| |
| Start on Linux: |
| ``` |
| ./datanode/sbin/start-cli.sh |
| ``` |
| |
| Start on Windows: |
| ``` |
| datanode\sbin\start-cli.bat |
| ``` |
| |
| ## Shrink the Cluster |
| |
| ### Remove ConfigNode |
| |
| Execute the remove-confignode shell on an active ConfigNode. |
| |
| Remove on Linux: |
| ``` |
| ./confignode/sbin/remove-confignode.sh <confignode_id> |
| ./confignode/sbin/remove-confignode.sh <internal_address>:<internal_port> |
| ``` |
| |
| Remove on Windows: |
| ``` |
| confignode\sbin\remove-confignode.bat <confignode_id> |
| confignode\sbin\remove-confignode.bat <internal_address>:<internal_port> |
| ``` |
| |
| ### Remove DataNode |
| |
| Execute the remove-datanode shell on an active DataNode. |
| |
| Remove on Linux: |
| ``` |
| ./datanode/sbin/remove-datanode.sh <datanode_id> |
| ./datanode/sbin/remove-datanode.sh <rpc_address>:<rpc_port> |
| ``` |
| |
| Remove on Windows: |
| ``` |
| datanode\sbin\remove-datanode.bat <datanode_id> |
| datanode\sbin\remove-datanode.bat <rpc_address>:<rpc_port> |
| ``` |
| |
| ## Quick Start |
| |
| This section uses a local environment as an example to |
| illustrate how to start, expand, and shrink a IoTDB Cluster. |
| |
| ### 1. Prepare the Start Environment |
| |
| Unzip the apache-iotdb-0.14.0-preview1-all-bin.zip file to cluster0 folder. |
| |
| ### 2. Starting a Minimum Cluster |
| |
| Starting the Cluster version with one ConfigNode and one DataNode(1C1D), |
| the default number of replica is one. |
| ``` |
| ./cluster0/confignode/sbin/start-confignode.sh |
| ./cluster0/datanode/sbin/start-datanode.sh |
| ``` |
| |
| ### 3. Verify the Minimum Cluster |
| |
| + The minimum cluster is successfully started. Start the Cli for verification. |
| ``` |
| ./cluster0/datanode/sbin/start-cli.sh |
| ``` |
| |
| + Execute the [show cluster](https://iotdb.apache.org/UserGuide/Master/Maintenance-Tools/Maintenance-Command.html#show-all-node-information) |
| command on the Cli. The result is shown below: |
| ``` |
| IoTDB> show cluster |
| +------+----------+-------+---------------+------------+ |
| |NodeID| NodeType| Status|InternalAddress|InternalPort| |
| +------+----------+-------+---------------+------------+ |
| | 0|ConfigNode|Running| 0.0.0.0| 22277| |
| | 1| DataNode|Running| 127.0.0.1| 9003| |
| +------+----------+-------+---------------+------------+ |
| Total line number = 2 |
| It costs 0.160s |
| ``` |
| |
| ### 4. Prepare the Expanding Environment |
| |
| Unzip the apache-iotdb-0.14.0-preview1-all-bin.zip file to cluster1 and cluster2 folder. |
| |
| ### 5. Modify the Node Configuration file |
| |
| For folder cluster1: |
| |
| + Modify ConfigNode address: |
| |
| | **configuration item** | **value** | |
| |------------------------|---------------| |
| | internal\_address | 0.0.0.0 | |
| | internal\_port | 22279 | |
| | consensus\_port | 22280 | |
| | target\_config\_nodes | 0.0.0.0:22277 | |
| |
| + Modify DataNode address: |
| |
| | **configuration item** | **value** | |
| |---------------------------------|-----------------| |
| | rpc\_address | 0.0.0.0 | |
| | rpc\_port | 6668 | |
| | internal\_address | 127.0.0.1 | |
| | internal\_port | 9004 | |
| | mpp\_data\_exchange\_port | 8778 | |
| | data\_region\_consensus\_port | 40011 | |
| | schema\_region\_consensus\_port | 50011 | |
| | target\_config\_nodes | 127.0.0.1:22277 | |
| |
| For folder cluster1: |
| |
| + Modify ConfigNode address: |
| |
| | **configuration item** | **value** | |
| |------------------------|---------------| |
| | internal\_address | 0.0.0.0 | |
| | internal\_port | 22281 | |
| | consensus\_port | 22282 | |
| | target\_config\_nodes | 0.0.0.0:22277 | |
| |
| + Modify DataNode address: |
| |
| | **configuration item** | **value** | |
| |---------------------------------|-----------------| |
| | rpc\_address | 0.0.0.0 | |
| | rpc\_port | 6669 | |
| | internal\_address | 127.0.0.1 | |
| | internal\_port | 9005 | |
| | mpp\_data\_exchange\_port | 8779 | |
| | data\_region\_consensus\_port | 40012 | |
| | schema\_region\_consensus\_port | 50012 | |
| | target\_config\_nodes | 127.0.0.1:22277 | |
| |
| ### 6. Expanding the Cluster |
| |
| Expanding the Cluster to three ConfigNode and three DataNode(3C3D). |
| The following commands can be executed in no particular order. |
| |
| ``` |
| ./cluster1/confignode/sbin/start-confignode.sh |
| ./cluster1/datanode/sbin/start-datanode.sh |
| ./cluster2/confignode/sbin/start-confignode.sh |
| ./cluster2/datanode/sbin/start-datanode.sh |
| ``` |
| |
| ### 7. Verify Cluster expansion |
| |
| Execute the show cluster command, the result is shown below: |
| ``` |
| IoTDB> show cluster |
| +------+----------+-------+---------------+------------+ |
| |NodeID| NodeType| Status|InternalAddress|InternalPort| |
| +------+----------+-------+---------------+------------+ |
| | 0|ConfigNode|Running| 0.0.0.0| 22277| |
| | 2|ConfigNode|Running| 0.0.0.0| 22279| |
| | 3|ConfigNode|Running| 0.0.0.0| 22281| |
| | 1| DataNode|Running| 127.0.0.1| 9003| |
| | 4| DataNode|Running| 127.0.0.1| 9004| |
| | 5| DataNode|Running| 127.0.0.1| 9005| |
| +------+----------+-------+---------------+------------+ |
| Total line number = 6 |
| It costs 0.012s |
| ``` |
| |
| ### 8. Shrinking the Cluster |
| |
| + Remove a ConfigNode: |
| ``` |
| ./cluster0/confignode/sbin/remove-confignode.sh 127.0.0.1:22279 |
| ``` |
| |
| + Remove a DataNode: |
| ``` |
| ./cluster0/datanode/sbin/remove-datanode.sh 127.0.0.1:6668 |
| ``` |
| |
| ### 9. Verify Cluster shrinkage |
| |
| Execute the show cluster command, the result is shown below: |
| ``` |
| IoTDB> show cluster |
| +------+----------+-------+---------------+------------+ |
| |NodeID| NodeType| Status|InternalAddress|InternalPort| |
| +------+----------+-------+---------------+------------+ |
| | 0|ConfigNode|Running| 0.0.0.0| 22277| |
| | 3|ConfigNode|Running| 0.0.0.0| 22281| |
| | 1| DataNode|Running| 127.0.0.1| 9003| |
| | 5| DataNode|Running| 127.0.0.1| 9005| |
| +------+----------+-------+---------------+------------+ |
| Total line number = 4 |
| It costs 0.007s |
| ``` |
