| --- |
| sidebar_label: "Overview" |
| title: Installation & Deployment |
| sidebar_position: 1 |
| --- |
| |
| # Overview |
| |
| Below, we provide an overview of the key components of a Fluss cluster, detailing their functionalities and implementations. Additionally, we will introduce the various deployment methods available for Fluss. |
| |
| ## Overview and Reference Architecture |
| |
| The figure below shows the building blocks of Fluss clusters: |
| |
| <img width="1200px" src={require('../assets/deployment_overview.png').default} /> |
| |
| |
| |
| When deploying Fluss, there are often multiple options available for each building block. |
| We have listed them in the table below the figure. |
| |
| |
| <table class="table table-bordered"> |
| <thead> |
| <tr> |
| <th class="text-left" width="250">Component</th> |
| <th class="text-left" width="600">Purpose</th> |
| <th class="text-left" width="300">Implementations</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td>Fluss Client</td> |
| <td> |
| <p> |
| The Fluss Client is the entry point for users to interact with Fluss Cluster. It is responsible for |
| managing Fluss Cluster like: |
| </p> |
| <ul> |
| <li> Admin operation: like create or delete database/table etc</li> |
| <li>Table operation: like write, read, delete data</li> |
| </ul> |
| </td> |
| <td> |
| <ul> |
| <li>[Flink Connector](engine-flink/getting-started.md)</li> |
| </ul> |
| </td> |
| </tr> |
| <tr> |
| <td>CoordinatorServer</td> |
| <td> |
| <p> |
| CoordinatorServer is the name of the central work coordination component of Fluss. |
| The coordinator server is responsible for: |
| </p> |
| <ul> |
| <li>Manage the TabletServer</li> |
| <li>Manage the metadata</li> |
| <li>Coordinate the whole cluster, e.g. data re-balance, recover data when tablet servers down</li> |
| </ul> |
| </td> |
| <td rowspan="2"> |
| <ul> |
| <li>[Local Cluster](install-deploy/deploying-local-cluster.md)</li> |
| <li>[Distributed Cluster](install-deploy/deploying-distributed-cluster.md)</li> |
| <li>[Docker run / Docker compose](install-deploy/deploying-with-docker.md)</li> |
| </ul> |
| </td> |
| </tr> |
| <tr> |
| <td>TabletServer</td> |
| <td> |
| <p> |
| TabletServers are the actual node to manage and store data. |
| </p> |
| </td> |
| </tr> |
| <tr> |
| <td colspan="3" style={{ textAlign: "center" }}> |
| <b>External Components</b> |
| </td> |
| </tr> |
| <tr> |
| <td>ZooKeeper</td> |
| <td> |
| :::warning |
| ZooKeeper will be removed to simplify deployment in the near future. For more details, please checkout [Roadmap](/roadmap/). |
| ::: |
| <p> |
| Fluss leverages ZooKeeper for distributed coordination between all running CoordinatorServer instances and for metadata management. |
| </p> |
| </td> |
| <td> |
| <ul> |
| <li><a href="https://zookeeper.apache.org/">ZooKeeper</a></li> |
| </ul> |
| </td> |
| </tr> |
| <tr> |
| <td>Remote Storage (optional)</td> |
| <td> |
| Fluss uses file systems as remote storage to store snapshots for Primary-Key Table and store tiered log segments for Log Table. |
| </td> |
| <td> |
| <li>[HDFS](maintenance/filesystems/hdfs.md)</li> |
| <li>[Aliyun OSS](maintenance/filesystems/oss.md)</li> |
| <li>[Amazon S3](maintenance/filesystems/s3.md)</li> |
| </td> |
| </tr> |
| <tr> |
| <td>Lakehouse Storage (optional)</td> |
| <td> |
| Fluss's DataLake Tiering Service will continuously compact Fluss's Arrow files into Parquet/ORC files in open lake format. |
| The data in Lakehouse storage can be read both by Fluss's client in a Union Read manner and accessed directly |
| by query engines such as Flink, Spark, StarRocks, Trino. |
| </td> |
| <td> |
| <li>[Paimon](streaming-lakehouse/integrate-data-lakes/paimon.md)</li> |
| <li>[Iceberg](streaming-lakehouse/integrate-data-lakes/iceberg.md)</li> |
| <li>[Lance](streaming-lakehouse/integrate-data-lakes/lance.md)</li> |
| </td> |
| </tr> |
| <tr> |
| <td>Metrics Storage (optional)</td> |
| <td> |
| CoordinatorServer/TabletServer report internal metrics and Fluss client (e.g., connector in Flink jobs) can report additional, client specific metrics as well. |
| </td> |
| <td> |
| <li>[JMX](maintenance/observability/metric-reporters.md#jmx)</li> |
| <li>[Prometheus](maintenance/observability/metric-reporters.md#prometheus)</li> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| |
| ## How to deploy Fluss |
| |
| Fluss can be deployed in three different ways: |
| - [Local Cluster](install-deploy/deploying-local-cluster.md) |
| - [Distributed Cluster](install-deploy/deploying-distributed-cluster.md) |
| - [Docker run / Docker Compose](install-deploy/deploying-with-docker.md) |
| |
| **NOTE**: |
| - Local Cluster is for testing purpose only. |