blob: 603dad8b747b023e1b0df2a5e2291a7767920106 [file] [view]
---
title: Secure Your Fluss Cluster
sidebar_position: 2
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# Secure Your Fluss Cluster in Minutes
This guide demonstrates how to secure your Fluss cluster using two practical examples:
1. Securing a Fluss cluster within a department with different roles
2. Enabling multi-tenant isolation in a Fluss cluster
These scenarios will help you understand how to configure authentication and authorization, manage access control, and implement data isolation in real-world use cases.
## Example 1: Secure Fluss with Different Roles
In this example, we assume there are three users within a department:
* `admin`: A superuser who can manage the entire Fluss cluster.
* `developer`: A user that is allowed to read and write data.
* `consumer`: A user that is allowed to read data only.
### Environment Setup
#### Prerequisites
Before proceeding with this guide, ensure that [Docker](https://docs.docker.com/engine/install/) and the [Docker Compose plugin](https://docs.docker.com/compose/install/linux/) are installed on your machine.
All commands were tested with Docker version 27.4.0 and Docker Compose version v2.30.3.
:::note
We encourage you to use a recent version of Docker and [Compose v2](https://docs.docker.com/compose/releases/migrate/) (however, Compose v1 might work with a few adaptations).
:::
#### Starting required components
We will use docker compose to spin up the required components for this tutorial.
1. Create a working directory for this guide.
```shell
mkdir fluss-quickstart-security
cd fluss-quickstart-security
```
2. Create a `docker-compose.yml` file with the following content:
```yaml
services:
#begin Fluss cluster
coordinator-server:
image: apache/fluss:$FLUSS_DOCKER_VERSION$
command: coordinatorServer
depends_on:
- zookeeper
environment:
- |
FLUSS_PROPERTIES=
zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://coordinator-server:0, CLIENT://coordinator-server:9123
internal.listener.name: INTERNAL
remote.data.dir: /tmp/fluss/remote-data
# security properties
security.protocol.map: CLIENT:SASL, INTERNAL:PLAINTEXT
security.sasl.enabled.mechanisms: PLAIN
security.sasl.plain.jaas.config: org.apache.fluss.security.auth.sasl.plain.PlainLoginModule required user_admin="admin-pass" user_developer="developer-pass" user_consumer="consumer-pass";
authorizer.enabled: true
super.users: User:admin
tablet-server:
image: apache/fluss:$FLUSS_DOCKER_VERSION$
command: tabletServer
depends_on:
- coordinator-server
environment:
- |
FLUSS_PROPERTIES=
zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://tablet-server:0, CLIENT://tablet-server:9123
internal.listener.name: INTERNAL
tablet-server.id: 0
kv.snapshot.interval: 0s
data.dir: /tmp/fluss/data
remote.data.dir: /tmp/fluss/remote-data
# security properties
security.protocol.map: CLIENT:SASL, INTERNAL:PLAINTEXT
security.sasl.enabled.mechanisms: PLAIN
security.sasl.plain.jaas.config: org.apache.fluss.security.auth.sasl.plain.PlainLoginModule required user_admin="admin-pass" user_developer="developer-pass" user_consumer="consumer-pass";
authorizer.enabled: true
super.users: User:admin
volumes:
- shared-tmpfs:/tmp/fluss
zookeeper:
restart: always
image: zookeeper:3.9.2
#end
#begin Flink cluster
jobmanager:
image: apache/fluss-quickstart-flink:$FLUSS_QUICKSTART_FLINK_DOCKER_VERSION$
ports:
- "8083:8081"
command: jobmanager
environment:
- |
FLINK_PROPERTIES=
jobmanager.rpc.address: jobmanager
volumes:
- shared-tmpfs:/tmp/paimon
taskmanager:
image: apache/fluss-quickstart-flink:$FLUSS_QUICKSTART_FLINK_DOCKER_VERSION$
depends_on:
- jobmanager
command: taskmanager
environment:
- |
FLINK_PROPERTIES=
jobmanager.rpc.address: jobmanager
taskmanager.numberOfTaskSlots: 10
taskmanager.memory.process.size: 2048m
taskmanager.memory.framework.off-heap.size: 256m
volumes:
- shared-tmpfs:/tmp/paimon
#end
volumes:
shared-tmpfs:
driver: local
driver_opts:
type: "tmpfs"
device: "tmpfs"
```
The Docker Compose environment consists of the following containers:
- **Fluss Cluster:** a Fluss `CoordinatorServer`, a Fluss `TabletServer` and a `ZooKeeper` server.
It uses SASL/PLAIN for user authentication and defines three users: `admin`, `developer`, and `consumer`. The `admin` user is a `super.users` who has full administrative privileges on the Fluss cluster.
- **Flink Cluster**: a Flink `JobManager` and a Flink `TaskManager` container to execute queries.
3. To start all containers, run:
```shell
docker compose up -d
```
This command automatically starts all the containers defined in the Docker Compose configuration in detached mode.
Run
```shell
docker container ls -a
```
to check whether all containers are running properly.
You can also visit http://localhost:8083/ to see if Flink is running normally.
### Enter into SQL-Client
First, use the following command to enter the Flink SQL CLI Container:
```shell
docker compose exec jobmanager bin/sql-client.sh
```
### Create Catalogs for Each User
Create separate catalogs for each user:
```sql title="Flink SQL"
CREATE CATALOG admin_catalog WITH (
'type' = 'fluss',
'bootstrap.servers' = 'coordinator-server:9123',
'client.security.protocol' = 'SASL',
'client.security.sasl.mechanism' = 'PLAIN',
'client.security.sasl.username' = 'admin',
'client.security.sasl.password' = 'admin-pass'
);
```
```sql title="Flink SQL"
CREATE CATALOG developer_catalog WITH (
'type' = 'fluss',
'bootstrap.servers' = 'coordinator-server:9123',
'client.security.protocol' = 'SASL',
'client.security.sasl.mechanism' = 'PLAIN',
'client.security.sasl.username' = 'developer',
'client.security.sasl.password' = 'developer-pass'
);
```
```sql title="Flink SQL"
CREATE CATALOG consumer_catalog WITH (
'type' = 'fluss',
'bootstrap.servers' = 'coordinator-server:9123',
'client.security.protocol' = 'SASL',
'client.security.sasl.mechanism' = 'PLAIN',
'client.security.sasl.username' = 'consumer',
'client.security.sasl.password' = 'consumer-pass'
);
```
### Add ACLs for Users
As the `admin` user, add ACLs to grant permissions:
Allow `developer` user to read and write data:
```sql
CALL admin_catalog.sys.add_acl(
resource => 'cluster',
permission => 'ALLOW',
principal => 'User:developer',
operation => 'WRITE'
);
CALL admin_catalog.sys.add_acl(
resource => 'cluster',
permission => 'ALLOW',
principal => 'User:developer',
operation => 'READ'
);
```
Allow `consumer` user to read data:
```sql
CALL admin_catalog.sys.add_acl(
resource => 'cluster',
permission => 'ALLOW',
principal => 'User:consumer',
operation => 'READ'
);
```
Lookup the ACLs:
```sql
CALL admin_catalog.sys.list_acl(
resource => 'cluster'
);
```
Output will show like:
```text
+---------------------------------------------------------------------------------------------+
| result |
+---------------------------------------------------------------------------------------------+
| resource="cluster";permission="ALLOW";principal="User:developer";operation="READ";host="*" |
| resource="cluster";permission="ALLOW";principal="User:developer";operation="WRITE";host="*" |
| resource="cluster";permission="ALLOW";principal="User:consumer";operation="READ";host="*" |
+---------------------------------------------------------------------------------------------+
3 rows in set
```
### Create Tables Using Different Users
Only the `admin` user can create tables:
```sql
-- switch to admin user context
USE CATALOG admin_catalog;
-- create table using admin credentials
CREATE TABLE fluss_order (
`order_key` INT NOT NULL,
`total_price` DECIMAL(15, 2),
PRIMARY KEY (`order_key`) NOT ENFORCED
);
```
**Output:**
```text
[INFO] Execute statement succeeded.
```
The `developer` user cannot create tables:
```sql
-- switch to developer user context
USE CATALOG developer_catalog;
-- create table using developer credentials
CREATE TABLE fluss_order1(
`order_key` INT NOT NULL,
`total_price` DECIMAL(15, 2),
PRIMARY KEY (`order_key`) NOT ENFORCED
);
```
**Output:**
```text
[ERROR] Could not execute SQL statement. Reason:
org.apache.fluss.exception.AuthorizationException: Principal FlussPrincipal{name='developer', type='User'} have no authorization to operate CREATE on resource Resource{type=DATABASE, name='fluss'}
```
The `consumer` user also cannot create tables:
```sql
-- switch to consumer user context
USE CATALOG consumer_catalog;
-- create table using consumer credentials
CREATE TABLE fluss_order2(
`order_key` INT NOT NULL,
`total_price` DECIMAL(15, 2),
PRIMARY KEY (`order_key`) NOT ENFORCED
);
```
**Output:**
```text
[ERROR] Could not execute SQL statement. Reason:
org.apache.fluss.exception.AuthorizationException: Principal FlussPrincipal{name='consumer', type='User'} have no authorization to operate CREATE on resource Resource{type=DATABASE, name='fluss'}
```
### Write Data
Write data using the `developer` user:
```sql
-- switch to developer user context
USE CATALOG developer_catalog;
-- write data using developer credentials
INSERT INTO fluss_order VALUES (1, 1.0);
```
The job should succeed as shown in the Flink UI.
Attempting to write data using the `consumer` user will fail in the Flink UI:
```sql
-- switch to consumer user context
USE CATALOG consumer_catalog;
-- write data using consumer credentials
INSERT INTO fluss_order VALUES (1, 1.0);
```
**Output:**
```text
Caused by: java.util.concurrent.CompletionException: org.apache.fluss.exception.AuthorizationException: No WRITE permission among all the tables: [fluss.fluss_order]
```
### Read Data
Read data using the `consumer` user:
```sql
SET 'execution.runtime-mode' = 'batch';
-- use tableau result mode
SET 'sql-client.execution.result-mode' = 'tableau';
-- switch to consumer user context
USE CATALOG consumer_catalog;
-- read data using consumer credentials
SELECT * FROM `consumer_catalog`.`fluss`.`fluss_order` LIMIT 10;
```
**Output:**
```text
+-----------+-------------+
| order_key | total_price |
+-----------+-------------+
| 1 | 1.00 |
+-----------+-------------+
1 row in set (5.27 seconds)
```
Attempting to read data using the `developer` user also get the same result:
```sql
SET 'execution.runtime-mode' = 'batch';
-- use tableau result mode
SET 'sql-client.execution.result-mode' = 'tableau';
-- switch to developer user context
USE CATALOG developer_catalog;
-- read data using developer credentials
SELECT * FROM `developer_catalog`.`fluss`.`fluss_order` LIMIT 10;
```
## Example 2: Multi-Tenant Isolation in a Fluss Cluster
This example shows how to enable multi-tenant isolation in a Fluss cluster.
We'll demonstrate two departments — `marketing` and `finance` — each with its own dedicated database. The cluster includes the following users:
* `admin`: A superuser with full access.
* `marketing`: A user who can only access the `marketing_db` database.
* `finance`: A user who can only access the `finance_db` database.
### Environment Setup
All the steps are same as Example 1, but update the JAAS configuration to include the new users:
```yaml
services:
#begin Fluss cluster
coordinator-server:
image: apache/fluss:$FLUSS_DOCKER_VERSION$
command: coordinatorServer
depends_on:
- zookeeper
environment:
- |
FLUSS_PROPERTIES=
zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://coordinator-server:0, CLIENT://coordinator-server:9123
internal.listener.name: INTERNAL
remote.data.dir: /tmp/fluss/remote-data
# security properties
security.protocol.map: CLIENT:SASL, INTERNAL:PLAINTEXT
security.sasl.enabled.mechanisms: PLAIN
security.sasl.plain.jaas.config: org.apache.fluss.security.auth.sasl.plain.PlainLoginModule required user_admin="admin-pass" user_marketing="marketing-pass" user_finance="finance-pass";
authorizer.enabled: true
super.users: User:admin
tablet-server:
image: apache/fluss:$FLUSS_DOCKER_VERSION$
command: tabletServer
depends_on:
- coordinator-server
environment:
- |
FLUSS_PROPERTIES=
zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://tablet-server:0, CLIENT://tablet-server:9123
internal.listener.name: INTERNAL
tablet-server.id: 0
kv.snapshot.interval: 0s
data.dir: /tmp/fluss/data
remote.data.dir: /tmp/fluss/remote-data
# security properties
security.protocol.map: CLIENT:SASL, INTERNAL:PLAINTEXT
security.sasl.enabled.mechanisms: PLAIN
security.sasl.plain.jaas.config: org.apache.fluss.security.auth.sasl.plain.PlainLoginModule required user_admin="admin-pass" user_marketing="marketing-pass" user_finance="finance-pass";
authorizer.enabled: true
super.users: User:admin
volumes:
- shared-tmpfs:/tmp/fluss
zookeeper:
restart: always
image: zookeeper:3.9.2
#end
#begin Flink cluster
jobmanager:
image: apache/fluss-quickstart-flink:$FLUSS_QUICKSTART_FLINK_DOCKER_VERSION$
ports:
- "8083:8081"
command: jobmanager
environment:
- |
FLINK_PROPERTIES=
jobmanager.rpc.address: jobmanager
volumes:
- shared-tmpfs:/tmp/paimon
taskmanager:
image: apache/fluss-quickstart-flink:$FLUSS_QUICKSTART_FLINK_DOCKER_VERSION$
depends_on:
- jobmanager
command: taskmanager
environment:
- |
FLINK_PROPERTIES=
jobmanager.rpc.address: jobmanager
taskmanager.numberOfTaskSlots: 10
taskmanager.memory.process.size: 2048m
taskmanager.memory.framework.off-heap.size: 256m
volumes:
- shared-tmpfs:/tmp/paimon
#end
volumes:
shared-tmpfs:
driver: local
driver_opts:
type: "tmpfs"
device: "tmpfs"
```
### Enter into SQL-Client
First, use the following command to enter the Flink SQL CLI Container:
```shell
docker compose exec jobmanager bin/sql-client.sh
```
### Create Catalogs for Each User
Create separate catalogs for the `admin`, `marketing`, and `finance` users:
```sql title="Flink SQL"
CREATE CATALOG admin_catalog WITH (
'type' = 'fluss',
'bootstrap.servers' = 'coordinator-server:9123',
'client.security.protocol' = 'SASL',
'client.security.sasl.mechanism' = 'PLAIN',
'client.security.sasl.username' = 'admin',
'client.security.sasl.password' = 'admin-pass'
);
```
```sql title="Flink SQL"
CREATE CATALOG marketing_catalog WITH (
'type' = 'fluss',
'bootstrap.servers' = 'coordinator-server:9123',
'client.security.protocol' = 'SASL',
'client.security.sasl.mechanism' = 'PLAIN',
'client.security.sasl.username' = 'marketing',
'client.security.sasl.password' = 'marketing-pass'
);
```
```sql title="Flink SQL"
CREATE CATALOG finance_catalog WITH (
'type' = 'fluss',
'bootstrap.servers' = 'coordinator-server:9123',
'client.security.protocol' = 'SASL',
'client.security.sasl.mechanism' = 'PLAIN',
'client.security.sasl.username' = 'finance',
'client.security.sasl.password' = 'finance-pass'
);
```
### Create Databases and Set ACLs
As the `admin` user, create two databases and assign appropriate ACLs:
```sql title="Flink SQL"
CREATE DATABASE `admin_catalog`.`marketing_db`;
CALL admin_catalog.sys.add_acl(
resource => 'cluster.marketing_db',
permission => 'ALLOW',
principal => 'User:marketing',
operation => 'ALL'
);
CREATE DATABASE `admin_catalog`.`finance_db`;
CALL admin_catalog.sys.add_acl(
resource => 'cluster.finance_db',
permission => 'ALLOW',
principal => 'User:finance',
operation => 'ALL'
);
```
Lookup the ACLs:
```sql
CALL admin_catalog.sys.list_acl(
resource => 'ANY'
);
```
Output will show like:
```text
+--------------------------------------------------------------------------------------------------------+
| result |
+--------------------------------------------------------------------------------------------------------+
| resource="cluster.marketing_db";permission="ALLOW";principal="User:marketing";operation="ALL";host="*" |
| resource="cluster.finance_db";permission="ALLOW";principal="User:finance";operation="ALL";host="*" |
+--------------------------------------------------------------------------------------------------------+
2 rows in set
```
### Granularity of Database Visibility
The `marketing` user can only see the `marketing_db` database
```sql title="Flink SQL"
-- switch to marketing user context
USE CATALOG marketing_catalog;
-- show databases using marketing user credentials
SHOW DATABASES;
```
**Output:**
```text
+---------------+
| database name |
+---------------+
| marketing_db |
+---------------+
1 row in set
```
The `finance` user can only see the `finance_db` database:
```sql title="Flink SQL"
-- switch to finance user context
USE CATALOG finance_catalog;
-- show databases using finance user credentials
SHOW DATABASES;
```
**Output:**
```text
+---------------+
| database name |
+---------------+
| finance_db |
+---------------+
1 row in set
```
The `marketing` user can operate on their own database:
```sql title="Flink SQL"
-- switch to marketing user context
USE CATALOG marketing_catalog;
-- create table using marketing user credentials
CREATE TABLE `marketing_db`.`order` (
`order_key` INT NOT NULL,
`total_price` DECIMAL(15, 2),
PRIMARY KEY (`order_key`) NOT ENFORCED
);
```
**Output:**
```text
[INFO] Execute statement succeeded.
```
The `finance` user cannot access the `marketing` database:
```sql title="Flink SQL"
-- switch to finance user context
USE CATALOG finance_catalog;
-- create table using finance user credentials
CREATE TABLE `marketing_db`.`order` (
`order_key` INT NOT NULL,
`total_price` DECIMAL(15, 2),
PRIMARY KEY (`order_key`) NOT ENFORCED
);
```
**Output:**
```text
[ERROR] Could not execute SQL statement. Reason:
org.apache.fluss.exception.AuthorizationException: Principal FlussPrincipal{name='finance', type='User'} have no authorization to operate CREATE on resource Resource{type=DATABASE, name='marketing_db'}
```