| --- |
| id: deploy-kubernetes |
| title: Kubernetes Deployment |
| sidebar_label: Kubernetes Deployment |
| --- |
| |
| ## Prerequisites |
| Requires Helm (https://helm.sh/) and an actively running Kubernetes cluster. |
| |
| ## Usage |
| We provide helm chart options to get you going in the `installer/k8s`folder. |
| |
| **Starting** the default helm chart option is as easy as simply running the following command from the root of this folder: |
| > **NOTE**: Starting might take a while since we also initially pull all Docker images from Dockerhub. |
| |
| ```bash |
| helm install streampipes ./ |
| ``` |
| After a while, all containers should successfully started, indicated by the `Running` status. |
| |
| The `values.yaml` file contains several configuration options to customize your StreamPipes installation. See the section below for all configuration options. |
| |
| ## Ingress |
| |
| The helm chart provides several options to configure an Ingress or to define an Ingressroute that directly integrates with Traefik. |
| |
| ## Dynamic Volume Provisioning |
| |
| You can override the `storageClassName` variable to configure StreamPipes for dynamic volume provisioning. |
| |
| ## Parameters |
| |
| Here is an overview of the supported parameters to configure StreamPipes. |
| |
| ### Common parameters |
| |
| | Parameter Name | Description | Value | |
| |--------------------------------------------------|---------------------------------------------------------|-----------------------------------------| |
| | deployment | Deployment type (lite or full) | lite | |
| | preferredBroker | Preferred broker for deployment | "nats" | |
| | monitoringSystem | Enable monitoring system (true/false) | false | |
| | pullPolicy | Image pull policy | "Always" | |
| | restartPolicy | Restart policy for the container | Always | |
| | persistentVolumeReclaimPolicy | Reclaim policy for persistent volumes | "Delete" | |
| | persistentVolumeAccessModes | Access mode for persistent volumes | "ReadWriteOnce" | |
| | initialDelaySeconds | Initial delay for liveness and readiness probes | 60 | |
| | periodSeconds | Interval between liveness and readiness probes | 30 | |
| | failureThreshold | Number of consecutive failures for readiness probes | 30 | |
| | hostPath | Host path for the application | "" | |
| |
| ### StreamPipes common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|---------------------------------------------------------|------------------------------------------| |
| | streampipes.version | StreamPipes version | "0.93.0-SNAPSHOT" | |
| | streampipes.registry | StreamPipes registry URL | "apachestreampipes" | |
| | streampipes.auth.secretName | The secret name for storing secrets | "sp-secrets" | |
| | streampipes.auth.users.admin.user | The initial admin user | "admin@streampipes.apache.org" | |
| | streampipes.auth.users.admin.password | The initial admin password (leave empty for autogen) | "admin" | |
| | streampipes.auth.users.service.user | The initial service account user | "sp-service-client" | |
| | streampipes.auth.users.service.secret | The initial service account secret | empty (auto-generated) | |
| | streampipes.auth.encryption.passcode | Passcode for value encryption | empty (auto-generated) | |
| | streampipes.core.appName | StreamPipes backend application name | "backend" | |
| | streampipes.core.port | StreamPipes backend port | 8030 | |
| | streampipes.core.persistence.storageClassName | Storage class name for backend PVs | "hostpath" | |
| | streampipes.core.persistence.storageSize | Size of the backend PV | "1Gi" | |
| | streampipes.core.persistence.claimName | Name of the backend PersistentVolumeClaim | "backend-pvc" | |
| | streampipes.core.persistence.pvName | Name of the backend PersistentVolume | "backend-pv" | |
| | streampipes.core.service.name | Name of the backend service | "backend" | |
| | streampipes.core.service.port | TargetPort of the StreamPipes backend service | 8030 | |
| | streampipes.ui.appName | StreamPipes UI application name | "ui" | |
| | streampipes.ui.resolverActive | Flag for enabling DNS resolver for Nginx proxy | true | |
| | streampipes.ui.port | StreamPipes UI port | 8088 | |
| | streampipes.ui.resolver | DNS resolver for Nginx proxy | "kube-dns.kube-system.svc.cluster.local" | |
| | streampipes.ui.service.name | Name of the UI service | "ui" | |
| | streampipes.ui.service.type | Type of the UI service | "ClusterIP" | |
| | streampipes.ui.service.nodePort | Node port for the UI service | 8088 | |
| | streampipes.ui.service.port | TargetPort of the StreamPipes UI service | 8088 | |
| | streampipes.ingress.active | Flag for enabling Ingress for StreamPipes | false | |
| | streampipes.ingress.annotations | Annotations for Ingress | {} | |
| | streampipes.ingress.host | Hostname for Ingress | "" | |
| | streampipes.ingressroute.active | Flag for enabling IngressRoute for StreamPipes | true | |
| | streampipes.ingressroute.annotations | Annotations for IngressRoute | {} | |
| | streampipes.ingressroute.entryPoints | Entry points for IngressRoute | ["web", "websecure"] | |
| | streampipes.ingressroute.host | Hostname for IngressRoute | "" | |
| | streampipes.ingressroute.certResolverActive | Flag for enabling certificate resolver for IngressRoute | true | |
| | streampipes.ingressroute.certResolver | Certificate resolver for IngressRoute | "" | |
| |
| |
| ### Extensions common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|---------------------------------------------------------|------------------------------------------| |
| | extensions.iiot.appName | IIoT extensions application name | extensions-all-iiot | |
| | extensions.iiot.port | Port for the IIoT extensions application | 8090 | |
| | extensions.iiot.service.name | Name of the IIoT extensions service | extensions-all-iiot | |
| | extensions.iiot.service.port | TargetPort of the IIoT extensions service | 8090 | |
| |
| |
| ### External common parameters |
| |
| #### Consul common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | external.consul.appName | Consul application name | "consul" | |
| | external.consul.version | Consul version | 1.14.3 | |
| | external.consul.webPort | Port number for the Consul web interface | 8500 | |
| | external.consul.dnsPort | Port number for the Consul DNS interface | 8600 | |
| | external.consul.persistence.storageClassName | Storage class name for Consul PVs | "hostpath" | |
| | external.consul.persistence.storageSize | Size of the Consul PV | "1Gi" | |
| | external.consul.persistence.claimName | Name of the Consul PersistentVolumeClaim | "consul-pvc" | |
| | external.consul.persistence.pvName | Name of the Consul PersistentVolume | "consul-pv" | |
| | external.consul.service.name | Name of the Consul service | "consul" | |
| | external.consul.service.webPort | TargetPort of the Consul service for web interface | 8500 | |
| | external.consul.service.dnsPort | TargetPort of the Consul service for DNS interface | 8600 | |
| |
| #### Couchdb common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | external.couchdb.appName | CouchDB application name | "couchdb" | |
| | external.couchdb.version | CouchDB version | 3.3.1 | |
| | external.couchdb.user | CouchDB admin username | "admin" | |
| | external.couchdb.password | CouchDB admin password | empty (auto-generated) | |
| | external.couchdb.port | Port for the CouchDB service | 5984 | |
| | external.couchdb.service.name | Name of the CouchDB service | "couchdb" | |
| | external.couchdb.service.port | TargetPort of the CouchDB service | 5984 | |
| | external.couchdb.persistence.storageClassName | Storage class name for CouchDB PVs | "hostpath" | |
| | external.couchdb.persistence.storageSize | Size of the CouchDB PV | "1Gi" | |
| | external.couchdb.persistence.claimName | Name of the CouchDB PersistentVolumeClaim | "couchdb-pvc" | |
| | external.couchdb.persistence.pvName | Name of the CouchDB PersistentVolume | "couchdb-pv" | |
| |
| #### Influxdb common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | external.influxdb.appName | InfluxDB application name | "influxdb" | |
| | external.influxdb.version | InfluxDB version | 2.6 | |
| | external.influxdb.username | InfluxDB admin username | "admin" | |
| | external.influxdb.password | InfluxDB admin password | empty (auto-generated) | |
| | external.influxdb.adminToken | InfluxDB admin token | empty (auto-generated) | |
| | external.influxdb.initOrg | InfluxDB initial organization | "sp" | |
| | external.influxdb.initBucket | InfluxDB initial bucket | "sp" | |
| | external.influxdb.initMode | InfluxDB initialization mode | "setup" | |
| | external.influxdb.apiPort | Port number for the InfluxDB service (API) | 8083 | |
| | external.influxdb.httpPort | Port number for the InfluxDB service (HTTP) | 8086 | |
| | external.influxdb.grpcPort | Port number for the InfluxDB service (gRPC) | 8090 | |
| | external.influxdb.service.name | Name of the InfluxDB service | "influxdb" | |
| | external.influxdb.service.apiPort | TargetPort of the InfluxDB service for API | 8083 | |
| | external.influxdb.service.httpPort | TargetPort of the InfluxDB service for HTTP | 8086 | |
| | external.influxdb.service.grpcPort | TargetPort of the InfluxDB service for gRPC | 8090 | |
| | external.influxdb.persistence.storageClassName | Storage class name for InfluxDB PVs | "hostpath" | |
| | external.influxdb.persistence.storageSize | Size of the InfluxDB PV | "1Gi" | |
| | external.influxdb.persistence.storageSizeV1 | Size of the InfluxDB PV for v1 databases | "1Gi" | |
| | external.influxdb.persistence.claimName | Name of the InfluxDBv2 PersistentVolumeClaim | "influxdb2-pvc" | |
| | external.influxdb.persistence.claimNameV1 | Name of the InfluxDBv1 PersistentVolumeClaim | "influxdb-pvc" | |
| | external.influxdb.persistence.pvName | Name of the InfluxDBv2 PersistentVolume | "influxdb2-pv" | |
| | external.influxdb.persistence.pvNameV1 | Name of the InfluxDBv1 PersistentVolume | "influxdb-pv" | |
| |
| |
| #### Nats common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | external.nats.appName | NATS application name | "nats" | |
| | external.nats.port | Port for the NATS service | 4222 | |
| | external.nats.version | NATS version | | |
| | external.nats.service.type | Type of the NATS service | "NodePort" | |
| | external.nats.service.externalTrafficPolicy | External traffic policy for the NATS service | "Local" | |
| | external.nats.service.name | Name of the NATS service | "nats" | |
| | external.nats.service.port | TargetPort of the NATS service | 4222 | |
| |
| |
| #### Kafka common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | external.kafka.appName | Kafka application name | "kafka" | |
| | external.kafka.version | Kafka version | 2.2.0 | |
| | external.kafka.port | Port for the Kafka service | 9092 | |
| | external.kafka.external.hostname | Name which will be advertised to external clients. Clients which use (default) port 9094 | "localhost" |
| | external.kafka.service.name | Name of the Kafka service | "kafka" | |
| | external.kafka.service.port | TargetPort of the Kafka service | 9092 | |
| | external.kafka.service.portOutside | Port for Kafka client outside of the cluster | 9094 | |
| | external.kafka.persistence.storageClassName | Storage class name for Kafka PVs | "hostpath" | |
| | external.kafka.persistence.storageSize | Size of the Kafka PV | "1Gi" | |
| | external.kafka.persistence.claimName | Name of the Kafka PersistentVolumeClaim | "kafka-pvc" | |
| | external.kafka.persistence.pvName | Name of the Kafka PersistentVolume | "kafka-pv" | |
| | |
| |
| #### Zookeeper common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | external.zookeeper.appName | ZooKeeper application name | "zookeeper" | |
| | external.zookeeper.version | ZooKeeper version | 3.4.13 | |
| | external.zookeeper.port | Port for the ZooKeeper service | 2181 | |
| | external.zookeeper.service.name | Name of the ZooKeeper service | "zookeeper" | |
| | external.zookeeper.service.port | TargetPort of the ZooKeeper service | 2181 | |
| | external.zookeeper.persistence.storageClassName | Storage class name for ZooKeeper PVs | "hostpath" | |
| | external.zookeeper.persistence.storageSize | Size of the ZooKeeper PV | "1Gi" | |
| | external.zookeeper.persistence.claimName | Name of the ZooKeeper PersistentVolumeClaim | "zookeeper-pvc" | |
| | external.zookeeper.persistence.pvName | Name of the ZooKeeper PersistentVolume | "zookeeper-pv" | |
| |
| |
| #### Pulsar common parameters |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | external.pulsar.appName | pulsar application name | "pulsar" | |
| | external.pulsar.version | pulsar version | 3.0.0 | |
| | external.pulsar.port | Port for the pulsar service | 6650 | |
| | external.pulsar.service.name | Name of the pulsar service | "pulsar" | |
| | external.pulsar.service.port | TargetPort of the pulsar service | 6650 | |
| | external.pulsar.persistence.storageClassName | Storage class name for pulsar PVs | "hostpath" | |
| | external.pulsar.persistence.storageSize | Size of the pulsar PV | "1Gi" | |
| | external.pulsar.persistence.claimName | Name of the pulsar PersistentVolumeClaim | "pulsar-pvc" | |
| | external.pulsar.persistence.pvName | Name of the pulsar PersistentVolume | "pulsar-pv" | |
| |
| ### Monitoring common parameters |
| |
| #### Monitoring - Prometheus |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | prometheus.appName | Prometheus application name | "prometheus" | |
| | prometheus.version | Prometheus version | 2.45.0 | |
| | prometheus.port | Prometheus port | 9090 | |
| | prometheus.service.name | Prometheus service name | "prometheus" | |
| | prometheus.service.port | Prometheus service port | 9090 | |
| | prometheus.persistence.storageClassName | Prometheus storage class name | "hostpath" | |
| | prometheus.persistence.storageSize | Prometheus storage size | "2Gi" | |
| | prometheus.persistence.claimName | Prometheus PVC claim name | "prometheus-pvc" | |
| | prometheus.persistence.pvName | Prometheus PV name | "prometheus-pv" | |
| | prometheus.persistence.tokenStorageSize | Prometheus token storage size | "16Ki" | |
| | prometheus.config.scrapeInterval | Prometheus scrape interval | 10s | |
| | prometheus.config.evaluationInterval | Prometheus evaluation interval | 15s | |
| | prometheus.config.backendJobName | Prometheus backend job name | "backend" | |
| | prometheus.config.extensionsName | Prometheus extensions job name | "extensions-all-iiot" | |
| | prometheus.config.tokenFileName | Prometheus token file name | "token" | |
| | prometheus.config.tokenFileDir | Prometheus token file directory | "/opt/data" |
| |
| #### Monitoring - Grafana |
| |
| | Parameter Name | Description | Value | |
| |-------------------------------------------------|----------------------------------------------------------|------------------------------------------| |
| | grafana.appName | Grafana application name | "grafana" | |
| | grafana.version | Grafana version | 10.1.2 | |
| | grafana.port | Grafana port | 3000 | |
| | grafana.service.name | Grafana service name | "grafana" | |
| | grafana.service.port | Grafana service port | 3000 | |
| | grafana.persistence.storageClassName | Grafana storage class name | "hostpath" | |
| | grafana.persistence.storageSize | Grafana storage size | "1Gi" | |
| | grafana.persistence.claimName | Grafana PVC claim name | "grafana-pvc" | |
| | grafana.persistence.pvName | Grafana PV name | "grafana-pv" | |
| |
| |
| ## Auto-generation of parameters. |
| |
| The helm chart includes a `secrets.yaml` file which auto-generates several settings as follows: |
| |
| ```yaml |
| |
| apiVersion: v1 |
| kind: Secret |
| metadata: |
| name: sp-secrets |
| namespace: {{ .Release.Namespace | quote }} |
| type: Opaque |
| data: |
| sp-initial-admin-password: {{ ternary (randAlphaNum 10) .Values.streampipes.auth.users.admin.password (empty .Values.streampipes.auth.users.admin.password) | b64enc | quote }} |
| sp-initial-client-secret: {{ ternary (randAlphaNum 35) .Values.streampipes.auth.users.service.secret (empty .Values.streampipes.auth.users.service.secret) | b64enc | quote }} |
| sp-encryption-passcode: {{ ternary (randAlphaNum 20) .Values.streampipes.auth.encryption.passcode (empty .Values.streampipes.auth.encryption.passcode) | b64enc | quote }} |
| sp-couchdb-password: {{ ternary (randAlphaNum 20) .Values.external.couchdb.password (empty .Values.external.couchdb.password) | b64enc | quote }} |
| sp-ts-storage-password: {{ ternary (randAlphaNum 20) .Values.external.influxdb.password (empty .Values.external.influxdb.password) | b64enc | quote }} |
| sp-ts-storage-token: {{ ternary (randAlphaNum 20) .Values.external.influxdb.adminToken (empty .Values.external.influxdb.adminToken) | b64enc | quote }} |
| |
| ``` |
| |
| |
| ## Deleting the current helm chart deployment: |
| ```bash |
| helm uninstall streampipes |
| ``` |
