server/apps/distributed-app/docs/modules/ROOT/pages/run/run-kubernetes.adoc - james-project - Git at Google

 = Distributed James Server &mdash; Run with Kubernetes
 :navtitle: Run with Kubernetes

 == Deploy Apache James on Kubernetes with Helm chart.

 This chart bootstraps a James mail server on a link:http://kubernetes.io[Kubernetes]
 cluster using the link:https://helm.sh[Helm] package manager.

 == Before Starting


 Before running the Helm chart, James should have particularly accessed to the following external products:

 Apache Cassandra::
 - You can deploy Cassandra on Kubernetes with a link:https://github.com/bitnami/charts/tree/master/bitnami/cassandra[Cassandra chart] for testing purpose. However we would recommend running Cassandra on VirtualMachine for better stablilty.
 OpenSearch::
 - OpenSearch community maintains link:https://github.com/opensearch-project/helm-charts[Helm charts]
 RabbitMQ::
 - link:https://github.com/rabbitmq/cluster-operator[RabbitMQ Cluster Operator] is a custom Kubernetes resource designed for the lifecycle (creation, upgrade, graceful shutdown) of a RabbitMQ cluster. Using Quickstart setup is enough for James deployment.


 == Configuration

 === Helm Configuration

 The James helm package take parameters that will configure and determine the behavior of James.
 There are 2 configuration files:

 * One contains global, non-confidential, configuration: `values.yaml`
 * The other one contains secrets: `secrets.yaml`. Values are encrypted with a key stored generated by link:https://github.com/mozilla/sops[sops].

 You need to pass these 2 files as parameters to the James Helm package.
 To verify the coherence and avoid errors, the packages can validate the input.

 This package contains a default `values.yaml` file, but that you should override with proper configuration for your deployment (`conf.yaml`).
 The file `secrets.sample.yaml` will show you the possible values of the secrets file. To generate the `secrets.yaml` file,
 you will need a plugin for Helm called link:https://github.com/jkroepke/helm-secrets[helm-secrets].

 Usually, you will save those configurations in different repositories, per deployment.

 Have a look at `values.md` and `secrets.md` in our Chart link:https://github.com/apache/james-project/tree/master/server/apps/distributed-app/helm-chart/james/doc/[documents] to have more information on the configuration James is using.

 You can find our Helm chart for James in link:https://github.com/apache/james-project/tree/master/server/apps/distributed-app/helm-chart[here]

 === James Configuration

 The configuration files proper to James are stored in the Chart `james/configs` folder. They are being stored in a `ConfigMap` and then being mounted in the James pod at /root/conf path.

 If you need to change a conf in James, this is where you need to look at.

 For now all configuration files are the same for every deployment, except for `mailetcontainer.xml` that has been excluded (see section below).

 You can get the template `mailetcontainer.xml` in our link:https://github.com/apache/james-project/blob/master/server/apps/distributed-app/sample-configuration/[sample-configuration] folder.

 [NOTE]
 ===============================
 The `mailetcontainer.xml` is the configuration file in James responsible of mail processing and delivery. It can differ greatly from one deployment to another. For that reason, we have excluded it from `james/configs` folder for the moment. You need to draft your own version of `mailetcontainer.xml` and add it to `james/configs` before start deploying Helm chart, as we can't actually mount a separate file into the same mounting point as the one sued by the `ConfigMap` on `/root/conf` path.

 However, the next release of Helm should allow to do something about that (hopefully). The helm package will be updated then.

 ===============================

 == Deploy James Helm chart
 === Verify the configuration

 We recommend to verify the configuration against the James Helm packages before deploying it. Usually it is best to have to position yourself in the repository having your deployment's values and secrets, and linking this helm package to it, like this:

     $ helm lint -f james/conf.yaml -f james/secrets.yaml /path/to/helm/james

 You can also generate the whole templates with values (can be useful for debugging sometimes), by typing:

     $ helm secrets templates -f james/conf.yaml -f james/secrets.yaml james /path/to/helm/james

 From there, you should be able to interact with the Kubernetes cluster with `kubectl` command.

 A few useful commands:
 ----
 kubectl get namespace # show namespaces on your cluster

 kubectl get pod -n mail-server # list pods running in mail-server namespace

 kubectl config set-context --current --namespace=mail-server # set the namespace mail-server as default one for next commands

 kubectl get configmaps # list configmaps depoyed on the namespace

 kubectl get secrets # list secrets deployed on the namespace

 kubectl get service # list services deployed in the namespace

 kubectl get nodes -o wide # list nodes related to the namespace

 kubectl describe pod my-pod # get a detailed description of my-pod (similar for other objects)

 kubectl scale --replicas=4 deployment james # scale up (or down) james to 4 pods

 kubectl rollout restart deployment james # force a rollout restart of your pods related to james

 kubectl logs my-pod -f --tail 100 # show and follow logs of a pod from the last 100 lines

 kubectl logs -f -l app=james --tail 100 # show and follow the combined logs of all james pods from the last 100 lines

 kubectl port-forward pod/my-pod 3000:3000 # allow you to access and interact with internal port 3000 of my-pod from localhost
 ----

 Usually you have a namespace per deployment. Always target your deployments on the corresponding namespace, by either adding the flag -n mail-server on your kubectl and helm commands, or by setting your namespace by default (see in the above commands).

 === Deploy James Chart

 Before doing a real deployment, you need to check that you have at least all necessary values for James to start and run smoothly on your environment. You can check the `checklist.md` in our document folder for this.

 Run the command when you think you are good to go:

     $ helm secrets install james james/ -f james/values.yaml -f james/secrets.yaml

 image::apache-james-lens.png[ Distributed Apache James in Kubernetes ]
 You can also wait for the deployment to be complete by adding a --wait flag, that is based on the readiness probe located inside the James pod.

 However sometimes things might not always go well at the start of James. Usually it is good to look at the logs. If it goes without issues until you start to see some successful health check logs, then you are likely good.

 === Upgrade Chart version

 If you want to deploy a new version of a installed package, use helm secrets upgrade:

     $ helm secrets upgrade -f james/conf.yaml -f james/secrets.yaml james /path/to/helm/james

 What Helm does is that it compares all the generated YAML files with those on the server.
 And it updates only those that changed. There are however some limitations:

 * pods in errors are not recreated during an upgrade. It allows to check the logs first. However, if you delete the pod by hand, it will be recreated from the last definition.
 * Some artifacts, such as jobs and stateful sets, have immutable properties. So, some upgrades may fail. Deleting by hand the element is sometimes a solution before doing the upgrade.

 TIP: Install link:https://github.com/databus23/helm-diff[helm-diff] plugin to display the changed configurations before run upgrade.

     $ helm secrets diff upgrade -f james/conf.yaml -f james/secrets.yaml james /path/to/helm/james

 === Rollback to previous Chart version
 Show chart revision history:

     $  helm history james -n <name space>

 Rollback to a previous version:

     $  helm rollbaack james <revision>

 === Uninstalling
 To uninstall the helm package:

     $ helm uninstall james -n <name space>

 == References

 A summary of different resources that could be interesting to have in hand.

 === This package

 * xref:run/k8s-values.adoc[Values]
 * xref:run/k8s-secrets.adoc[Secrets]
 * xref:run/k8s-checklist.adoc[Check List]
 * xref:run/k8s-logsMetrics.adoc[Logs and Metrics]

 ### James documentation

 * link:https://james.apache.org/[James Official Website]
 * link:https://github.com/apache/james-project[James GitHub Project]
 * xref:operate/webadmin.adoc[WebAdmin]
 * xref:operate/cli.adoc[James CLI]
 * link:https://james.apache.org/server/manage-guice-distributed-james.html[Manage Guice Distributed James]

 ### Kubernetes

 * link:https://kubernetes.io/docs/concepts/overview/components/[Kubernetes Components Overview]
 * link:https://kubernetes.io/docs/concepts/workloads/pods/[Pods]
 * link:https://kubernetes.io/docs/concepts/services-networking/service/[Services]
 * link:https://kubernetes.io/docs/concepts/services-networking/ingress/[Ingress]
 * link:https://kubernetes.io/docs/concepts/configuration/configmap/[Configmaps]
 * link:https://kubernetes.io/docs/concepts/configuration/secret/[Secrets]
 * link:https://kubernetes.io/docs/concepts/workloads/controllers/deployment/[Deployments]

 ### Helm

 * link:https://helm.sh/docs/topics/architecture/[Helm Architecture]
 * link:https://helm.sh/docs/topics/charts/[Charts]
 * link:https://helm.sh/docs/intro/using_helm/[Using Helm]
 * link:https://helm.sh/docs/chart_template_guide/getting_started/[Helm templates]
	= Distributed James Server — Run with Kubernetes
	:navtitle: Run with Kubernetes

	== Deploy Apache James on Kubernetes with Helm chart.

	This chart bootstraps a James mail server on a link:http://kubernetes.io[Kubernetes]
	cluster using the link:https://helm.sh[Helm] package manager.

	== Before Starting


	Before running the Helm chart, James should have particularly accessed to the following external products:

	Apache Cassandra::
	- You can deploy Cassandra on Kubernetes with a link:https://github.com/bitnami/charts/tree/master/bitnami/cassandra[Cassandra chart] for testing purpose. However we would recommend running Cassandra on VirtualMachine for better stablilty.
	OpenSearch::
	- OpenSearch community maintains link:https://github.com/opensearch-project/helm-charts[Helm charts]
	RabbitMQ::
	- link:https://github.com/rabbitmq/cluster-operator[RabbitMQ Cluster Operator] is a custom Kubernetes resource designed for the lifecycle (creation, upgrade, graceful shutdown) of a RabbitMQ cluster. Using Quickstart setup is enough for James deployment.


	== Configuration

	=== Helm Configuration

	The James helm package take parameters that will configure and determine the behavior of James.
	There are 2 configuration files:

	* One contains global, non-confidential, configuration: `values.yaml`
	* The other one contains secrets: `secrets.yaml`. Values are encrypted with a key stored generated by link:https://github.com/mozilla/sops[sops].

	You need to pass these 2 files as parameters to the James Helm package.
	To verify the coherence and avoid errors, the packages can validate the input.

	This package contains a default `values.yaml` file, but that you should override with proper configuration for your deployment (`conf.yaml`).
	The file `secrets.sample.yaml` will show you the possible values of the secrets file. To generate the `secrets.yaml` file,
	you will need a plugin for Helm called link:https://github.com/jkroepke/helm-secrets[helm-secrets].

	Usually, you will save those configurations in different repositories, per deployment.

	Have a look at `values.md` and `secrets.md` in our Chart link:https://github.com/apache/james-project/tree/master/server/apps/distributed-app/helm-chart/james/doc/[documents] to have more information on the configuration James is using.

	You can find our Helm chart for James in link:https://github.com/apache/james-project/tree/master/server/apps/distributed-app/helm-chart[here]

	=== James Configuration

	The configuration files proper to James are stored in the Chart `james/configs` folder. They are being stored in a `ConfigMap` and then being mounted in the James pod at /root/conf path.

	If you need to change a conf in James, this is where you need to look at.

	For now all configuration files are the same for every deployment, except for `mailetcontainer.xml` that has been excluded (see section below).

	You can get the template `mailetcontainer.xml` in our link:https://github.com/apache/james-project/blob/master/server/apps/distributed-app/sample-configuration/[sample-configuration] folder.

	[NOTE]
	===============================
	The `mailetcontainer.xml` is the configuration file in James responsible of mail processing and delivery. It can differ greatly from one deployment to another. For that reason, we have excluded it from `james/configs` folder for the moment. You need to draft your own version of `mailetcontainer.xml` and add it to `james/configs` before start deploying Helm chart, as we can't actually mount a separate file into the same mounting point as the one sued by the `ConfigMap` on `/root/conf` path.

	However, the next release of Helm should allow to do something about that (hopefully). The helm package will be updated then.

	===============================

	== Deploy James Helm chart
	=== Verify the configuration

	We recommend to verify the configuration against the James Helm packages before deploying it. Usually it is best to have to position yourself in the repository having your deployment's values and secrets, and linking this helm package to it, like this:

	$ helm lint -f james/conf.yaml -f james/secrets.yaml /path/to/helm/james

	You can also generate the whole templates with values (can be useful for debugging sometimes), by typing:

	$ helm secrets templates -f james/conf.yaml -f james/secrets.yaml james /path/to/helm/james

	From there, you should be able to interact with the Kubernetes cluster with `kubectl` command.

	A few useful commands:
	----
	kubectl get namespace # show namespaces on your cluster

	kubectl get pod -n mail-server # list pods running in mail-server namespace

	kubectl config set-context --current --namespace=mail-server # set the namespace mail-server as default one for next commands

	kubectl get configmaps # list configmaps depoyed on the namespace

	kubectl get secrets # list secrets deployed on the namespace

	kubectl get service # list services deployed in the namespace

	kubectl get nodes -o wide # list nodes related to the namespace

	kubectl describe pod my-pod # get a detailed description of my-pod (similar for other objects)

	kubectl scale --replicas=4 deployment james # scale up (or down) james to 4 pods

	kubectl rollout restart deployment james # force a rollout restart of your pods related to james

	kubectl logs my-pod -f --tail 100 # show and follow logs of a pod from the last 100 lines

	kubectl logs -f -l app=james --tail 100 # show and follow the combined logs of all james pods from the last 100 lines

	kubectl port-forward pod/my-pod 3000:3000 # allow you to access and interact with internal port 3000 of my-pod from localhost
	----

	Usually you have a namespace per deployment. Always target your deployments on the corresponding namespace, by either adding the flag -n mail-server on your kubectl and helm commands, or by setting your namespace by default (see in the above commands).

	=== Deploy James Chart

	Before doing a real deployment, you need to check that you have at least all necessary values for James to start and run smoothly on your environment. You can check the `checklist.md` in our document folder for this.

	Run the command when you think you are good to go:

	$ helm secrets install james james/ -f james/values.yaml -f james/secrets.yaml

	image::apache-james-lens.png[ Distributed Apache James in Kubernetes ]
	You can also wait for the deployment to be complete by adding a --wait flag, that is based on the readiness probe located inside the James pod.

	However sometimes things might not always go well at the start of James. Usually it is good to look at the logs. If it goes without issues until you start to see some successful health check logs, then you are likely good.

	=== Upgrade Chart version

	If you want to deploy a new version of a installed package, use helm secrets upgrade:

	$ helm secrets upgrade -f james/conf.yaml -f james/secrets.yaml james /path/to/helm/james

	What Helm does is that it compares all the generated YAML files with those on the server.
	And it updates only those that changed. There are however some limitations:

	* pods in errors are not recreated during an upgrade. It allows to check the logs first. However, if you delete the pod by hand, it will be recreated from the last definition.
	* Some artifacts, such as jobs and stateful sets, have immutable properties. So, some upgrades may fail. Deleting by hand the element is sometimes a solution before doing the upgrade.

	TIP: Install link:https://github.com/databus23/helm-diff[helm-diff] plugin to display the changed configurations before run upgrade.

	$ helm secrets diff upgrade -f james/conf.yaml -f james/secrets.yaml james /path/to/helm/james

	=== Rollback to previous Chart version
	Show chart revision history:

	$ helm history james -n <name space>

	Rollback to a previous version:

	$ helm rollbaack james <revision>

	=== Uninstalling
	To uninstall the helm package:

	$ helm uninstall james -n <name space>

	== References

	A summary of different resources that could be interesting to have in hand.

	=== This package

	* xref:run/k8s-values.adoc[Values]
	* xref:run/k8s-secrets.adoc[Secrets]
	* xref:run/k8s-checklist.adoc[Check List]
	* xref:run/k8s-logsMetrics.adoc[Logs and Metrics]

	### James documentation

	* link:https://james.apache.org/[James Official Website]
	* link:https://github.com/apache/james-project[James GitHub Project]
	* xref:operate/webadmin.adoc[WebAdmin]
	* xref:operate/cli.adoc[James CLI]
	* link:https://james.apache.org/server/manage-guice-distributed-james.html[Manage Guice Distributed James]

	### Kubernetes

	* link:https://kubernetes.io/docs/concepts/overview/components/[Kubernetes Components Overview]
	* link:https://kubernetes.io/docs/concepts/workloads/pods/[Pods]
	* link:https://kubernetes.io/docs/concepts/services-networking/service/[Services]
	* link:https://kubernetes.io/docs/concepts/services-networking/ingress/[Ingress]
	* link:https://kubernetes.io/docs/concepts/configuration/configmap/[Configmaps]
	* link:https://kubernetes.io/docs/concepts/configuration/secret/[Secrets]
	* link:https://kubernetes.io/docs/concepts/workloads/controllers/deployment/[Deployments]

	### Helm

	* link:https://helm.sh/docs/topics/architecture/[Helm Architecture]
	* link:https://helm.sh/docs/topics/charts/[Charts]
	* link:https://helm.sh/docs/intro/using_helm/[Using Helm]
	* link:https://helm.sh/docs/chart_template_guide/getting_started/[Helm templates]