To install Docker, visit its official page and install the correct version for your system.
The walkthrough uses Minikube to guide you through the setup process. Visit the official Minikube page to install Minikube.
You can use git clone
to download repository to your computer.
Once you have cloned the project repo and started Docker and Minikube, in the terminal:
$ eval $(minikube docker-env)
$ cd trafficserver-ingress-controller
$ git submodule update --init
$ docker build -t ats-ingress .
$ docker build -t ats-ingress-exporter k8s/images/trafficserver_exporter/
$ docker build -t node-app-1 k8s/images/node-app-1/
$ docker build -t node-app-2 k8s/images/node-app-2/
$ docker pull fluent/fluentd:v1.6-debian-1
$ kubectl create namespace trafficserver-test
$ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=atssvc/O=atssvc"
$ kubectl create secret tls tls-secret --key tls.key --cert tls.crt -n trafficserver-test --dry-run=client -o yaml | kubectl apply -f -
$ kubectl apply -f k8s/configmaps/fluentd-configmap.yaml
$ kubectl apply -f k8s/traffic-server/
The following steps can be executed in any order
$ kubectl apply -f k8s/apps/
trafficserver-test-2
and trafficserver-test-3
if not already existappsvc1
and appsvc2
appsvc1
, and appsvc2
pods in trafficserver-test-2
, totally 4 pods in said namespace.appsvc1
, and appsvc2
pods in trafficserver-test-3
, totally 4 pods in this namespace. We now have 8 pods in total for the 2 services we have created and deployed in the 2 namespaces.$ kubectl apply -f k8s/ingresses/
trafficserver-test-2
and trafficserver-test-3
if not already existtrafficserver-test-2
and trafficserver-test-3
trafficserver-test-2
defines domain name test.media.com
with /app1
and /app2
as its pathstest.edge.com
; however, test.edge.com/app1
is only defined in trafficserver-test-2
and test.edge.com/app2
is only defined in trafficserver-test-3
test.edge.com/app2
in namespace trafficserver-test-3
ATS proxying should have started to work. To see proxy in action, we can use curl:
$ curl -vH "HOST:test.media.com" "$(minikube ip):30000/app1"
$ curl -vH "HOST:test.media.com" "$(minikube ip):30000/app2"
$ curl -vH "HOST:test.edge.com" "$(minikube ip):30000/app1"
$ curl -vH "HOST:test.edge.com" "$(minikube ip):30000/app2"
$ curl -vH "HOST:test.edge.com" -k "https://$(minikube ip):30043/app2"
You may have problem with minikube using docker driver as localhost (i.e. 127.0.0.1) will be used as the cluster ip. So you will need to forward the traffic designated for the port to the ports of the ATS pods inside the cluster before the above curl commands will work. Each command below needs to be run in separate terminal.
$ kubectl port-forward <pod name> 30043:443 -n trafficserver-test
$ kubectl port-forward <pod name> 30000:80 -n trafficserver-test
Below is an example of configuring Apache Traffic Server reloadable configurations using kubernetes configmap resource:
$ kubectl apply -f k8s/configmaps/ats-configmap.yaml
trafficserver-test
with the annotation "ats-configmap":"true"
if not already existproxy.config.output.logfile.rolling_enabled: "1"
proxy.config.output.logfile.rolling_interval_sec: "3000"
proxy.config.restart.active_client_threshold: "0"
You can specifiy the list of namespaces to look for ingress object by providing INGRESS_NS
. The default is all
, which tells the controller to look for ingress objects in all namespaces. Alternatively you can provide a comma-separated list of namespaces for the controller to look for ingresses. Similarly you can specifiy a comma-separated list of namespaces to ignore while the controller is looking for ingresses by providing INGRESS_IGNORE_NS
.
You can attach ATS lua script to an ingress object and ATS will execute it for requests matching the routing rules defined in the ingress object. See an example in annotation section of yaml file here
You can provide an environment variable called INGRESS_CLASS
in the deployment to specify the ingress class. See an example commented out here. Only ingress object with annotation kubernetes.io/ingress.class
with value equal to the environment variable value will be used by ATS for routing
You can specify a different logging.yaml and ssl_server_name.yaml by providing environment variable LOG_CONFIG_FNAME
and SSL_SERVER_FNAME
respsectively. See an example commented out here. The new contents of them can be provided through a ConfigMap and loaded to a volume mounted for the ATS container (Example here ). Similarly certificates needed for the connection between ATS and origin can be provided through a Secret that loaded to a volume mounted for the ATS container as well (Example here ). To refresh these certificates we may need to override the entrypoint with our own command and add extra script to watch for changes in those secret in order to reload ATS (Example here ).
You can specify extra plugins for plugin.config by providing environment variable EXTRA_PLUGIN_FNAME
. Its contents can be provided through a ConfigMap and loaded to a volume mounted for the ATS container (Example here ).
This project ships with Fluentd already integrated with the Apache Traffic Server. The configuration file used for the same can be found here
As can be seen from the default configuration file, Fluentd reads the Apache Traffic Server access logs located at /usr/local/var/log/trafficserver/squid.log
and outputs them to stdout
. The ouput plugin for Fluentd can be changed to send the logs to any desired location supported by Fluentd including Elasticsearch, Kafka, MongoDB etc. You can read more about output plugins here.
Use the following steps to install Prometheus and Grafana and use them to monitor the Apache Traffic Server statistics.
$ kubectl apply -f k8s/prometheus/ats-stats.yaml
$ kubectl apply -f k8s/configmaps/prometheus-configmap.yaml
$ kubectl apply -f k8s/prometheus/prometheus-deployment.yaml
x.x.x.x:30090
in your web browser to access Prometheus where x.x.x.x
is the IP returned by the command: $ minikube ip
x.x.x.x:30030
in your web browser to access the Grafana dashboard where x.x.x.x
is the IP returned by the command: $ minikube ip
.admin:admin
localhost:9090
as the URL trafficserver_responses_total
and press Shift + Enter. Helm Chart is provided. You can delete the namespace of trafficserver-test
and monitoring-layer
created above and continue the tutorial by following instructions here. The curl commands here will continue to work.