Deploying OpenWhisk on IBM Cloud Kubernetes Service (IKS)

Overview

IBM provides both a “Lite” and a “Standard” Kubernetes offering in its public cloud Kubernetes service (IKS). These differ in capabilities, so they are described separately below.

Initial setup

Creating the Kubernetes Cluster

Follow IBM's instructions to provision your cluster.

Configuring OpenWhisk

IBM Cloud Standard cluster

An IBM Cloud Standard cluster has full support for TLS including a wild-card certificate for subdomains and can be configured with additional annotations to fine tune ingress performance.

First, determine the values for and for your cluster by running the command:

bx cs cluster-get <mycluster>

The CLI output will look something like

bx cs cluster-get <mycluster>
Retrieving cluster <mycluster>...
OK
Name:    <mycluster>
ID:    b9c6b00dc0aa487f97123440b4895f2d
Created:  2017-04-26T19:47:08+0000
State:    normal
Master URL:  https://169.57.40.165:1931
Ingress subdomain:  <domain>
Ingress secret:  <ibmtlssecret>
Workers:  3

As described in IBM's ingress documentation, to enable applications deployed in multiple namespaces to share the ingress resource, you should use a unique subdomain name for each namespace. We suggest a convention of using the namespace name as the subdomain name. So if you are deploying openwhisk into the openwhisk namespace, use openwhisk as your subdomain (as shown below in the example mycluster.yaml).

Now define mycluster.yaml as below (substituting the real values for <domain> and <ibmtlssecret>).

whisk:
  ingress:
    apiHostName: openwhisk.<domain>
    apiHostPort: 443
    apiHostProto: https
    type: Standard
    domain: openwhisk.<domain>
    tls:
      enabled: true
      secretenabled: true
      createsecret: false
      secretname: <ibmtlssecret>
    annotations:
      # A blocking request is held open by the controller for slightly more than 60 seconds
      # before it is responded to with HTTP status code 202 (accepted) and closed.
      # Set to 75s to be on the safe side.
      # See https://console.bluemix.net/docs/containers/cs_annotations.html#proxy-connect-timeout
      # See http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout
      ingress.bluemix.net/proxy-read-timeout: "75s"

      # Allow up to 50 MiB body size to support creation of large actions and large
      # parameter sizes.
      # See https://console.bluemix.net/docs/containers/cs_annotations.html#client-max-body-size
      # See http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size
      ingress.bluemix.net/client-max-body-size: "size=50m"

      # Add the request_id, generated by nginx, to the request against the controllers. This id will be used as tid there.
      # Note that the serviceName includes the argument to --name from the helm deploy command. (owdev in this example)
      # https://console.bluemix.net/docs/containers/cs_annotations.html#proxy-add-headers
      ingress.bluemix.net/proxy-add-headers: |
        serviceName=owdev-controller {
          'X-Request-ID' $request_id;
        }

invoker:
  containerFactory:
    impl: kubernetes

k8s:
  persistence:
    hasDefaultStorageClass: false
    explicitStorageClass: default

Starting with IKS 1.11, the underlying container runtime is now containerd instead of docker. As a result, you cannot use the DockerContainerFactory on IKS and must use the KubernetesContainerFactory.

IKS does not provide a properly configured DefaultStorageClass, instead you need to tell the Helm chart to use the default StorageClassName as shown above. This StorageClass does have a dynamic provisioner, so it is not necessary to manually create the PersistentVolumes. Note that it is not unusual for it to take several minutes for your PersistentVolumes to be created (dependent resources will be in Pending state).

IBM Cloud Lite cluster

The only available ingress method for an IBM Cloud Lite cluster is to use a NodePort. Obtain the Public IP address of the sole worker node by using the command

bx cs workers <my-cluster>

Then define mycluster.yaml as

whisk:
  ingress:
    type: NodePort
    apiHostName: YOUR_WORKERS_PUBLIC_IP_ADDR
    apiHostPort: 31001

nginx:
  httpsNodePort: 31001

k8s:
  persistence:
    hasDefaultStorageClass: false
    explicitStorageClass: default

IKS does not provide a properly configured DefaultStorageClass, instead you need to tell the Helm chart to use the default StorageClassName as shown above. This StorageClass does have a dynamic provisioner, so it is not necessary to manually create the PersistentVolumes. Note that it is not unusual for it to take several minutes for your PersistentVolumes to be created (dependent resources will be in Pending state).

Hints and Tips

On IBM Standard clusters, you can configure OpenWhisk to integrate with platform logging and monitoring services following the general instructions for enabling these services for pods deployed on Kubernetes.

Limitations

Using an IBM Cloud Lite cluster is only appropriate for development and testing purposes. It is not recommended for production deployments of OpenWhisk.

When using an IBM Cloud Lite cluster, TLS termination will be handled by OpenWhisk's nginx service and will use self-signed certificates. You will need to invoke wsk with the -i command line argument to bypass certificate checking.