Added more docs about node labels and how to start kubelet
1 file changed
tree: 83b0510d73afa36e19c3cf0a4d4774e80667d844
  1. cmd/
  2. vendor/
  3. .gitignore
  4. cloudstack.go
  5. cloudstack_instances.go
  6. cloudstack_loadbalancer.go
  7. cloudstack_test.go
  8. deployment.yaml
  9. Dockerfile
  10. go.mod
  11. go.sum
  12. LICENSE-2.0
  13. Makefile
  14. metadata.go
  15. metadata_linux.go
  16. metadata_other.go
  17. README.md
  18. service.yaml
README.md

Cloudstack Cloud Controller Manager

A Cloud Controller Manager to facilitate Kubernetes deployments on Cloudstack.

Based on the old Cloudstack provider in Kubernetes that will be removed soon.

Migration

There are several notable differences from the old cloud provider that need to be taken into account when migrating to the standalone provider.

Load Balancer

Load balancer rule names now include the protocol as well as the LB name and service port. This was added to distinguish tcp, udp and tcp-proxy service operating on the same port. Without this change, it would not be possible to map, for example, a service that runs on both TCP and UDP port 8000.

:warning: If you have existing rules, remove them before upgrading and add them back afterwards.

If you don't do this, you need to manually remove the rules in CloudStack when they are replaced.

Metadata

When kubelet still contained cloud provider code, node metadata was fetched from the DHCP server on the instance's Virtual Router.

This is no longer possible with the standalone cloud controller, so all metadata now comes from the Cloudstack API. Some metadata may be missing or wrong, please file bugs when this happens to you.

Node Labels

It is recommended to launch kubelet with the following parameter:

--register-with-taints=node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule

This will treat the node as ‘uninitialized’ and cause the CCM to apply metadata labels from CloudStack automatically.

Supported labels are:

  • kubernetes.io/hostname (= the instance name)
  • beta.kubernetes.io/instance-type (= the compute offering)
  • failure-domain.beta.kubernetes.io/zone (= the zone)
  • failure-domain.beta.kubernetes.io/region (also = the zone)

It is also possible to trigger this process manually by issuing the following command:

kubectl taint nodes <my-node-without-labels> node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule

Build

All dependencies are vendored. You need GNU make, git and Go 1.11 to build cloudstack-ccm.

It's still possible to build with Go 1.10, but you need to remove the option -mod vendor from the cloudstack-ccm compilation target in the Makefile.

go get github.com/swisstxt/cloudstack-cloud-controller-manager
cd ${GOPATH}/src/github.com/swisstxt/cloudstack-cloud-controller-manager
make

To build the cloudstack-cloud-controller-manager container, please use the provided Docker file:

docker build . -t swisstxt/cloudstack-cloud-controller-manager:latest

Use

Prebuilt containers are posted on Docker Hub.

Kubernetes

To deploy the ccm in the cluster see deployment.yaml and configure your cloudstack and api server connection. See the comments.

Protocols

This CCM supports TCP, UDP and TCP-Proxy LoadBalancer deployments.

For UDP and Proxy Protocol support, CloudStack 4.6 or later is required.

Since kube-proxy does not support the Proxy Protocol or UDP, you should connect this directly to containers, for example by deploying a DaemonSet and setting hostNetwork: true.

See service.yaml for an example Service deployment and part of a suitable configuration for an ingress controller.

Development

Make sure your apiserver is running locally and keep your cloudstack config ready:

./cloudstack-ccm --cloud-provider external-cloudstack --cloud-config cloud.config --master localhost

Copyright

© 2018 SWISS TXT AG and the Kubernetes authors.

See LICENSE-2.0 for permitted usage.