There are several ways to setup a local development environment for Kubernetes, the two most common ones are Minikube
(docs) and docker-desktop
. Minikube
provisions a local Kubernetes cluster on several Virtual Machines (via VirtualBox or something similar). docker-desktop
on the other hand, sets up Kubernetes cluster in docker containers.
In this tutorial, we will base all the installs on Docker Desktop. Even in this case we can use a lightweight minikube setup which gives the same functionality with less impact.
Download and install Docker-Desktop on your laptop. Latest version has an embedded version of Kubernetes so no additional install is needed. Just simply follow the instruction here to get Kubernetes up and running within docker-desktop.
Once Kubernetes is started in docker desktop, you should see something similar below:
This means that:
kubctl
is installed in the /usr/local/bin
directory.docker-desktop
.After setting up the local Kubernetes you need to deploy the dashboard using the following steps:
kubectl proxy &
The dashboard as deployed in the previous step requires a token or config to sign in. Here we use the token to sign in. The token is generated automatically and can be retrieved from the system.
kubectl -n kube-system get secret | grep kubernetes-dashboard-token
kubectl -n kube-system describe secret kubernetes-dashboard-token-tf6n8
Data
section with the tag token
.Minikube can be added to an existing Docker Desktop install. Minikube can either use the pre-installed hypervisor or use a hypervisor of choice. These instructions use HyperKit which is embedded in Docker Desktop.
If you want to use a different hypervisor then HyperKit make sure that you follow the generic minikube install instructions. Do not forget to install the correct driver for the chosen hypervisor if required. The basic instructions are provided in the minikube install instructions.
Check hypervisor Docker Desktop should have already installed HyperKit. In a terminal run: hyperkit
to confirm. Any response other than hyperkit: command not found
confirms that HyperKit is installed and on the path. If it is not found you can choose a different hypervisor or fix the Docker Desktop install.
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64 chmod +x minikube sudo mv minikube /usr/local/bin
curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-hyperkit sudo install -o root -g wheel -m 4755 docker-machine-driver-hyperkit /usr/local/bin/
minikube config set vm-driver hyperkit
After the installation is done you can start a new cluster.
minikube start --kubernetes-version v1.14.2
minikube dashboard &
When you create images make sure that the build is run after pointing it to the right environment. Without setting the enviromnent minikube might not find the docker images when deploying the scheduler.
eval $(minikube docker-env)
make image
Note, this instruction requires you have GoLand IDE for development.
In GoLand, go to yunikorn-k8shim project. Then click “Run” -> “Debug...” -> “Edit Configuration...” to get the pop-up configuration window. Note, you need to click “+” to create a new profile if the Go Build
option is not available at the first time.
The highlighted fields are the configurations you need to add. These include:
pkg/shim
packageconf
directory, this is where the program loads configuration file from-kubeConfig=/path/to/.kube/config -interval=1s -clusterId=mycluster -clusterVersion=0.1 -name=yunikorn -policyGroup=queues -logEncoding=console -logLevel=-1
. Note, you need to replace /path/to/.kube/config
with the local path to the kubeconfig file. And if you want to change or add more options, you can run _output/bin/k8s-yunikorn-scheduler -h
to find out.Once the changes are done, click “Apply”, then “Debug”. You will need to set proper breakpoints in order to debug the program.
This setup assumes you have already installed a remote Kubernetes cluster. For a generic view on how to access a multiple cluster and integrate it follow the accessing multiple clusters documentation from Kubernetes.
Or follow these simplified steps:
config
file from remote cluster, copy it to the local machine and give it a unique name i.e. config-remote
KUBECONFIG
environment variable (if set)export KUBECONFIG_SAVED=$KUBECONFIG
export KUBECONFIG=$KUBECONFIG:config-remote
kubectl config view
to check that both configs can be accessedkubectl config use-context my-remote-cluster
kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE docker-for-desktop docker-for-desktop-cluster docker-for-desktop * my-remote-cluster kubernetes kubernetes-admin
More docs can be found here