docs/how-to-add-new-crd-and-controller.md

How to add CRD and Controller in SWCK?

The guide intends to help contributors who want to add CRDs and Controllers in SWCK.

1. Install the kubebuilder

Notice, SWCK is built by kubebuilder v3.2.0, so you need to install it at first.

SWCK is based on the kubebuilder, and you could download the kubebuilder by the script.

2. Create CRD and Controller

You can use kubebuilder create api to scaffold a new Kind and corresponding controller. Here we use the Demo as an example.

$ cd operator && kubebuilder create api --group operator --version v1alpha1 --kind Demo(Your CRD)

Then you need to input twice y to create the Resource and Controller, and there will be some newly added files.

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   PROJECT
	modified:   apis/operator/v1alpha1/zz_generated.deepcopy.go
	modified:   config/crd/bases/operator.skywalking.apache.org_swagents.yaml
	modified:   config/crd/kustomization.yaml
	modified:   config/rbac/role.yaml
	modified:   go.mod
	modified:   go.sum
	modified:   main.go

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	apis/operator/v1alpha1/demo_types.go
	config/crd/bases/operator.skywalking.apache.org_demoes.yaml
	config/crd/patches/cainjection_in_operator_demoes.yaml
	config/crd/patches/webhook_in_operator_demoes.yaml
	config/rbac/operator_demo_editor_role.yaml
	config/rbac/operator_demo_viewer_role.yaml
	config/samples/operator_v1alpha1_demo.yaml
	controllers/operator/demo_controller.go
	controllers/operator/suite_test.go

no changes added to commit (use "git add" and/or "git commit -a")

Next, we need to focus on the file apis/operator/v1alpha1/demo_types.go which defines your CRD, and the file controllers/operator/configuration_controller.go which defines the Controller. The others files are some configurations generated by the kubebuilder markers. Here are some references:

Kubebuilder project demo, in which you can understand the overall architecture.
How to add new-api, which you can find more details for oapserverconfig_types.go.
Controller-overview, where you can find more details about oapserverconfig_controller.go.

3. Create webhook

If you want to fields or set defaults to CRs, creating webhooks is a good practice:

kubebuilder create webhook --group operator --version v1alpha1 --kind Demo --defaulting --programmatic-validation

The newly generated files are as follows.

$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   PROJECT
	modified:   config/webhook/manifests.yaml
	modified:   main.go

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	apis/operator/v1alpha1/demo_webhook.go
	apis/operator/v1alpha1/webhook_suite_test.go

no changes added to commit (use "git add" and/or "git commit -a")

You can get more details through webhook-overview.

4. Create the template

Generally, a controller would generate a series of resources, such as workload, rbac, service, etc based on CRDs. SWCK is using the Go standard template engine to generate these resources. All template files are stored in the ./operator/pkg/operator/manifests. You could create a directory there such as demo to hold templates. The framework would transfer the CR as the arguments to these templates. More than CR, it supports passing custom rendering functions by setting up the TmplFunc. At last, you need to change the comment and add a field demo there to embed the template files into golang binaries.

Notice, every file under the template directory can only contain one resource and we can't use the --- to create multiple resources in a single file.

5. Build and Test

SWCK needs to run in the k8s environment, so we highly recommend using the kind if you don't have a cluster in hand. There are currently two ways to test your implementation.

Before testing, please make sure you have the kind installed.

Test locally. After finishing your implementation, you could use the following steps to test locally:

Disable the webhook

export ENABLE_WEBHOOKS=false

Run the main.go with the kubeconfig file.

go run main.go --kubeconfig=(use your kubeconfig file here, and the default is ~/.kube/config)

If you want to test the webhook, please refer the guide.

Test in-cluster.

Before testing the swck, please install cert-manager to provide the certificate for webhook in swck.

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.9.1/cert-manager.yaml

At first, you should build the swck image and load it into the kind cluster, and then you could install the crds and the operator as follows.

make docker-build && kind load docker-image controller:latest && make install && make deploy

After the swck is installed, and then you could use the following command to get the logs produced by the operator.

kubectl logs -f [skywalking-swck-controller-manager-*](use the swck deployment name) -n skywalking-swck-system