docs/how-to-add-new-crd-and-controller.md

## How to add CRD and Controller in SWCK?

The guide intends to help contributors who want to add CRDs and Controllers in SWCK.

#### 1. Install the kubebuilder

> Notice, SWCK is built by kubebuilder v3.2.0, so you need to install it at first.

SWCK is based on the [kubebuilder](https://github.com/kubernetes-sigs/kubebuilder), and you could download the kubebuilder by the [script](../hack/install-kubebuilder.sh). 



#### 2. Create CRD and Controller

You can use `kubebuilder create api` to scaffold a new Kind and corresponding controller. Here we use the `Demo` as an example.

```sh
$ cd operator && kubebuilder create api --group operator --version v1alpha1 --kind Demo(Your CRD)
```

Then you need to input twice `y` to create the Resource and Controller, and there will be some newly added files.

```sh
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   PROJECT
	modified:   apis/operator/v1alpha1/zz_generated.deepcopy.go
	modified:   config/crd/bases/operator.skywalking.apache.org_swagents.yaml
	modified:   config/crd/kustomization.yaml
	modified:   config/rbac/role.yaml
	modified:   go.mod
	modified:   go.sum
	modified:   main.go

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	apis/operator/v1alpha1/demo_types.go
	config/crd/bases/operator.skywalking.apache.org_demoes.yaml
	config/crd/patches/cainjection_in_operator_demoes.yaml
	config/crd/patches/webhook_in_operator_demoes.yaml
	config/rbac/operator_demo_editor_role.yaml
	config/rbac/operator_demo_viewer_role.yaml
	config/samples/operator_v1alpha1_demo.yaml
	controllers/operator/demo_controller.go
	controllers/operator/suite_test.go

no changes added to commit (use "git add" and/or "git commit -a")
```

Next, we need to focus on the file `apis/operator/v1alpha1/demo_types.go` which defines your CRD, and the file `controllers/operator/configuration_controller.go` which defines the Controller. The others files are some configurations generated by the [kubebuilder markers](https://book.kubebuilder.io/reference/markers.html). Here are some references:

* [Kubebuilder project demo](https://github.com/kubernetes-sigs/kubebuilder/tree/master/testdata/project-v3), in which you can understand the overall architecture.

* [How to add new-api](https://book.kubebuilder.io/cronjob-tutorial/new-api.html), which you can find more details for `oapserverconfig_types.go`.

* [Controller-overview](https://book.kubebuilder.io/cronjob-tutorial/controller-overview.html), where you can find more details about `oapserverconfig_controller.go`.


### 3. Create webhook

If you want to fields or set defaults to CRs, creating webhooks is a good practice:

```sh
kubebuilder create webhook --group operator --version v1alpha1 --kind Demo --defaulting --programmatic-validation
```

The newly generated files are as follows.

```sh
$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   PROJECT
	modified:   config/webhook/manifests.yaml
	modified:   main.go

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	apis/operator/v1alpha1/demo_webhook.go
	apis/operator/v1alpha1/webhook_suite_test.go

no changes added to commit (use "git add" and/or "git commit -a")
```

You can get more details through [webhook-overview](https://book.kubebuilder.io/reference/webhook-overview.html).



### 4. Create the template

Generally, a controller would generate a series of resources, such as [workload](https://kubernetes.io/docs/concepts/workloads/), [rbac](https://kubernetes.io/docs/reference/access-authn-authz/rbac/), [service](https://kubernetes.io/docs/concepts/services-networking/service/), etc based on CRDs. SWCK is using the Go standard template engine to generate these resources. All template files are stored in the `./operator/pkg/operator/manifests`. You could create a directory there such as `demo` to hold templates. The framework would transfer the CR as the arguments to these templates. More than CR, it supports passing custom rendering functions by setting up the [TmplFunc](https://github.com/apache/skywalking-swck/blob/master/operator/pkg/kubernetes/apply.go#L49). At last, you need to change the [comment](https://github.com/apache/skywalking-swck/blob/bf4d1346a9869f67187b9b9202bf14d190728c56/operator/pkg/operator/manifests/repo.go#L31) and add a field `demo` there to embed the template files into golang binaries.

>  Notice, every file under the template directory can only contain one resource and we can't use the `---` to create multiple resources in a single file.



### 5. Build and Test

SWCK needs to run in the k8s environment, so we highly recommend using the [kind](https://kind.sigs.k8s.io/) if you don't have a cluster in hand. There are currently two ways to test your implementation. 

>  Before testing, please make sure you have the kind installed. 

* Test locally. After finishing your implementation, you could use the following steps to test locally:

1. Disable the webhook

```sh
export ENABLE_WEBHOOKS=false
```

2. Run the main.go with the kubeconfig file.

```sh
go run main.go --kubeconfig=(use your kubeconfig file here, and the default is ~/.kube/config)
```

> If you want to test the webhook, please refer the [guide](https://book.kubebuilder.io/cronjob-tutorial/running.html#running-webhooks-locally).

* Test in-cluster. 

1. Before testing the swck, please install cert-manager to provide the certificate for webhook in swck.

```sh
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.9.1/cert-manager.yaml
```

2. At first, you should build the swck image and load it into the kind cluster, and then you could install the crds and the operator as follows.

```sh
make docker-build && kind load docker-image controller:latest && make install && make deploy
```

3. After the swck is installed, and then you could use the following command to get the logs produced by the operator. 

```sh
kubectl logs -f [skywalking-swck-controller-manager-*](use the swck deployment name) -n skywalking-swck-system
```