SETUP.md

# What you want to do?

- Get access to the test environments
- Generate all the secrets to run the tests locally
- Upload the secrets and run the tests on GitHub actions
- Rebuild the CI environment

# Recover access to the CI environment

- Copy `.env.dist` in `.env` and put the secrets in it (ask us of course)
- Execute `task kubeconfig` to load all the kubeconfig in ops.
- Execute `ops config use` then `ops config use <n>` to select a configuration
- Execute `ops setup nuvolaris login` to log into one of them

# Generate all the secrets

- Copy `.env.dist` in `.env` and put the secrets in it
- `task secrets` generates a  `.secrets` with all the secrets for github
- now you can run the tests, most notably the `tests/1-deploy.sh` that builds environments

# Setup of Continuous Integration

Those notes are a remindinder of the steps executed to build our CI environment.

## Overview

We have a stable test environment on Azure/AWS/GCloud with:

- in Azure, a domain (opstest.top) with subzones. All subzones are currently in Azure
- in Azure, an AKS cluster
- in AWS, an EKS cluster
- in Gcloud, a GKE cluster
- in Gcloud, an OpenShift cluster 
- in Gcloud, a virtual machine k3s-test and mk8s-test


# Setup

## Prereq

You to install a few tools:

- [`task`](https://taskfile.dev)
- `aws` cli
- `azure` cli 
- `gcloud` cli 

## Permissions in AWS

Generated an user with admin power and extacted the Access and Secret Key 

## Permissions on Gcloud

- Enabled the services required:

```
gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable dns.googleapis.com
gcloud services enable iamcredentials.googleapis.com
gcloud services enable iam.googleapis.com
gcloud services enable servicemanagement.googleapis.com
gcloud services enable serviceusage.googleapis.com
gcloud services enable storage-api.googleapis.com
gcloud services enable storage-component.googleapis.com
gcloud services enable deploymentmanager.googleapis.com
gcloud services enable resourcemanager.projects.delete
```

Also manually  enabled Kubernetes cluster creation and IAM management

-  Generated a service account for GCloud with administrative power. 

The service account email available in: https://console.cloud.google.com/iam-admin/serviceaccounts, assigned the role "Owner" to the service account, then extracted the json for the service account:

```
gcloud iam service-accounts keys create ~/.ssh/gcloud.json --iam-account=<account-email>
```

## SSH 

- Generate a id_rsa and and id_rsa.key,
stored in ~/.ssh/id_rsa and ~/.id_rsa.pub

# DNS

- Created the zone `oshgcp.opstest.top` in Gcloud

- Created the following zones in AWS Route53
  - k3s.opstest.top
  - mk8s.opstest.top
  - eks.opstest.top
  - aks.opstest.top 
  - gke.opstest.top 

- Registered a domain in AWS (opstest.top) and delegated all the subzones.

# OpenShift

- Created conf/gcp-install-config.yaml and conf/aws-install-config.yaml

First running the openshift-install and then manually tweaked the configuration.

Note you need:
- an id_rsa.pub
- for gcloud, the service account file 
- the dns zone ub GCP we created (oshgcp.opstest.top)
- the pullSecret for OKD (open source openshift) as follows: 

```
{"auths":{"fake":{"auth":"aWQ6cGFzcwo="}}}
```

# Configure variables

Configure `env` copying it from env.dit and filling it with all the required secrets.

# Create all the clusters and vms

Once everything is configured we can build all the clusters:

- `task k3s:create`
- `task mk8s:create`
- `task gke:create`
- `task aks:create`
- `task eks:create`
- `task osh:create`

*NOTE*: many parameters are wired in the taskfiles: look for the `*:config` tasks in `Taskfile*.yml` if you want to tune them.

# Upload the secrets

Once you created the clusters, you can upload their kubeconfig or ip as secrets to GitHub with:

- `task secrets`
- `task upload-secrets`