<!--
    Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.
-->

The Beam Playground is a web application to run Beam code snippets in a modern browser. This directory holds code to
build, test, and deploy the frontend and backend services.

- [Setup development prerequisites](#setup-development-prerequisites)
- [Common tasks](#common-tasks)
- [Run Beam Playground locally](#run-beam-playground-locally)
  - [Configure frontend to use local backend](#configure-frontend-to-use-local-backend)
  - [Run local deployment using Gradle task](#run-local-deployment-using-gradle-task)
  - [Deploy examples](#deploy-examples)
- [How to add your own example](#how-to-add-your-own-example)
- [Deployment guide](#deployment-guide)
  - [Manual deployment](#manual-deployment)
  - [Manual CloudBuild setup](#manual-cloudbuild-setup)
- [Project structure](#project-structure)
- [Contribution guide](#contribution-guide)

# Setup development prerequisites
> ***Google Cloud Shell note***: Google Cloud Shell already has most of the prerequisites installed. Only the following has to be installed manually:
> - Flutter - run `flutter precache`
> - Go protobuf dependencies
> - Dart protobuf dependencies
> - buf
> - sbt

1. Install Go 1.23+

    **Ubuntu 22.04 and newer:**
    ```shell
    sudo apt install golang
    ```

    **Other Linux variants:** Follow manual at https://go.dev/doc/install
1. Install [flutter](https://docs.flutter.dev/get-started/install/linux)

    **Ubuntu 22.04 or newer:**
    ```shell
    sudo apt install flutter
    ```

    **Other Linux variants:** Follow manual at https://flutter.dev/docs/get-started/install/linux

1. Install [protoc](https://grpc.io/docs/protoc-installation/)

    **Ubuntu 22.04 or newer/Debian 11 or newer:**
    ```shell
    sudo apt install protobuf-compiler
    ```
    **Other Linux variants:** Follow manual at https://grpc.io/docs/protoc-installation/

1. Install Go protobuf dependencies: [Go gRPC Quickstart](https://grpc.io/docs/languages/go/quickstart/)
    > Do not forget to update your `PATH` environment variable
1. Install Dart protobuf dependencies: [Dart gRPC Quickstart](https://grpc.io/docs/languages/dart/quickstart/)
    > Do not forget to update your `PATH` environment variable
1. Install npm
    **Ubuntu 22.04 and newer/Debian 11 and newer:**
    ```shell
    sudo apt install npm
    ```
    **Other Linux variants:** Follow manual at https://docs.npmjs.com/downloading-and-installing-node-js-and-npm
1. Install [buf](https://docs.buf.build/installation)
    ```shell
    npm install -g @bufbuild/buf
    ```
1. Install Docker
    **Ubuntu 22.04 and newer/Debian 11 and newer:**
    ```shell
    sudo apt install docker.io
    ```
    **Other Linux variants:** Follow manual at https://docs.docker.com/engine/install/

    To verify your docker installation, run:
    ```shell
    docker ps
    ```
    It should finish without any errors. If you get a permission denied error, you need to add your user to the docker group:
    ```shell
    sudo usermod -aG docker $USER
    ```
    Then, log out and log back in to apply the changes.
1. Install Docker Compose
    **Ubuntu 22.04 and newer/Debian 11 and newer:**
    ```shell
    sudo apt install docker-compose
    ```
    **Other Linux variants:** Follow manual at https://docs.docker.com/compose/install/
1. Install gcloud CLI by following the [manual](https://cloud.google.com/sdk/docs/install) for your system
1. Install Cloud Datastore Emulator
    ```shell
    gcloud components install cloud-datastore-emulator
    ```
    or, if you have install gcloud CLI using APT:
    ```shell
    sudo apt install google-cloud-cli-datastore-emulator
    ```
1. Install `sbt` following instruction at https://www.scala-sbt.org/1.x/docs/Installing-sbt-on-Linux.html
    > **Optional**: Run `sbt` comamnd once after installation to let it cache its dependencies
1. Make sure that you have the full installation of Python 3.8 or newer. On Debian or Ubuntu it can be installed using `sudo apt install python3-full`.

# Common tasks
To get an overview of common tasks, see [TASKS.md](TASKS.md)

# Run Beam Playground locally
## Configure frontend to use local backend
> **Note:** Follow this step only if you want to have a local deployment of Playground. Skip this step entirely if you want to deploy Playground to Google Cloud.

Uncommend lines after `// Uncomment the following lines to use local backend.` in [frontend/playground_components/lib/src/constants/backend_urls.dart](/playground/frontend/playground_components/lib/src/constants/backend_urls.dart)

## Run local deployment using Gradle task
> For more information read the corresponding section in [TASKS.md](./TASKS.md#router-runners-and-frontend)

Run the deployment script:
```shell
./gradlew :playground:dockerComposeLocalUp
```

To shut down the playground, run:
```shell
./gradlew :playground:dockerComposeLocalDown
```

## Deploy examples
1. Go to `/playground/infrastucture` directory
    ```shell
    cd playground/infrastucture
    ```
1. Setup python venv
    ```shell
    python3 -m venv venv
    ```
1. Activate venv
    ```shell
    source venv/bin/activate
    ```
1. Install dependencies
    ```shell
    pip install -r requirements.txt
    ```
1. Setup the environment variables
    ```shell
    export BEAM_ROOT_DIR=$(realpath ../../)
    export SDK_CONFIG="../../playground/sdks.yaml"
    export BEAM_EXAMPLE_CATEGORIES="../categories.yaml"
    export BEAM_USE_WEBGRPC=yes
    export BEAM_CONCURRENCY=4
    export DATASTORE_EMULATOR_HOST=localhost:8081
    ```

1. Run the `ci_cd.py` script

    ```shell
    export SERVER_ADDRESS=<runner_address> # see the note below
    python ci_cd.py --step CD \
                    --sdk <SDK> \ # SDK_GO, SDK_JAVA, SDK_PYTHON, SDK_SCIO
                    --namespace Playground \
                    --datastore-project test \
                    --origin PG_EXAMPLES \
                    --subdirs $BEAM_ROOT_DIR/sdks $BEAM_ROOT_DIR/examples $BEAM_ROOT_DIR/learning/katas # For SCIO examples see note below
    ```
    > **Note:** The `SERVER_ADDRESS` variable should be set to the address of the runner server for the particular SDK. For the local deployment the default values are:
    > | SDK | Address |
    > | --- | --- |
    > | Go | `localhost:8084` |
    > | Java | `localhost:8086` |
    > | Python | `localhost:8088` |
    > | SCIO | `localhost:8090` |

    > **Note:** SCIO examples are not committed to the Beam tree. You will need to use [`fetch_scala_examples.py`](infrastructure/fetch_scala_examples.py) script to download them to some directory and then use `--subdirs` option to point to that directory. For example:
    > ```shell
    > python fetch_scala_examples.py --output-dir /tmp/scio-examples
    > python ci_cd.py --step CD \
    >                 --sdk SDK_SCIO \
    >                 --namespace Playground \
    >                 --datastore-project test \
    >                 --origin PG_EXAMPLES \
    >                 --subdirs /tmp/scio-examples
    > ```
    > See [this section](TASKS.md##obtaining-scio-examples) for more information.

# How to add your own example
Please refer to [this document](load_your_code.md).

# Deployment guide
## Manual deployment
See deployment guide at [terraform/README.md](/playground/terraform/README.md)
## Manual CloudBuild setup
To set up CloudBuild triggers manually please refer to [this guide](/playground/terraform/infrastructure/cloudbuild-manual-setup/README.md)

# Project structure

Several directories in this repository are used for the Beam Playground project. The list of the directories used can be seen below.

| Directory | Purpose |
|-----------|---------|
| [`/examples`](/examples) | Contains code of the examples for Java SDK. The examples are getting loaded into the main catalog. |
| [`/learning/beamdoc`](/learning/beamdoc) | Contains code of the examples which should not be available in the main Playground catalog. |
| [`/learning/katas`](/learning/katas) | Containes small code examples, loaded into main catalog.
| [`/playground`](/playground) | Root of Playground sources. |
| [`/playground/api`](/playground/api) | Protobuf definitions for the Playground gRPC API. |
| [`/playground/backend`](/playground/backend) | Root of Playground backend sources. See [this document](/playground/backend/CONTRIBUTE.md) for detailed description. |
| [`/playground/frontend`](/playground/frontend) | Root of Playground frontend sources. See [this document](/playground/frontend/CONTRIBUTE.md) for detailed description. |
| [`/playground/infrastructure`](/playground/infrastructure) | Scripts used in Playground CI/CD pipelines for verifying and uploading examples. |
| [`/playground/kafka-emulator`](/playground/kafka-emulator) | Sources of a Kafka emulator used for demonstrating KafkaIO examples. |
| [`/playground/terraform`](/playground/terraform) | Terraform configuration files for Playground deployment to Google Cloud Platform. |
| [`/sdks`](/sdks) | Source of the BEAM SDKs. Used by Playground as a source of examples for Go and Python SDKs. |

# Contribution guide
- Backend: see [backend/README.md](/playground/backend/README.md) and [backend/CONTRIBUTE.md](/playground/backend/CONTRIBUTE.md)
- Frontend: see [frontend/README.md](/playground/frontend/README.md) and [frontend/CONTRIBUTE.md](/playground/frontend/CONTRIBUTE.md)