SonataFlow Data Index Use Cases

Collection of artifacts to test SonataFlow Use Cases TP2.

Prereqs for all the use cases

1. Minikube installed

We recommend that you start Minikube with the following parameters, note that the registry addon must be enabled. Additionally, the ingress addon is also enabled facilitate data-index querying by using an Ingress. Note that the Ingress provided is just en example to expose the data-index graphiql. This is not mandatory, and in production environments you must provide your own setup if needed, or even use an OpenShift route, etc.

minikube start --cpus 4 --memory 10240 --addons registry --addons metrics-server --addons ingress --insecure-registry "10.0.0.0/24" --insecure-registry "localhost:5000"

To verify that the registry addon was property added you can execute this command:

minikube addons list | grep registry

| registry                    | minikube | enabled ✅   | Google                         |
| registry-aliases            | minikube | disabled     | 3rd party (unknown)            |
| registry-creds              | minikube | disabled     | 3rd party (UPMC Enterprises)   |

To verify that the ingress addon was property added you can execute this command:

minikube addons list | grep ingress

| ingress                     | minikube | enabled ✅   | Kubernetes                     |

2. kubectl installed

3. SonataFlow operator installed if workflows are deployed

To install the operator you can see SonataFlow Installation.

Use cases

This is the list of available use cases:

NOTE: To facilitate the switch between use cases, it's strongly recommended to install each use case in a dedicated namespace.

Deploy Data Index locally

Example of how to deploy Data Index on Kubernetes that uses a Postgresql DB.

NOTE: The workflow related use cases that needs a data index service already includes this step.

Procedure

Open a terminal and run the following commands:

1. Create the namespace:

kubectl create namespace data-index-usecase

2. Deploy the Data Index Service:

kubectl kustomize platforms/data_index_as_platform_service | kubectl apply -f - -n data-index-usecase

persistentvolumeclaim/postgres-pvc created
deployment.apps/postgres created
service/postgres created
sonataflowplatform.sonataflow.org/sonataflow-platform created
ingress.networking.k8s.io/data-index-service-ingress created
secret/postgres-secrets created

This will deploy a Data Index for you in the data-index-usecase namespace. (If you don't use a namespace the default is used instead) Data Index will be backed by a Postgres Data Base deployment. This setup is not intended for production environments since this simple Postgres Deployment does not scale well. Please see the Postgres Operator for more information.

To check that the data index is running you can execute this command.

kubectl get pod -n data-index-usecase

data-index-service-postgresql-5d76dc4468-69hm6   1/1     Running   0              2m11s
postgres-7f78499688-j6282                        1/1     Running   0              2m11s

To access the Data Index, using Minikube you can run:

minikube ip

Example output:

192.168.49.2

Use the returned ip to access the data-index-service GraphiQL by using the Ingress with a url like this http://192.168.49.2/graphiql/ (that ip might be different in your installation.)

For more information about Data Index and this deployment see Data Index standalone service in SonataFlow guides.

To execute queries see: Querying Index Queries

3. Clean the use case:

kubectl delete namespace data-index-usecase

Use case 1

This use case is intended to represent an installation with:

• A singleton Data Index Service with PostgreSQL persistence
• The greeting workflow (no persistence), that is configured to register events to the Data Index Service.

Procedure

Open a terminal and run the following commands:

1. Create the namespace:

kubectl create namespace usecase1

2. Deploy the Data Index Service:

kubectl kustomize platforms/data_index_as_platform_service | kubectl apply -f - -n usecase1

persistentvolumeclaim/postgres-pvc created
deployment.apps/postgres created
service/postgres created
sonataflowplatform.sonataflow.org/sonataflow-platform created
ingress.networking.k8s.io/data-index-service-ingress created
secret/postgres-secrets created

Give some time for the data index to start, you can check that it's running by executing.

kubectl get pod -n usecase1

NAME                                             READY   STATUS    RESTARTS       AGE
data-index-service-postgresql-5d76dc4468-lb259   1/1     Running   0              2m11s
postgres-7f78499688-lc8n6                        1/1     Running   0              2m11s

3. Deploy the workflow:

 kubectl kustomize usecases/usecase1 | kubectl apply -f - -n usecase1

sonataflow.sonataflow.org/greeting created

Give some time for the sonataflow operator to build and deploy the workflow. To check that the workflow is ready you can use this command.

kubectl get workflow -n usecase1

NAME       PROFILE   VERSION   URL   READY   REASON
greeting             0.0.1           True    

4. Expose the workflow and get the url:

kubectl patch svc greeting -p '{"spec": {"type": "NodePort"}}' -n usecase1

 minikube service greeting --url -n usecase1

5. Create a workflow instance:

You must use the URLs calculated in step 4.

curl -X POST -H 'Content-Type:application/json' -H 'Accept:application/json' -d '{"name": "John", "language": "English"}'    http://192.168.49.2:32407/greeting

To execute queries and see the workflows information see: Querying Index Queries

6. Clean the use case:

kubectl delete namespace usecase1

Use case 2

This use case is intended to represent an installation with:

• A singleton Data Index Service with PostgreSQL persistence
• The greeting workflow (no persistence)
• The helloworkflow workflow (no persistence)
• The workflows are configured to register the process events on the Data Index Service.

Procedure

Open a terminal and run the following commands:

1. Create the namespace:

kubectl create namespace usecase2

2. Deploy the Data Index Service:

kubectl kustomize platforms/data_index_as_platform_service | kubectl apply -f - -n usecase2

persistentvolumeclaim/postgres-pvc created
deployment.apps/postgres created
service/postgres created
sonataflowplatform.sonataflow.org/sonataflow-platform created
ingress.networking.k8s.io/data-index-service-ingress created
secret/postgres-secrets created

Give some time for the data index to start, you can check that it's running by executing.

kubectl get pod -n usecase2

NAME                                             READY   STATUS    RESTARTS       AGE
data-index-service-postgresql-5d76dc4468-lb259   1/1     Running   0              2m11s
postgres-7f78499688-lc8n6                        1/1     Running   0              2m11s

3. Deploy the workflows:

 kubectl kustomize usecases/usecase2 | kubectl apply -f - -n usecase2

sonataflow.sonataflow.org/greeting created
sonataflow.sonataflow.org/helloworld created

Give some time for the sonataflow operator to build and deploy the workflows. To check that the workflows are ready you can use this command.

kubectl get workflow -n usecase2

NAME       PROFILE   VERSION   URL   READY   REASON
greeting             0.0.1           True
helloworld           0.0.1           True        

4. Expose the workflows and get the urls:

kubectl patch svc greeting helloworld  -p '{"spec": {"type": "NodePort"}}' -n usecase2

 minikube service greeting --url -n usecase2

 minikube service helloworld --url -n usecase2

5. Create a workflow instances:

You must use the URLs calculated in step 4.

curl -X POST -H 'Content-Type:application/json' -H 'Accept:application/json' -d '{"name": "John", "language": "English"}'    http://192.168.49.2:32407/greeting

curl -X POST -H 'Content-Type:application/json' -H 'Accept:application/json' -d '{}'    http://192.168.49.2:32327/helloworld

To execute queries and see the workflows information see: Querying Index Queries

6. Clean the use case:

kubectl delete namespace usecase2

Querying Data Index

You can use the public Data Index endpoint to play around with the GraphiQL interface.

Procedure

This procedure apply to all use cases with that deploys the Data Index Service.

1. Get the Data Index Url:

minikube ip

2. Open the GrahiqlUI

Using the ip returned in 1, open a browser window in the following url http://192.168.49.2/graphiql/, note that IP will be different in your installation, and don‘t forget to add the last slash “/” to the url, otherwise the GraphiqlUI won’t be opened.

To see the process instances information you can execute this query:

{
  ProcessInstances {
    id,
    processId,
    processName,
    variables,
    state,
    endpoint,
    serviceUrl,
    start,
    end
  }
}

The results should be something like:

{
  "data": {
    "ProcessInstances": [
      {
        "id": "3ed8bf63-85c9-425d-9099-49bfb63608cb",
        "processId": "greeting",
        "processName": "workflow",
        "variables": "{\"workflowdata\":{\"name\":\"John\",\"greeting\":\"Hello from JSON Workflow, \",\"language\":\"English\"}}",
        "state": "COMPLETED",
        "endpoint": "/greeting",
        "serviceUrl": "http://greeting",
        "start": "2023-09-13T06:59:24.319Z",
        "end": "2023-09-13T06:59:24.400Z"
      }
    ]
  }
}

To see the jobs instances information, if any, you can execute this query:

{
  Jobs {
    id,
    processId,
    processInstanceId,
    status,
    expirationTime,
    retries,
    endpoint,
    callbackEndpoint
  }
}

The results should be something like:

{
  "data": {
    "Jobs": [
      {
        "id": "55c7aadb-3dff-4b97-af8e-cc45014b1c0d",
        "processId": "callbackstatetimeouts",
        "processInstanceId": "299886b7-2b78-4965-a701-16783c4162d8",
        "status": "EXECUTED",
        "expirationTime": null,
        "retries": 0,
        "endpoint": "http://jobs-service-postgresql/jobs",
        "callbackEndpoint": "http://callbackstatetimeouts:80/management/jobs/callbackstatetimeouts/instances/299886b7-2b78-4965-a701-16783c4162d8/timers/-1"
      }
    ]
  }
}