Explore sample yaml files under this package to submit applications and get them scheduled by yunikorn.
A simple one pod deployment of a nginx image.
Several examples for affinity and anti affinity:
Two sample deployments using sleep jobs:
A sample application using Apache Spark.
Submit a spark job with the run.sh script, for full details see the user guide in the yunikorn-core.
Deployment files for the driver and executor:
A simple example that runs a kubeflow/tensorflow job. In this example it will run a distributed mnist model for e2e test, for full more detail see the dist-mnist section.
A sample application which implement gang in application level. Start via the gangDeploy.sh script, for full more detail see the gang section.
Deployment file for gang-coordinator and gang-job:
The volumes directory contains three examples:
All examples will generate an unending stream of data in a file called dates.txt
on the mounted volume. Run times should be limited or the disk usage must be monitored for longer runs.
kubectl get services | grep nfs-server | awk '{print $3}'Use the cluster IP address returned to update the client pod yaml for the next step.
NOTE: The NFS server will not work on a Docker Desktop in a Mac OS-X environment. Even with kubernetes turned on. The image used by Docker Desktop does not support the kernel NFS driver used by the server. Use minikube as a workaround and follow the environment setup to add it to Docker Desktop.
EBS volumes cannot be used outside of AWS. Therefor you must have an EKS cluster on AWS. Pre-requisites and installation that is not described in this document:
Three example cases:
--enable-admission-plugins DefaultStorageClass
set. see Dynamic Volume Provisioning in the kubernetes docs.NOTE: dynamic provisioning can interfere with the existing volume examples given.
The namespace example uses a placement rule and special queue configuration. The pods are a simple sleep pod which will be scheduled based on the namespace it runs in. The pod specification does not have a queue set.
kubectl create -f namespaces.yaml
development
queue using the local sleeppod_dev.yaml: kubectl create -f sleeppod_dev.yaml
. The pod spec does not specify a queue just a namespace but the application will be run in the root.development
queue.production
queue using the local sleeppod_prod.yaml: kubectl create -f namespaces.yaml
. The pod spec does not specify a queue just a namespace but the application will be run in the newly created root.production
queue. This queue does not exist in the queue configuration.The placements' rules are described in Yunikorn website. App placements rules in Yunikorn contains Provided Rule
, User Name Rule
, Fixed Rule
, Tag Rule
. Every placement example includes a example yaml file and a config yaml file. The sleep pod is described in the example file and the partition is provided in the config yaml file. Before deploying the pods, the configuration field in yunikorn-release/helm/yunikorn/value.yaml should be replaced by the configuration field in the config.yaml.