Explore sample yaml files under this package to submit applications and get them scheduled by yunikorn.
A simple one pod deployment of a nginx image.
Several examples for affinity and anti affinity:
Two sample deployments using sleep jobs:
A sample application using Apache Spark.
Submit a spark job with the run.sh script, for full details see the user guide in the yunikorn-core.
Deployment files for the driver and executor:
A simple example that runs a kubeflow/tensorflow job. In this example it will run a distributed mnist model for e2e test, for full more detail see the dist-mnist section.
A sample application which implement gang in application level. Start via the gangDeploy.sh script, for full more detail see the gang section.
Deployment file for gang-coordinator and gang-job:
The volumes directory contains three examples:
All examples will generate an unending stream of data in a file called dates.txt
on the mounted volume. Run times should be limited or the disk usage must be monitored for longer runs.
kubectl get services | grep nfs-server | awk '{print $3}'Use the cluster IP address returned to update the client pod yaml for the next step.
NOTE: The NFS server will not work on a Docker Desktop in a Mac OS-X environment. Even with kubernetes turned on. The image used by Docker Desktop does not support the kernel NFS driver used by the server. Use minikube as a workaround and follow the environment setup to add it to Docker Desktop.
EBS volumes cannot be used outside of AWS. Therefor you must have an EKS cluster on AWS. Pre-requisites and installation that is not described in this document:
Three example cases:
--enable-admission-plugins DefaultStorageClass
set. see Dynamic Volume Provisioning in the kubernetes docs.NOTE: dynamic provisioning can interfere with the existing volume examples given.
The namespace example uses a placement rule and special queue configuration. The pods are a simple sleep pod which will be scheduled based on the namespace it runs in. The pod specification does not have a queue set.
kubectl create -f namespaces.yaml
development
queue using the local sleeppod_dev.yaml: kubectl create -f sleeppod_dev.yaml
. The pod spec does not specify a queue just a namespace but the application will be run in the root.development
queue.production
queue using the local sleeppod_prod.yaml: kubectl create -f namespaces.yaml
. The pod spec does not specify a queue just a namespace but the application will be run in the newly created root.production
queue. This queue does not exist in the queue configuration.