| :py:mod:`airflow.providers.plexus.operators.job` |
| ================================================ |
| |
| .. py:module:: airflow.providers.plexus.operators.job |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.plexus.operators.job.PlexusJobOperator |
| |
| |
| |
| |
| Attributes |
| ~~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.plexus.operators.job.logger |
| |
| |
| .. py:data:: logger |
| |
| |
| |
| |
| .. py:class:: PlexusJobOperator(job_params, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Submits a Plexus job. |
| |
| :param job_params: parameters required to launch a job. |
| |
| Required job parameters are the following |
| - "name": job name created by user. |
| - "app": name of the application to run. found in Plexus UI. |
| - "queue": public cluster name. found in Plexus UI. |
| - "num_nodes": number of nodes. |
| - "num_cores": number of cores per node. |
| |
| |
| .. py:method:: execute(self, context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| .. py:method:: construct_job_params(self, hook) |
| |
| Creates job_params dict for api call to |
| launch a Plexus job. |
| |
| Some parameters required to launch a job |
| are not available to the user in the Plexus |
| UI. For example, an app id is required, but |
| only the app name is provided in the UI. |
| This function acts as a backend lookup |
| of the required param value using the |
| user-provided value. |
| |
| :param hook: plexus hook object |
| |
| |
| |
