| :py:mod:`airflow.providers.apache.pig.operators.pig` |
| ==================================================== |
| |
| .. py:module:: airflow.providers.apache.pig.operators.pig |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.apache.pig.operators.pig.PigOperator |
| |
| |
| |
| |
| .. py:class:: PigOperator(*, pig, pig_cli_conn_id = 'pig_cli_default', pigparams_jinja_translate = False, pig_opts = None, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Executes pig script. |
| |
| :param pig: the pig latin script to be executed. (templated) |
| :param pig_cli_conn_id: reference to the Hive database |
| :param pigparams_jinja_translate: when True, pig params-type templating |
| ${var} gets translated into jinja-type templating {{ var }}. Note that |
| you may want to use this along with the |
| ``DAG(user_defined_macros=myargs)`` parameter. View the DAG |
| object documentation for more details. |
| :param pig_opts: pig options, such as: -x tez, -useHCatalog, ... |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['pig'] |
| |
| |
| |
| .. py:attribute:: template_ext |
| :annotation: :Sequence[str] = ['.pig', '.piglatin'] |
| |
| |
| |
| .. py:attribute:: ui_color |
| :annotation: = #f0e4ec |
| |
| |
| |
| .. py:method:: prepare_template(self) |
| |
| Hook triggered after the templated fields get replaced by their content. |
| |
| If you need your operator to alter the content of the file before the |
| template is rendered, it should override this method to do so. |
| |
| |
| .. py:method:: execute(self, context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| .. py:method:: on_kill(self) |
| |
| Override this method to cleanup subprocesses when a task instance |
| gets killed. Any use of the threading, subprocess or multiprocessing |
| module within an operator needs to be cleaned up or it will leave |
| ghost processes behind. |
| |
| |
| |
