| |
| |
| |
| |
| <!DOCTYPE html> |
| <!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]--> |
| <!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]--> |
| <head> |
| <meta charset="utf-8"> |
| |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| |
| <title>Changelog — Airflow Documentation</title> |
| |
| |
| |
| |
| |
| |
| |
| |
| <script type="text/javascript" src="_static/js/modernizr.min.js"></script> |
| |
| |
| <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script> |
| <script type="text/javascript" src="_static/jquery.js"></script> |
| <script type="text/javascript" src="_static/underscore.js"></script> |
| <script type="text/javascript" src="_static/doctools.js"></script> |
| <script type="text/javascript" src="_static/language_data.js"></script> |
| |
| <script type="text/javascript" src="_static/js/theme.js"></script> |
| |
| |
| |
| |
| <link rel="stylesheet" href="_static/css/theme.css" type="text/css" /> |
| <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> |
| <link rel="index" title="Index" href="genindex.html" /> |
| <link rel="search" title="Search" href="search.html" /> |
| <link rel="next" title="FAQ" href="faq.html" /> |
| <link rel="prev" title="Lineage" href="lineage.html" /> |
| |
| <script> |
| document.addEventListener('DOMContentLoaded', function() { |
| var el = document.getElementById('changelog'); |
| if (el !== null ) { |
| // [AIRFLOW-...] |
| el.innerHTML = el.innerHTML.replace( |
| /\[(AIRFLOW-[\d]+)\]/g, |
| `<a href="https://issues.apache.org/jira/browse/$1">[$1]</a>` |
| ); |
| // (#...) |
| el.innerHTML = el.innerHTML.replace( |
| /\(#([\d]+)\)/g, |
| `<a href="https://github.com/apache/airflow/pull/$1">(#$1)</a>` |
| ); |
| }; |
| }) |
| </script> |
| <style> |
| .example-header { |
| position: relative; |
| background: #9AAA7A; |
| padding: 8px 16px; |
| margin-bottom: 0; |
| } |
| .example-header--with-button { |
| padding-right: 166px; |
| } |
| .example-header:after{ |
| content: ''; |
| display: table; |
| clear: both; |
| } |
| .example-title { |
| display:block; |
| padding: 4px; |
| margin-right: 16px; |
| color: white; |
| overflow-x: auto; |
| } |
| .example-header-button { |
| top: 8px; |
| right: 16px; |
| position: absolute; |
| } |
| .example-header + .highlight-python { |
| margin-top: 0 !important; |
| } |
| .viewcode-button { |
| display: inline-block; |
| padding: 8px 16px; |
| border: 0; |
| margin: 0; |
| outline: 0; |
| border-radius: 2px; |
| -webkit-box-shadow: 0 3px 5px 0 rgba(0,0,0,.3); |
| box-shadow: 0 3px 6px 0 rgba(0,0,0,.3); |
| color: #404040; |
| background-color: #e7e7e7; |
| cursor: pointer; |
| font-size: 16px; |
| font-weight: 500; |
| line-height: 1; |
| text-decoration: none; |
| text-overflow: ellipsis; |
| overflow: hidden; |
| text-transform: uppercase; |
| -webkit-transition: background-color .2s; |
| transition: background-color .2s; |
| vertical-align: middle; |
| white-space: nowrap; |
| } |
| .viewcode-button:visited { |
| color: #404040; |
| } |
| .viewcode-button:hover, .viewcode-button:focus { |
| color: #404040; |
| background-color: #d6d6d6; |
| } |
| </style> |
| <!-- Matomo --> |
| <script> |
| var _paq = window._paq = window._paq || []; |
| _paq.push(['disableCookies']); |
| _paq.push(['trackPageView']); |
| _paq.push(['enableLinkTracking']); |
| (function() { |
| var u="https://analytics.apache.org/"; |
| _paq.push(['setTrackerUrl', u+'matomo.php']); |
| _paq.push(['setSiteId', '13']); |
| var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; |
| g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); |
| })(); |
| </script> |
| <!-- End Matomo --> |
| <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow/stable/changelog.html" /> |
| </head> |
| |
| |
| <body class="wy-body-for-nav"> |
| |
| |
| <div class="wy-grid-for-nav"> |
| |
| <nav data-toggle="wy-nav-shift" class="wy-nav-side"> |
| <div class="wy-side-scroll"> |
| <div class="wy-side-nav-search" > |
| |
| |
| |
| <a href="index.html" class="icon icon-home"> Airflow |
| |
| |
| |
| </a> |
| |
| |
| |
| |
| <div class="version"> |
| 1.10.4 |
| </div> |
| |
| |
| |
| |
| <div role="search"> |
| <form id="rtd-search-form" class="wy-form" action="search.html" method="get"> |
| <input type="text" name="q" placeholder="Search docs" /> |
| <input type="hidden" name="check_keywords" value="yes" /> |
| <input type="hidden" name="area" value="default" /> |
| </form> |
| </div> |
| |
| |
| </div> |
| |
| <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> |
| |
| |
| |
| |
| |
| |
| <ul class="current"> |
| <li class="toctree-l1"><a class="reference internal" href="project.html">Project</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="license.html">License</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="start.html">Quick Start</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="howto/index.html">How-to Guides</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="ui.html">UI / Screenshots</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="concepts.html">Concepts</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="profiling.html">Data Profiling</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="cli.html">Command Line Interface</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="scheduler.html">Scheduling & Triggers</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="plugins.html">Plugins</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="security.html">Security</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="timezone.html">Time zones</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="api.html">Experimental Rest API</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="integration.html">Integration</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="metrics.html">Metrics</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="kubernetes.html">Kubernetes</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="lineage.html">Lineage</a></li> |
| <li class="toctree-l1 current"><a class="current reference internal" href="#">Changelog</a><ul> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-10-4-2019-08-06">Airflow 1.10.4, - 2019-08-06</a><ul> |
| <li class="toctree-l3"><a class="reference internal" href="#new-features">New Features</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#improvement">Improvement</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#bug-fixes">Bug fixes</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#misc-internal">Misc/Internal</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#doc-only-changes">Doc-only changes</a></li> |
| </ul> |
| </li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-10-3-2019-04-09">Airflow 1.10.3, 2019-04-09</a><ul> |
| <li class="toctree-l3"><a class="reference internal" href="#new-feature">New Feature</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id1">Improvement</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id2">Bug fixes</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id3">Misc/Internal</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id4">Doc-only changes</a></li> |
| </ul> |
| </li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-10-2-2019-01-19">Airflow 1.10.2, 2019-01-19</a><ul> |
| <li class="toctree-l3"><a class="reference internal" href="#id5">New features</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#improvements">Improvements</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id6">Bug fixes</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id7">Doc-only changes</a></li> |
| </ul> |
| </li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-10-1-2018-11-13">Airflow 1.10.1, 2018-11-13</a><ul> |
| <li class="toctree-l3"><a class="reference internal" href="#id8">New features</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id9">Improvements</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id10">Doc-only changes</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#id11">Bug fixes</a></li> |
| </ul> |
| </li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-10-0-2018-08-03">Airflow 1.10.0, 2018-08-03</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-9-0-2018-01-02">Airflow 1.9.0, 2018-01-02</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-8-2-2017-09-04">Airflow 1.8.2, 2017-09-04</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-8-1-2017-05-09">Airflow 1.8.1, 2017-05-09</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-8-0-2017-03-12">Airflow 1.8.0, 2017-03-12</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-7-2">Airflow 1.7.2</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="#airflow-1-7-1-2016-05-19">Airflow 1.7.1, 2016-05-19</a></li> |
| </ul> |
| </li> |
| <li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="macros.html">Macros reference</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="_api/index.html">API Reference</a></li> |
| </ul> |
| |
| |
| |
| </div> |
| </div> |
| </nav> |
| |
| <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> |
| |
| |
| <nav class="wy-nav-top" aria-label="top navigation"> |
| |
| <i data-toggle="wy-nav-top" class="fa fa-bars"></i> |
| <a href="index.html">Airflow</a> |
| |
| </nav> |
| |
| |
| <div class="wy-nav-content"> |
| |
| <div class="rst-content"> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| <div role="navigation" aria-label="breadcrumbs navigation"> |
| |
| <ul class="wy-breadcrumbs"> |
| |
| <li><a href="index.html">Docs</a> »</li> |
| |
| <li>Changelog</li> |
| |
| |
| <li class="wy-breadcrumbs-aside"> |
| |
| |
| <a href="_sources/changelog.rst.txt" rel="nofollow"> View page source</a> |
| |
| |
| </li> |
| |
| </ul> |
| |
| |
| <hr/> |
| </div> |
| <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> |
| <div itemprop="articleBody"> |
| |
| <div class="section" id="changelog"> |
| <h1>Changelog<a class="headerlink" href="#changelog" title="Permalink to this headline">¶</a></h1> |
| <div class="section" id="airflow-1-10-4-2019-08-06"> |
| <h2>Airflow 1.10.4, - 2019-08-06<a class="headerlink" href="#airflow-1-10-4-2019-08-06" title="Permalink to this headline">¶</a></h2> |
| <div class="section" id="new-features"> |
| <h3>New Features<a class="headerlink" href="#new-features" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-4811] Implement GCP Data Loss Prevention Hook and Operators (#5539)</p></li> |
| <li><p>[AIRFLOW-5035] Replace multiprocessing.Manager with a golang-“channel” style (#5615)</p></li> |
| <li><p>[AIRFLOW-4883] Kill hung file process managers (#5605)</p></li> |
| <li><p>[AIRFLOW-4929] Pretty print JSON Variables in UI (#5573)</p></li> |
| <li><p>[AIRFLOW-4884] Roll up import_errors in RBAC UI (#5516)</p></li> |
| <li><p>[AIRFLOW-4871] Allow creating DagRuns via RBAC UI (#5507)</p></li> |
| <li><p>[AIRFLOW-4591] Make default_pool a real pool (#5349)</p></li> |
| <li><p>[AIRFLOW-4844] Add optional is_paused_upon_creation argument to DAG (#5473)</p></li> |
| <li><p>[AIRFLOW-4456] Add sub-classable BaseBranchOperator (#5231)</p></li> |
| <li><p>[AIRFLOW-4343] Show warning in UI if scheduler is not running (#5127)</p></li> |
| <li><p>[AIRFLOW-4739] Add ability to arbitrarily define kubernetes worker pod labels (#5376)</p></li> |
| <li><p>[AIRFLOW-4348] Add GCP console link in BigQueryOperator (#5195)</p></li> |
| <li><p>[AIRFLOW-4306] Global operator extra links (#5094)</p></li> |
| <li><p>[AIRFLOW-4812] Add batch images annotation (#5433)</p></li> |
| <li><p>[AIRFLOW-4135] Add Google Cloud Build operator and hook (#5251)</p></li> |
| <li><p>[AIRFLOW-4781] Add the ability to specify ports in kubernetesOperator (#5410)</p></li> |
| <li><p>[AIRFLOW-4521] Pause dag also pause its subdags (#5283)</p></li> |
| <li><p>[AIRFLOW-4738] Enforce exampleinclude for example DAGs (#5375)</p></li> |
| <li><p>[AIRFLOW-4326] Airflow AWS SQS Operator (#5110)</p></li> |
| <li><p>[AIRFLOW-3729] Support “DownwardAPI” in env variables for KubernetesPodOperator (#4554)</p></li> |
| <li><p>[AIRFLOW-4585] Implement Kubernetes Pod Mutation Hook (#5359)</p></li> |
| <li><p>[AIRFLOW-161] New redirect route and extra links (#5059)</p></li> |
| <li><p>[AIRFLOW-4420] Backfill respects task_concurrency (#5221)</p></li> |
| <li><p>[AIRFLOW-4147] Add Operator to publish event to Redis (#4967)</p></li> |
| <li><p>[AIRFLOW-3359] Add option to pass customer encryption keys to Dataproc (#4200)</p></li> |
| <li><p>[AIRFLOW-4318] Create Google Cloud Translate Speech Operator (#5102)</p></li> |
| <li><p>[AIRFLOW-3960] Adds Google Cloud Speech operators (#4780)</p></li> |
| <li><p>[AIRFLOW-1501] Add GoogleCloudStorageDeleteOperator (#5230)</p></li> |
| <li><p>[AIRFLOW-3672] Add support for Mongo DB DNS Seedlist Connection Format (#4481)</p></li> |
| <li><p>[AIRFLOW-4397] add integrations docs manually for gcs sensors (#5204)</p></li> |
| <li><p>[AIRFLOW-4251] Instrument DagRun schedule delay (#5050)</p></li> |
| <li><p>[AIRFLOW-4118] instrument DagRun duration (#4946)</p></li> |
| <li><p>[AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5182)</p></li> |
| <li><p>[AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5140)</p></li> |
| <li><p>[AIRFLOW-4168] Create Google Cloud Video Intelligence Operators (#4985)</p></li> |
| <li><p>[AIRFLOW-4397] Add GCSUploadSessionCompleteSensor (#5166)</p></li> |
| <li><p>[AIRFLOW-4335] Add default num_retries to GCP connection (#5117)</p></li> |
| <li><p>[AIRFLOW-3808] Add cluster_fields to BigQueryHook’s create_empty_table (#4654)</p></li> |
| <li><p>[AIRFLOW-4362] Fix test_execution_limited_parallelism (#5141)</p></li> |
| <li><p>[AIRFLOW-4307] Backfill respects concurrency limit (#5128)</p></li> |
| <li><p>[AIRFLOW-4268] Add MsSqlToGoogleCloudStorageOperator (#5077)</p></li> |
| <li><p>[AIRFLOW-4169] Add Google Cloud Vision Detect Operators (#4986)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix WS-2019-0032 (#5384)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix CVE-2019-11358 (#5197)</p></li> |
| <li><p>[AIRFLOW-XXX] Change allowed version of Jinja2 to fix CVE-2019-10906 (#5075)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="improvement"> |
| <h3>Improvement<a class="headerlink" href="#improvement" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-5022] Fix DockerHook for registries with port numbers (#5644)</p></li> |
| <li><p>[AIRFLOW-4961] Insert TaskFail.duration as int match DB schema column type (#5593)</p></li> |
| <li><p>[AIRFLOW-5038] skip pod deleted log message when pod deletion is disabled (#5656)</p></li> |
| <li><p>[AIRFLOW-5067] Update pagination symbols (#5682)</p></li> |
| <li><p>[AIRFLOW-4981][AIRFLOW-4788] Always use pendulum DateTimes in task instance context (#5654)</p></li> |
| <li><p>[AIRFLOW-4880] Add success, failure and fail_on_empty params to SqlSensor (#5488)</p></li> |
| <li><p>[AIRFLOW-3617] Add gpu limits option in configurations for Kube executor and pod (#5643)</p></li> |
| <li><p>[AIRFLOW-4998] Run multiple queries in BigQueryOperator (#5619)</p></li> |
| <li><p>[AIRFLOW-4929] Improve display of JSON Variables in UI (#5641)</p></li> |
| <li><p>[AIRFLOW-4959] Add .hql support for the DataProcHiveOperator (#5591)</p></li> |
| <li><p>[AIRFLOW-4962] Fix Werkzeug v0.15 deprecation notice for DispatcherMiddleware import (#5595)</p></li> |
| <li><p>[AIRFLOW-4797] Improve performance and behaviour of zombie detection (#5511)</p></li> |
| <li><p>[AIRFLOW-4911] Silence the FORBIDDEN errors from the KubernetesExecutor (#5547)</p></li> |
| <li><p>[AIRFLOW-3495] Validate one of query and query_uri passed to DataProcSparkSqlOperator (#5510)</p></li> |
| <li><p>[AIRFLOW-4925] Improve css style for Variables Import file field (#5552)</p></li> |
| <li><p>[AIRFLOW-4906] Improve debugging for the SparkSubmitHook (#5542)</p></li> |
| <li><p>[AIRFLOW-4904] unittest.cfg name and path can be overriden by setting $AIRFLOW_TEST_CONFIG (#5540)</p></li> |
| <li><p>[AIRFLOW-4920] Use html.escape instead of cgi.escape to fix DeprecationWarning (#5551)</p></li> |
| <li><p>[AIRFLOW-4919] DataProcJobBaseOperator dataproc_*_properties templated (#5555)</p></li> |
| <li><p>[AIRFLOW-4478] Lazily instantiate default resources objects. (#5259)</p></li> |
| <li><p>[AIRFLOW-4564] AzureContainerInstance bugfixes and improvements (#5319)</p></li> |
| <li><p>[AIRFLOW-4237] Including Try Number of Task in Gantt Chart (#5037)</p></li> |
| <li><p>[AIRFLOW-4862] Allow directly using IP address as hostname for webserver logs (#5501)</p></li> |
| <li><p>[AIRFLOW-4857] Add templated fields to SlackWebhookOperator (#5490)</p></li> |
| <li><p>[AIRFLOW-3502] Add celery config option for setting “pool” (#4308)</p></li> |
| <li><p>[AIRFLOW-3217] Button to toggle line wrapping in log and code views (#4277)</p></li> |
| <li><p>[AIRFLOW-4491] Add a “Jump to end” button for logs (#5266)</p></li> |
| <li><p>[AIRFLOW-4422] Pool utilization stats (#5453)</p></li> |
| <li><p>[AIRFLOW-4805] Add py_file as templated field in DataflowPythonOperator (#5451)</p></li> |
| <li><p>[AIRFLOW-4838] Surface Athena errors in AWSAthenaOperator (#5467)</p></li> |
| <li><p>[AIRFLOW-4831] conf.has_option no longer throws if section is missing. (#5455)</p></li> |
| <li><p>[AIRFLOW-4829] More descriptive exceptions for EMR sensors (#5452)</p></li> |
| <li><p>[AIRFLOW-4414] AWSAthenaOperator: Push QueryExecutionID to XCom (#5276)</p></li> |
| <li><p>[AIRFLOW-4791] add “schema” keyword arg to SnowflakeOperator (#5415)</p></li> |
| <li><p>[AIRFLOW-4759] Don’t error when marking sucessful run as failed (#5435)</p></li> |
| <li><p>[AIRFLOW-4716] Instrument dag loading time duration (#5350)</p></li> |
| <li><p>[AIRFLOW-3958] Support list tasks as upstream in chain (#4779)</p></li> |
| <li><p>[AIRFLOW-4409] Prevent task duration break by null value (#5178)</p></li> |
| <li><p>[AIRFLOW-4418] Add “failed only” option to task modal (#5193)</p></li> |
| <li><p>[AIRFLOW-4740] Accept string <code class="docutils literal notranslate"><span class="pre">end_date</span></code> in DAG default_args (#5381)</p></li> |
| <li><p>[AIRFLOW-4423] Improve date handling in mysql to gcs operator. (#5196)</p></li> |
| <li><p>[AIRFLOW-4447] Display task duration as human friendly format in UI (#5218)</p></li> |
| <li><p>[AIRFLOW-4377] Remove needless object conversion in DAG.owner() (#5144)</p></li> |
| <li><p>[AIRFLOW-4766] Add autoscaling option for DataprocClusterCreateOperator (#5425)</p></li> |
| <li><p>[AIRFLOW-4795] Upgrade alembic to latest release. (#5411)</p></li> |
| <li><p>[AIRFLOW-4793] Add signature_name to mlengine operator (#5417)</p></li> |
| <li><p>[AIRFLOW-3211] Reattach to GCP Dataproc jobs upon Airflow restart (#4083)</p></li> |
| <li><p>[AIRFLOW-4750] Log identified zombie task instances (#5389)</p></li> |
| <li><p>[AIRFLOW-3870] STFPOperator: Update log level and return value (#4355)</p></li> |
| <li><p>[AIRFLOW-4759] Batch queries in set_state API. (#5403)</p></li> |
| <li><p>[AIRFLOW-2737] Restore original license header to airflow.api.auth.backend.kerberos_auth</p></li> |
| <li><p>[AIRFLOW-3635] Fix incorrect logic in detele_dag (introduced in PR#4406) (#4445)</p></li> |
| <li><p>[AIRFLOW-3599] Removed Dagbag from delete dag (#4406)</p></li> |
| <li><p>[AIRFLOW-4737] Increase and document celery queue name limit (#5383)</p></li> |
| <li><p>[AIRFLOW-4505] Correct Tag ALL for PY3 (#5275)</p></li> |
| <li><p>[AIRFLOW-4743] Add environment variables support to SSHOperator (#5385)</p></li> |
| <li><p>[AIRFLOW-4725] Fix setup.py PEP440 & Sphinx-PyPI-upload dependency (#5363)</p></li> |
| <li><p>[AIRFLOW-3370] Add stdout output options to Elasticsearch task log handler (#5048)</p></li> |
| <li><p>[AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. (#5164)</p></li> |
| <li><p>[AIRFLOW-1381] Allow setting host temporary directory in DockerOperator (#5369)</p></li> |
| <li><p>[AIRFLOW-4598] Task retries are not exhausted for K8s executor (#5347)</p></li> |
| <li><p>[AIRFLOW-4218] Support to Provide http args to K8executor while calling k8 python client lib apis (#5060)</p></li> |
| <li><p>[AIRFLOW-4159] Add support for additional static pod labels for K8sExecutor (#5134)</p></li> |
| <li><p>[AIRFLOW-4720] Allow comments in .airflowignore files. (#5355)</p></li> |
| <li><p>[AIRFLOW-4486] Add AWS IAM authenication in MySqlHook (#5334)</p></li> |
| <li><p>[AIRFLOW-4417] Add AWS IAM authenication for PostgresHook (#5223)</p></li> |
| <li><p>[AIRFLOW-3990] Compile regular expressions. (#4813)</p></li> |
| <li><p>[AIRFLOW-4572] Rename prepare_classpath() to prepare_syspath() (#5328)</p></li> |
| <li><p>[AIRFLOW-3869] Raise consistent exception in AirflowConfigParser.getboolean (#4692)</p></li> |
| <li><p>[AIRFLOW-4571] Add headers to templated field for SimpleHttpOperator (#5326)</p></li> |
| <li><p>[AIRFLOW-3867] Rename GCP’s subpackage (#4690)</p></li> |
| <li><p>[AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df (#4549)</p></li> |
| <li><p>[AIRFLOW-4546] Upgrade google-cloud-bigtable. (#5307)</p></li> |
| <li><p>[AIRFLOW-4519] Optimise operator classname sorting in views (#5282)</p></li> |
| <li><p>[AIRFLOW-4503] Support fully pig options (#5271)</p></li> |
| <li><p>[AIRFLOW-4468] add sql_alchemy_max_overflow parameter (#5249)</p></li> |
| <li><p>[AIRFLOW-4467] Add dataproc_jars to templated fields in Dataproc oper… (#5248)</p></li> |
| <li><p>[AIRFLOW-4381] Use get_direct_relative_ids get task relatives (#5147)</p></li> |
| <li><p>[AIRFLOW-3624] Add masterType parameter to MLEngineTrainingOperator (#4428)</p></li> |
| <li><p>[AIRFLOW-3143] Support Auto-Zone in DataprocClusterCreateOperator (#5169)</p></li> |
| <li><p>[AIRFLOW-3874] Improve BigQueryHook.run_with_configuration’s location support (#4695)</p></li> |
| <li><p>[AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag (#5165)</p></li> |
| <li><p>[AIRFLOW-4031] Allow for key pair auth in snowflake hook (#4875)</p></li> |
| <li><p>[AIRFLOW-3901] add role as optional config parameter for SnowflakeHook (#4721)</p></li> |
| <li><p>[AIRFLOW-3455] add region in snowflake connector (#4285)</p></li> |
| <li><p>[AIRFLOW-4073] add template_ext for AWS Athena operator (#4907)</p></li> |
| <li><p>[AIRFLOW-4093] AWSAthenaOperator: Throw exception if job failed/cancelled/reach max retries (#4919)</p></li> |
| <li><p>[AIRFLOW-4356] Add extra RuntimeEnvironment keys to DataFlowHook (#5149)</p></li> |
| <li><p>[AIRFLOW-4337] Fix docker-compose deprecation warning in CI (#5119)</p></li> |
| <li><p>[AIRFLOW-3603] QuboleOperator: Remove SQLCommand from SparkCmd documentation (#4411)</p></li> |
| <li><p>[AIRFLOW-4328] Fix link to task instances from Pool page (#5124)</p></li> |
| <li><p>[AIRFLOW-4255] Make GCS Hook Backwards compatible (#5089)</p></li> |
| <li><p>[AIRFLOW-4103] Allow uppercase letters in dataflow job names (#4925)</p></li> |
| <li><p>[AIRFLOW-4255] Replace Discovery based api with client based for GCS (#5054)</p></li> |
| <li><p>[AIRFLOW-4311] Remove sleep in localexecutor (#5096)</p></li> |
| <li><p>[AIRFLOW-2836] Minor improvement-contrib.sensors.FileSensor (#3674)</p></li> |
| <li><p>[AIRFLOW-4104] Add type annotations to common classes. (#4926)</p></li> |
| <li><p>[AIRFLOW-3910] Raise exception explicitly in Connection.get_hook() (#4728)</p></li> |
| <li><p>[AIRFLOW-3322] Update QuboleHook to fetch args dynamically from qds_sdk (#4165)</p></li> |
| <li><p>[AIRFLOW-4565] instrument celery executor (#5321)</p></li> |
| <li><p>[AIRFLOW-4573] Import airflow_local_settings after prepare_classpath (#5330)</p></li> |
| <li><p>[AIRFLOW-4448] Don’t bake ENV and _cmd into tmp config for non-sudo (#4050)</p></li> |
| <li><p>[AIRFLOW-4295] Make <code class="docutils literal notranslate"><span class="pre">method</span></code> attribute case insensitive in HttpHook (#5313)</p></li> |
| <li><p>[AIRFLOW-3703] Add dnsPolicy option for KubernetesPodOperator (#4520)</p></li> |
| <li><p>[AIRFLOW-3057] add prev_*_date_success to template context (#5372)</p></li> |
| <li><p>[AIRFLOW-4336] Stop showing entire GCS files bytes in log for gcs_download_operator (#5151)</p></li> |
| <li><p>[AIRFLOW-4528] Cancel DataProc task on timeout (#5293)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="bug-fixes"> |
| <h3>Bug fixes<a class="headerlink" href="#bug-fixes" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-5089] Change version requirement on google-cloudp-spanner to work around version incompatibility (#5703)</p></li> |
| <li><p>[AIRFLOW-4289] fix spark_binary argument being ignored in SparkSubmitHook (#5564)</p></li> |
| <li><p>[AIRFLOW-5075] Let HttpHook handle connections with empty host fields (#5686)</p></li> |
| <li><p>[AIRFLOW-4822] Fix bug where parent-dag task instances are wrongly cleared when using subdags (#5444)</p></li> |
| <li><p>[AIRFLOW-5050] Correctly delete FAB permission m2m objects in <code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">sync_perms</span></code> (#5679)</p></li> |
| <li><p>[AIRFLOW-5030] fix env var expansion for config key contains __ (#5650)</p></li> |
| <li><p>[AIRFLOW-4590] changing log level to be proper library to suppress warning in WinRM (#5337)</p></li> |
| <li><p>[AIRFLOW-4451] Allow named tuples to be templated (#5673)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix bug where Kube pod limts were not applied (requests were, but not limits) (#5657)</p></li> |
| <li><p>[AIRFLOW-4775] Fix incorrect parameter order in GceHook (#5613)</p></li> |
| <li><p>[AIRFLOW-4995] Fix DB initialisation on MySQL >=8.0.16 (#5614)</p></li> |
| <li><p>[AIRFLOW-4934] Fix ProxyFix due to Werkzeug upgrade (#5563) (#5571)</p></li> |
| <li><p>[AIRFLOW-4136] fix key_file of hook is overwritten by SSHHook connection (#5558)</p></li> |
| <li><p>[AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5545)</p></li> |
| <li><p>[AIRFLOW-1740] Fix xcom creation and update via UI (#5530) (#5531)</p></li> |
| <li><p>[AIRFLOW-4900] Resolve incompatible version of Werkzeug (#5535)</p></li> |
| <li><p>[AIRFLOW-4510] Don’t mutate default_args during DAG initialization (#5277)</p></li> |
| <li><p>[AIRFLOW-3360] Make the DAGs search respect other querystring parameters with url-search-params-polyfill for IE support (#5503)</p></li> |
| <li><p>[AIRFLOW-4896] Make KubernetesExecutorConfig’s default args immutable (#5534)</p></li> |
| <li><p>[AIRFLOW-4494] Remove <code class="docutils literal notranslate"><span class="pre">shell=True</span></code> in DaskExecutor (#5273)</p></li> |
| <li><p>[AIRFLOW-4890] Fix Log link in TaskInstance’s View for Non-RBAC (#5525)</p></li> |
| <li><p>[AIRFLOW-4892] Fix connection creation via UIs (#5527)</p></li> |
| <li><p>[AIRFLOW-4406] Fix a method name typo: NullFernet.decrpyt to decrypt (#5509)</p></li> |
| <li><p>[AIRFLOW-4849] Add gcp_conn_id to cloudsqldatabehook class to use correctly CloudSqlProxyRunner class (#5478)</p></li> |
| <li><p>[AIRFLOW-4769] Pass gcp_conn_id to BigtableHook (#5445)</p></li> |
| <li><p>[AIRFLOW-4524] Fix incorrect field names in view for Mark Success/Failure (#5486)</p></li> |
| <li><p>[AIRFLOW-3671] Remove arg <code class="docutils literal notranslate"><span class="pre">replace</span></code> of MongoToS3Operator from <code class="docutils literal notranslate"><span class="pre">kwargs</span></code> (#4480)</p></li> |
| <li><p>[AIRFLOW-4845] Fix bug where runAsUser 0 doesn’t get set in k8s security context (#5474)</p></li> |
| <li><p>[AIRFLOW-4354] Fix exception in “between” date filter in classic UI (#5480)</p></li> |
| <li><p>[AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5462)</p></li> |
| <li><p>[AIRFLOW-4516] K8s runAsUser and fsGroup cannot be strings (#5429)</p></li> |
| <li><p>[AIRFLOW-4298] Stop Scheduler repeatedly warning “connection invalidated” (#5470)</p></li> |
| <li><p>[AIRFLOW-4559] JenkinsJobTriggerOperator bugfix (#5318)</p></li> |
| <li><p>[AIRFLOW-4841] Pin Sphinx AutoApi to 1.0.0 (#5468)</p></li> |
| <li><p>[AIRFLOW-4479] Include s3_overwrite kwarg in load_bytes method (#5312)</p></li> |
| <li><p>[AIRFLOW-3746] Fix DockerOperator missing container exit (#4583)</p></li> |
| <li><p>[AIRFLOW-4233] Remove Template Extension from Bq to GCS Operator (#5456)</p></li> |
| <li><p>[AIRFLOW-2141][AIRFLOW-3157][AIRFLOW-4170] Serialize non-str value by JSON when importing Variables (#4991)</p></li> |
| <li><p>[AIRFLOW-4826] Remove warning from <code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">resetdb</span></code> command (#5447)</p></li> |
| <li><p>[AIRFLOW-4148] Fix editing DagRuns when clicking state column (#5436)</p></li> |
| <li><p>[AIRFLOW-4455] dag_details broken for subdags in RBAC UI (#5234)</p></li> |
| <li><p>[AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods (#4551)</p></li> |
| <li><p>[AIRFLOW-4459] Fix wrong DAG count in /home page when DAG count is zero (#5235)</p></li> |
| <li><p>[AIRFLOW-3876] AttributeError: module ‘distutils’ has no attribute ‘util’</p></li> |
| <li><p>[AIRFLOW-4146] Fix CgroupTaskRunner errors (#5224)</p></li> |
| <li><p>[AIRFLOW-4524] Fix bug with “Ignore *” toggles in RBAC mode (#5378)</p></li> |
| <li><p>[AIRFLOW-4765] Fix DataProcPigOperator execute method (#5426)</p></li> |
| <li><p>[AIRFLOW-4798] obviate interdependencies for dagbag and TI tests (#5422)</p></li> |
| <li><p>[AIRFLOW-4800] fix GKEClusterHook ctor calls (#5424)</p></li> |
| <li><p>[AIRFLOW-4799] don’t mutate self.env in BashOperator execute method (#5421)</p></li> |
| <li><p>[AIRFLOW-4393] Add retry logic when fetching pod status and/or logs in KubernetesPodOperator (#5284)</p></li> |
| <li><p>[AIRFLOW-4174] Fix HttpHook run with backoff (#5213)</p></li> |
| <li><p>[AIRFLOW-4463] Handle divide-by-zero errors in short retry intervals (#5243)</p></li> |
| <li><p>[AIRFLOW-2614] Speed up trigger_dag API call when lots of DAGs in system</p></li> |
| <li><p>[AIRFLOW-4756] add ti.state to ti.start_date as criteria for gantt (#5399)</p></li> |
| <li><p>[AIRFLOW-4760] Fix zip-packaged DAGs disappearing from DagBag when reloaded (#5404)</p></li> |
| <li><p>[AIRFLOW-4731] Fix GCS hook with google-storage-client 1.16 (#5368)</p></li> |
| <li><p>[AIRFLOW-3506] use match_phrase to query log_id in elasticsearch (#4342)</p></li> |
| <li><p>[AIRFLOW-4084] fix ElasticSearch log download (#5177)</p></li> |
| <li><p>[AIRFLOW-4501] Register pendulum datetime converter for sqla+pymysql (#5190)</p></li> |
| <li><p>[AIRFLOW-986] HiveCliHook ignores ‘proxy_user’ value in a connection’s extra parameter (#5305)</p></li> |
| <li><p>[AIRFLOW-4442] fix hive_tblproperties in HiveToDruidTransfer (#5211)</p></li> |
| <li><p>[AIRFLOW-4557] Add gcp_conn_id parameter to get_sqlproxy_runner() of CloudSqlDatabaseHook (#5314)</p></li> |
| <li><p>[AIRFLOW-4545] Upgrade FAB to latest version (#4955)</p></li> |
| <li><p>[AIRFLOW-4492] Change Dataproc Cluster operators to poll Operations (#5269)</p></li> |
| <li><p>[AIRFLOW-4452] Webserver and Scheduler keep crashing because of slackclient update (#5225)</p></li> |
| <li><p>[AIRFLOW-4450] Fix request arguments in has_dag_access (#5220)</p></li> |
| <li><p>[AIRFLOW-4434] Support Impala with the HiveServer2Hook (#5206)</p></li> |
| <li><p>[AIRFLOW-3449] Write local dag parsing logs when remote logging enabled. (#5175)</p></li> |
| <li><p>[AIRFLOW-4300] Fix graph modal call when DAG has not yet run (#5185)</p></li> |
| <li><p>[AIRFLOW-4401] Use managers for Queue synchronization (#5200)</p></li> |
| <li><p>[AIRFLOW-3626] Fixed triggering DAGs contained within zip files (#4439)</p></li> |
| <li><p>[AIRFLOW-3720] Fix missmatch while comparing GCS and S3 files (#4766)</p></li> |
| <li><p>[AIRFLOW-4403] search by <code class="docutils literal notranslate"><span class="pre">dag_id</span></code> or <code class="docutils literal notranslate"><span class="pre">owners</span></code> in UI (#5184)</p></li> |
| <li><p>[AIRFLOW-4308] Fix TZ-loop around DST on python 3.6+ (#5095)</p></li> |
| <li><p>[AIRFLOW-4324] fix DAG fuzzy search in RBAC UI (#5131)</p></li> |
| <li><p>[AIRFLOW-4297] Temporary hot fix on manage_slas() for 1.10.4 release (#5150)</p></li> |
| <li><p>[AIRFLOW-4299] Upgrade to Celery 4.3.0 to fix crashing workers (#5116)</p></li> |
| <li><p>[AIRFLOW-4291] Correctly render doc_md in DAG graph page (#5121)</p></li> |
| <li><p>[AIRFLOW-4310] Fix incorrect link on Dag Details page (#5122)</p></li> |
| <li><p>[AIRFLOW-4331] Correct filter for Null-state runs from Dag Detail page (#5123)</p></li> |
| <li><p>[AIRFLOW-4294] Fix missing dag & task runs in UI dag_id contains a dot (#5111)</p></li> |
| <li><p>[AIRFLOW-4332] Upgrade sqlalchemy to remove security Vulnerability (#5113)</p></li> |
| <li><p>[AIRFLOW-4312] Add template_fields & template_ext to BigQueryCheckO… (#5097)</p></li> |
| <li><p>[AIRFLOW-4293] Fix downgrade in d4ecb8fbee3_add_schedule_interval_to_dag.py (#5086)</p></li> |
| <li><p>[AIRFLOW-4267] Fix TI duration in Graph View (#5071)</p></li> |
| <li><p>[AIRFLOW-4163] IntervalCheckOperator supports relative diff and not ignore 0 (#4983)</p></li> |
| <li><p>[AIRFLOW-3938] QuboleOperator Fixes and Support for SqlCommand (#4832)</p></li> |
| <li><p>[AIRFLOW-2903] Change default owner to <code class="docutils literal notranslate"><span class="pre">airflow</span></code> (#4151)</p></li> |
| <li><p>[aIRFLOW-4136] Fix overwrite of key_file by constructor (#5155)</p></li> |
| <li><p>[AIRFLOW-3241] Remove Invalid template ext in GCS Sensors (#4076)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="misc-internal"> |
| <h3>Misc/Internal<a class="headerlink" href="#misc-internal" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-4338] Change k8s pod_request_factory to use yaml safe_load (#5120)</p></li> |
| <li><p>[AIRFLOW-4869] Reorganize sql to gcs operators. (#5504)</p></li> |
| <li><p>[AIRFLOW-5021] move gitpython into setup_requires (#5640)</p></li> |
| <li><p>[AIRFLOW-4583] Fixes type error in GKEPodOperator (#5612)</p></li> |
| <li><p>[AIRFLOW-4116] Dockerfile now supports CI image build on DockerHub (#4937)</p></li> |
| <li><p>[AIRFLOW-4115] Multi-staging Aiflow Docker image (#4936)</p></li> |
| <li><p>[AIRFLOW-4963] Avoid recreating task context (#5596)</p></li> |
| <li><p>[AIRFLOW-4865] Add context manager to set temporary config values in tests. (#5569)</p></li> |
| <li><p>[AIRFLOW-4937] Fix lodash security issue with version below 4.17.13 (#5572) (used only in build-pipeline, not runtime)</p></li> |
| <li><p>[AIRFLOW-4868] Fix typo in kubernetes/docker/build.sh (#5505)</p></li> |
| <li><p>[AIRFLOW-4211] Add tests for WebHDFSHook (#5015)</p></li> |
| <li><p>[AIRFLOW-4320] Add tests for SegmentTrackEventOperator (#5104)</p></li> |
| <li><p>[AIRFLOW-4319] Add tests for Bigquery related Operators (#5101)</p></li> |
| <li><p>[AIRFLOW-4014] Change DatastoreHook and add tests (#4842)</p></li> |
| <li><p>[AIRFLOW-4322] Add test for VerticaOperator (#5107)</p></li> |
| <li><p>[AIRFLOW-4323] Add 2 tests for WinRMOperator (#5108)</p></li> |
| <li><p>[AIRFLOW-3677] Improve CheckOperator test coverage (#4756)</p></li> |
| <li><p>[AIRFLOW-4659] Fix pylint problems for api module (#5398)</p></li> |
| <li><p>[AIRFLOW-4358] Speed up test_jobs by not running tasks (#5162)</p></li> |
| <li><p>[AIRFLOW-4394] Don’t test behaviour of BackfillJob from CLI tests (#5160)</p></li> |
| <li><p>[AIRFLOW-3471] Move XCom out of models.py (#4629)</p></li> |
| <li><p>[AIRFLOW-4379] Remove duplicate code & Add validation in gcs_to_gcs.py (#5145)</p></li> |
| <li><p>[AIRFLOW-4259] Move models out of models.py (#5056)</p></li> |
| <li><p>[AIRFLOW-XXX] Speed up building of Cassanda module on Travis (#5233)</p></li> |
| <li><p>[AIRFLOW-4535] Break jobs.py into multiple files (#5303)</p></li> |
| <li><p>[AIRFLOW-1464] Batch update task_instance state (#5323)</p></li> |
| <li><p>[AIRFLOW-4554] Test for sudo command, add some other test docs (#5310)</p></li> |
| <li><p>[AIRFLOW-4419] Refine concurrency check in scheduler (#5194)</p></li> |
| <li><p>[AIRFLOW-4269] Minor acceleration of jobs._process_task_instances() (#5076)</p></li> |
| <li><p>[AIRFLOW-4341] Remove <code class="docutils literal notranslate"><span class="pre">View.render()</span></code> already exists in fab.BaseView (#5125)</p></li> |
| <li><p>[AIRFLOW-4342] Use @cached_property instead of re-implementing it each time (#5126)</p></li> |
| <li><p>[AIRFLOW-4256] Remove noqa from migrations (#5055)</p></li> |
| <li><p>[AIRFLOW-4034] Remove unnecessary string formatting with <code class="docutils literal notranslate"><span class="pre">**locals()</span></code> (#4861)</p></li> |
| <li><p>[AIRFLOW-3944] Remove code smells (#4762)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="doc-only-changes"> |
| <h3>Doc-only changes<a class="headerlink" href="#doc-only-changes" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-XXX] Add missing doc for annotations param of KubernetesPodOperator (#5666)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix typos in CONTRIBUTING.md (#5626)</p></li> |
| <li><p>[AIRFLOW-XXX] Correct BaseSensorOperator docs (#5562)</p></li> |
| <li><p>[AIRFLOW-4926] Fix example dags where its start_date is datetime.utcnow() (#5553)</p></li> |
| <li><p>[AIRFLOW-4860] Remove Redundant Information in Example Dags (#5497)</p></li> |
| <li><p>[AIRFLOW-4767] Fix errors in the documentation of Dataproc Operator (#5487)</p></li> |
| <li><p>[AIRFLOW-1684] Branching based on XCom variable (Docs) (#4365)</p></li> |
| <li><p>[AIRFLOW-3341] FAQ return DAG object example (#4605)</p></li> |
| <li><p>[AIRFLOW-4433] Add missing type in DockerOperator doc string (#5205)</p></li> |
| <li><p>[AIRFLOW-4321] Replace incorrect info of Max Size limit of GCS Object Size (#5106)</p></li> |
| <li><p>[AIRFLOW-XXX] Add information about user list (#5341)</p></li> |
| <li><p>[AIRFLOW-XXX] Clarify documentation related to autodetect parameter in GCS_to_BQ Op (#5294)</p></li> |
| <li><p>[AIRFLOW-XXX] Remove mention of pytz compatibility from timezone documentation (#5316)</p></li> |
| <li><p>[AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator (#5274)</p></li> |
| <li><p>[AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection (#5242)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators (#5192)</p></li> |
| <li><p>[AIRFLOW-XXX] Link to correct class for timedelta in macros.rst (#5226)</p></li> |
| <li><p>[AIRFLOW-XXX] Add Kamil as committer (#5216)</p></li> |
| <li><p>[AIRFLOW-XXX] Add Joshua and Kevin as committer (#5207)</p></li> |
| <li><p>[AIRFLOW-XXX] Reduce log spam in tests (#5174)</p></li> |
| <li><p>[AIRFLOW-XXX] Speed up tests for PythonSensor (#5158)</p></li> |
| <li><p>[AIRFLOW-XXX] Add Bas Harenslak to committer list (#5157)</p></li> |
| <li><p>[AIRFLOW-XXX] Add Jarek Potiuk to committer list (#5132)</p></li> |
| <li><p>[AIRFLOW-XXX] Update docstring for SchedulerJob (#5105)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix docstrings for CassandraToGoogleCloudStorageOperator (#5103)</p></li> |
| <li><p>[AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator docstring (#5074)</p></li> |
| <li><p>[AIRFLOW-XXX] Ignore python files under node_modules in docs (#5063)</p></li> |
| <li><p>[AIRFLOW-XXX] Build a universal wheel with LICNESE files (#5052)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix docstrings of SQSHook (#5099)</p></li> |
| <li><p>[AIRFLOW-XXX] Use Py3.7 on readthedocs</p></li> |
| <li><p>[AIRFLOW-4446] Fix typos (#5217)</p></li> |
| </ul> |
| </div> |
| </div> |
| <div class="section" id="airflow-1-10-3-2019-04-09"> |
| <h2>Airflow 1.10.3, 2019-04-09<a class="headerlink" href="#airflow-1-10-3-2019-04-09" title="Permalink to this headline">¶</a></h2> |
| <div class="section" id="new-feature"> |
| <h3>New Feature<a class="headerlink" href="#new-feature" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-4232] Add <code class="docutils literal notranslate"><span class="pre">none_skipped</span></code> trigger rule (#5032)</p></li> |
| <li><p>[AIRFLOW-3971] Add Google Cloud Natural Language operators (#4980)</p></li> |
| <li><p>[AIRFLOW-4069] Add Opsgenie Alert Hook and Operator (#4903)</p></li> |
| <li><p>[AIRFLOW-3552] Fix encoding issue in ImapAttachmentToS3Operator (#5040)</p></li> |
| <li><p>[AIRFLOW-3552] Add ImapAttachmentToS3Operator (#4476)</p></li> |
| <li><p>[AIRFLOW-1526] Add dingding hook and operator (#4895)</p></li> |
| <li><p>[AIRFLOW-3490] Add BigQueryHook’s Ability to Patch Table/View (#4299)</p></li> |
| <li><p>[AIRFLOW-3918] Add ssh private-key support to git-sync for KubernetesExecutor (#4777)</p></li> |
| <li><p>[AIRFLOW-3659] Create Google Cloud Transfer Service Operators (#4792)</p></li> |
| <li><p>[AIRFLOW-3939] Add Google Cloud Translate operator (#4755)</p></li> |
| <li><p>[AIRFLOW-3541] Add Avro logical type conversion to bigquery hook (#4553)</p></li> |
| <li><p>[AIRFLOW-4106] instrument staving tasks in pool (#4927)</p></li> |
| <li><p>[AIRFLOW-2568] Azure Container Instances operator (#4121)</p></li> |
| <li><p>[AIRFLOW-4107] instrument executor (#4928)</p></li> |
| <li><p>[AIRFLOW-4033] record stats of task duration (#4858)</p></li> |
| <li><p>[AIRFLOW-3892] Create Redis pub sub sensor (#4712)</p></li> |
| <li><p>[AIRFLOW-4124] add get_table and get_table_location in aws_glue_hook and tests (#4942)</p></li> |
| <li><p>[AIRFLOW-1262] Adds missing docs for email configuration (#4557)</p></li> |
| <li><p>[AIRFLOW-3701] Add Google Cloud Vision Product Search operators (#4665)</p></li> |
| <li><p>[AIRFLOW-3766] Add support for kubernetes annotations (#4589)</p></li> |
| <li><p>[AIRFLOW-3741] Add extra config to Oracle hook (#4584)</p></li> |
| <li><p>[AIRFLOW-1262] Allow configuration of email alert subject and body (#2338)</p></li> |
| <li><p>[AIRFLOW-2985] Operators for S3 object copying/deleting (#3823)</p></li> |
| <li><p>[AIRFLOW-2993] s3_to_sftp and sftp_to_s3 operators (#3828)</p></li> |
| <li><p>[AIRFLOW-3799] Add compose method to GoogleCloudStorageHook (#4641)</p></li> |
| <li><p>[AIRFLOW-3218] add support for poking a whole DAG (#4058)</p></li> |
| <li><p>[AIRFLOW-3315] Add ImapAttachmentSensor (#4161)</p></li> |
| <li><p>[AIRFLOW-3556] Add cross join set dependency function (#4356)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id1"> |
| <h3>Improvement<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-4120] Modify SchedulerJob.manage_slas to respect zero timedelta SLAs (#4939)</p></li> |
| <li><p>[AIRFLOW-3823] Exclude branch’s downstream tasks from the tasks to skip (#4666)</p></li> |
| <li><p>[AIRFLOW-3274] Add run_as_user and fs_group options for Kubernetes (#4648)</p></li> |
| <li><p>[AIRFLOW-4247] Template Region on the DataprocOperators (#5046)</p></li> |
| <li><p>[AIRFLOW-4008] add envFrom for Kubernetes Executor (#4952)</p></li> |
| <li><p>[AIRFLOW-3947] Flash msg for no DAG-level access error (#4767)</p></li> |
| <li><p>[AIRFLOW-3287] Moving database clean-up code into the CoreTest.tearDown() (#4122)</p></li> |
| <li><p>[AIRFLOW-4058] Name models test file to get automatically picked up (#4901)</p></li> |
| <li><p>[AIRFLOW-3830] Remove DagBag from /dag_details (#4831)</p></li> |
| <li><p>[AIRFLOW-3596] Clean up undefined template variables. (#4401)</p></li> |
| <li><p>[AIRFLOW-3573] Remove DagStat table (#4378)</p></li> |
| <li><p>[AIRFLOW-3623] Fix bugs in Download task logs (#5005)</p></li> |
| <li><p>[AIRFLOW-4173] Improve SchedulerJob.process_file() (#4993)</p></li> |
| <li><p>[AIRFLOW-3540] Warn if old airflow.cfg file is found (#5006)</p></li> |
| <li><p>[AIRFLOW-4000] Return response when no file (#4822)</p></li> |
| <li><p>[AIRFLOW-3383] Rotate fernet keys. (#4225)</p></li> |
| <li><p>[AIRFLOW-3003] Pull the krb5 image instead of building (#3844)</p></li> |
| <li><p>[AIRFLOW-3862] Check types with mypy. (#4685)</p></li> |
| <li><p>[AIRFLOW-251] Add option SQL_ALCHEMY_SCHEMA parameter to specify schema for metadata (#4199)</p></li> |
| <li><p>[AIRFLOW-1814] Temple PythonOperator {op_args,op_kwargs} fields (#4691)</p></li> |
| <li><p>[AIRFLOW-3730] Standarization use of logs mechanisms (#4556)</p></li> |
| <li><p>[AIRFLOW-3770] Validation of documentation on CI] (#4593)</p></li> |
| <li><p>[AIRFLOW-3866] Run docker-compose pull silently in CI (#4688)</p></li> |
| <li><p>[AIRFLOW-3685] Move licence header check (#4497)</p></li> |
| <li><p>[AIRFLOW-3670] Add stages to Travis build (#4477)</p></li> |
| <li><p>[AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef and secretRef (#4772)</p></li> |
| <li><p>[AIRFLOW-3408] Remove outdated info from Systemd Instructions (#4269)</p></li> |
| <li><p>[AIRFLOW-3202] add missing documentation for AWS hooks/operator (#4048)</p></li> |
| <li><p>[AIRFLOW-3908] Add more Google Cloud Vision operators (#4791)</p></li> |
| <li><p>[AIRFLOW-2915] Add example DAG for GoogleCloudStorageToBigQueryOperator (#3763)</p></li> |
| <li><p>[AIRFLOW-3062] Add Qubole in integration docs (#3946)</p></li> |
| <li><p>[AIRFLOW-3288] Add SNS integration (#4123)</p></li> |
| <li><p>[AIRFLOW-3148] Remove unnecessary arg “parameters” in RedshiftToS3Transfer (#3995)</p></li> |
| <li><p>[AIRFLOW-3049] Add extra operations for Mongo hook (#3890)</p></li> |
| <li><p>[AIRFLOW-3559] Add missing options to DatadogHook. (#4362)</p></li> |
| <li><p>[AIRFLOW-1191] Simplify override of spark submit command. (#4360)</p></li> |
| <li><p>[AIRFLOW-3155] Add ability to filter by a last modified time in GCS Operator (#4008)</p></li> |
| <li><p>[AIRFLOW-2864] Fix docstrings for SubDagOperator (#3712)</p></li> |
| <li><p>[AIRFLOW-4062] Improve docs on install extra package commands (#4966)</p></li> |
| <li><p>[AIRFLOW-3743] Unify different methods of working out AIRFLOW_HOME (#4705)</p></li> |
| <li><p>[AIRFLOW-4002] Option to open debugger on errors in <cite>airflow test</cite>. (#4828)</p></li> |
| <li><p>[AIRFLOW-3997] Extend Variable.get so it can return None when var not found (#4819)</p></li> |
| <li><p>[AIRFLOW-4009] Fix docstring issue in GCSToBQOperator (#4836)</p></li> |
| <li><p>[AIRFLOW-3980] Unify logger (#4804)</p></li> |
| <li><p>[AIRFLOW-4076] Correct port type of beeline_default in init_db (#4908)</p></li> |
| <li><p>[AIRFLOW-4046] Add validations for poke_interval & timeout for Sensor (#4878)</p></li> |
| <li><p>[AIRFLOW-3744] Abandon the use of obsolete aliases of methods (#4568)</p></li> |
| <li><p>[AIRFLOW-3865] Add API endpoint to get python code of dag by id (#4687)</p></li> |
| <li><p>[AIRFLOW-3516] Support to create k8 worker pods in batches (#4434)</p></li> |
| <li><p>[AIRFLOW-2843] Add flag in ExternalTaskSensor to check if external DAG/task exists (#4547)</p></li> |
| <li><p>[AIRFLOW-2224] Add support CSV files in MySqlToGoogleCloudStorageOperator (#4738)</p></li> |
| <li><p>[AIRFLOW-3895] GoogleCloudStorageHook/Op create_bucket takes optional resource params (#4717)</p></li> |
| <li><p>[AIRFLOW-3950] Improve AirflowSecurityManager.update_admin_perm_view (#4774)</p></li> |
| <li><p>[AIRFLOW-4006] Make better use of Set in AirflowSecurityManager (#4833)</p></li> |
| <li><p>[AIRFLOW-3917] Specify alternate kube config file/context when running out of cluster (#4859)</p></li> |
| <li><p>[AIRFLOW-3911] Change Harvesting DAG parsing results to DEBUG log level (#4729)</p></li> |
| <li><p>[AIRFLOW-3584] Use ORM DAGs for index view. (#4390)</p></li> |
| <li><p>[AIRFLOW-2821] Refine Doc “Plugins” (#3664)</p></li> |
| <li><p>[AIRFLOW-3561] Improve queries (#4368)</p></li> |
| <li><p>[AIRFLOW-3600] Remove dagbag from trigger (#4407)</p></li> |
| <li><p>[AIRFLOW-3713] Updated documentation for GCP optional project_id (#4541)</p></li> |
| <li><p>[AIRFLOW-2767] Upgrade gunicorn to 19.5.0 to avoid moderate-severity CVE (#4795)</p></li> |
| <li><p>[AIRFLOW-3795] provide_context param is now used (#4735)</p></li> |
| <li><p>[AIRFLOW-4012] Upgrade tabulate to 0.8.3 (#4838)</p></li> |
| <li><p>[AIRFLOW-3623] Support download logs by attempts from UI (#4425)</p></li> |
| <li><p>[AIRFLOW-2715] Use region setting when launching Dataflow templates (#4139)</p></li> |
| <li><p>[AIRFLOW-3932] Update unit tests and documentation for safe mode flag. (#4760)</p></li> |
| <li><p>[AIRFLOW-3932] Optionally skip dag discovery heuristic. (#4746)</p></li> |
| <li><p>[AIRFLOW-3258] K8S executor environment variables section. (#4627)</p></li> |
| <li><p>[AIRFLOW-3931] set network, subnetwork when launching dataflow template (#4744)</p></li> |
| <li><p>[AIRFLOW-4095] Add template_fields for S3CopyObjectOperator & S3DeleteObjectsOperator (#4920)</p></li> |
| <li><p>[AIRFLOW-2798] Remove needless code from models.py</p></li> |
| <li><p>[AIRFLOW-3731] Constrain mysqlclient to <1.4 (#4558)</p></li> |
| <li><p>[AIRFLOW-3139] include parameters into log.info in SQL operators, if any (#3986)</p></li> |
| <li><p>[AIRFLOW-3174] Refine Docstring for SQL Operators & Hooks (#4043)</p></li> |
| <li><p>[AIRFLOW-3933] Fix various typos (#4747)</p></li> |
| <li><p>[AIRFLOW-3905] Allow using “parameters” in SqlSensor (#4723)</p></li> |
| <li><p>[AIRFLOW-2761] Parallelize enqueue in celery executor (#4234)</p></li> |
| <li><p>[AIRFLOW-3540] Respect environment config when looking up config file. (#4340)</p></li> |
| <li><p>[AIRFLOW-2156] Parallelize Celery Executor task state fetching (#3830)</p></li> |
| <li><p>[AIRFLOW-3702] Add backfill option to run backwards (#4676)</p></li> |
| <li><p>[AIRFLOW-3821] Add replicas logic to GCP SQL example DAG (#4662)</p></li> |
| <li><p>[AIRFLOW-3547] Fixed Jinja templating in SparkSubmitOperator (#4347)</p></li> |
| <li><p>[AIRFLOW-3647] Add archives config option to SparkSubmitOperator (#4467)</p></li> |
| <li><p>[AIRFLOW-3802] Updated documentation for HiveServer2Hook (#4647)</p></li> |
| <li><p>[AIRFLOW-3817] Corrected task ids returned by BranchPythonOperator to match the dummy operator ids (#4659)</p></li> |
| <li><p>[AIRFLOW-3782] Clarify docs around celery worker_autoscale in default_airflow.cfg (#4609)</p></li> |
| <li><p>[AIRFLOW-1945] Add Autoscale config for Celery workers (#3989)</p></li> |
| <li><p>[AIRFLOW-3590] Change log message of executor exit status (#4616)</p></li> |
| <li><p>[AIRFLOW-3591] Fix start date, end date, duration for rescheduled tasks (#4502)</p></li> |
| <li><p>[AIRFLOW-3709] Validate <cite>allowed_states</cite> for ExternalTaskSensor (#4536)</p></li> |
| <li><p>[AIRFLOW-3522] Add support for sending Slack attachments (#4332)</p></li> |
| <li><p>[AIRFLOW-3569] Add “Trigger DAG” button in DAG page (/www only) (#4373)</p></li> |
| <li><p>[AIRFLOW-3569] Add “Trigger DAG” button in DAG page (/www_rbac only) (#4373)</p></li> |
| <li><p>[AIRFLOW-3044] Dataflow operators accept templated job_name param (#3887)</p></li> |
| <li><p>[AIRFLOW-3023] Fix docstring datatypes</p></li> |
| <li><p>[AIRFLOW-2928] Use uuid4 instead of uuid1 (#3779)</p></li> |
| <li><p>[AIRFLOW-2988] Run specifically python2 for dataflow (#3826)</p></li> |
| <li><p>[AIRFLOW-3697] Vendorize nvd3 and slugify (#4513)</p></li> |
| <li><p>[AIRFLOW-3692] Remove ENV variables to avoid GPL (#4506)</p></li> |
| <li><p>[AIRFLOW-3907] Upgrade flask and set cookie security flags. (#4725)</p></li> |
| <li><p>[AIRFLOW-3698] Add documentation for AWS Connection (#4514)</p></li> |
| <li><p>[AIRFLOW-3616][AIRFLOW-1215] Add aliases for schema with underscore (#4523)</p></li> |
| <li><p>[AIRFLOW-3375] Support returning multiple tasks with BranchPythonOperator (#4215)</p></li> |
| <li><p>[AIRFLOW-3742] Fix handling of “fallback” for AirflowConfigParsxer.getint/boolean (#4674)</p></li> |
| <li><p>[AIRFLOW-3742] Respect the <cite>fallback</cite> arg in airflow.configuration.get (#4567)</p></li> |
| <li><p>[AIRFLOW-3789] Fix flake8 3.7 errors. (#4617)</p></li> |
| <li><p>[AIRFLOW-3602] Improve ImapHook handling of retrieving no attachments (#4475)</p></li> |
| <li><p>[AIRFLOW-3631] Update flake8 and fix lint. (#4436)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id2"> |
| <h3>Bug fixes<a class="headerlink" href="#id2" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-2994] Fix command status check in Qubole Check operator (#3790)</p></li> |
| <li><p>[AIRFLOW-2563] Fix PigCliHook Python 3 string/bytes use</p></li> |
| <li><p>[AIRFLOW-4248] Fix ‘FileExistsError’ makedirs race in file_processor_handler (#5047)</p></li> |
| <li><p>[AIRFLOW-4240] State-changing actions should be POST requests (#5039)</p></li> |
| <li><p>[AIRFLOW-4246] Flask-Oauthlib needs downstream dependencies pinning due to breaking changes (#5045)</p></li> |
| <li><p>[AIRFLOW-3887] Downgrade dagre-d3 to 0.4.18 (#4713)</p></li> |
| <li><p>[AIRFLOW-3419] Fix S3Hook.select_key on Python3 (#4970)</p></li> |
| <li><p>[AIRFLOW-4127] Correct AzureContainerInstanceHook._get_instance_view’s return (#4945)</p></li> |
| <li><p>[AIRFLOW-4172] Fix changes for driver class path option in Spark Submit (#4992)</p></li> |
| <li><p>[AIRFLOW-3615] Preserve case of UNIX socket paths in Connections (#4591)</p></li> |
| <li><p>[AIRFLOW-3417] ECSOperator: pass platformVersion only for FARGATE launch type (#4256)</p></li> |
| <li><p>[AIRFLOW-3884] Fixing doc checker, no warnings allowed anymore and fixed the current… (#4702)</p></li> |
| <li><p>[AIRFLOW-2652] implement / enhance baseOperator deepcopy</p></li> |
| <li><p>[AIRFLOW-4001] Update docs about how to run tests (#4826)</p></li> |
| <li><p>[AIRFLOW-3699] Speed up Flake8 (#4515)</p></li> |
| <li><p>[AIRFLOW-4160] Fix redirecting of ‘Trigger Dag’ Button in DAG Page (#4982)</p></li> |
| <li><p>[AIRFLOW-3650] Skip running on mysql for the flaky test (#4457)</p></li> |
| <li><p>[AIRFLOW-3423] Fix mongo hook to work with anonymous access (#4258)</p></li> |
| <li><p>[AIRFLOW-3982] Fix race condition in CI test (#4968)</p></li> |
| <li><p>[AIRFLOW-3982] Update DagRun state based on its own tasks (#4808)</p></li> |
| <li><p>[AIRFLOW-3737] Kubernetes executor cannot handle long dag/task names (#4636)</p></li> |
| <li><p>[AIRFLOW-3945] Stop inserting row when permission views unchanged (#4764)</p></li> |
| <li><p>[AIRFLOW-4123] Add Exception handling for _change_state method in K8 Executor (#4941)</p></li> |
| <li><p>[AIRFLOW-3771] Minor refactor securityManager (#4594)</p></li> |
| <li><p>[AIRFLOW-987] pass kerberos cli args keytab and principal to kerberos.run() (#4238)</p></li> |
| <li><p>[AIRFLOW-3736] Allow int value in SqoopOperator.extra_import_options(#4906)</p></li> |
| <li><p>[AIRFLOW-4063] Fix exception string in BigQueryHook [2/2] (#4902)</p></li> |
| <li><p>[AIRFLOW-4063] Fix exception string in BigQueryHook (#4899)</p></li> |
| <li><p>[AIRFLOW-4037] Log response in SimpleHttpOperator even if the response check fails</p></li> |
| <li><p>[AIRFLOW-4044] The documentation of <cite>query_params</cite> in <cite>BigQueryOperator</cite> is wrong. (#4876)</p></li> |
| <li><p>[AIRFLOW-4015] Make missing API endpoints available in classic mode</p></li> |
| <li><p>[AIRFLOW-3153] Send DAG processing stats to statsd (#4748)</p></li> |
| <li><p>[AIRFLOW-2966] Catch ApiException in the Kubernetes Executor (#4209)</p></li> |
| <li><p>[AIRFLOW-4129] Escape HTML in generated tooltips (#4950)</p></li> |
| <li><p>[AIRFLOW-4070] AirflowException -> log.warning for duplicate task dependencies (#4904)</p></li> |
| <li><p>[AIRFLOW-4054] Fix assertEqualIgnoreMultipleSpaces util & add tests (#4886)</p></li> |
| <li><p>[AIRFLOW-3239] Fix test recovery further (#4074)</p></li> |
| <li><p>[AIRFLOW-4053] Fix KubePodOperator Xcom on Kube 1.13.0 (#4883)</p></li> |
| <li><p>[AIRFLOW-2961] Refactor tests.BackfillJobTest.test_backfill_examples test (#3811)</p></li> |
| <li><p>[AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken (#4415)</p></li> |
| <li><p>[AIRFLOW-3543] Fix deletion of DAG with rescheduled tasks (#4646)</p></li> |
| <li><p>[AIRFLOW-2548] Output plugin import errors to web UI (#3930)</p></li> |
| <li><p>[AIRFLOW-4019] Fix AWS Athena Sensor object has no attribute ‘mode’ (#4844)</p></li> |
| <li><p>[AIRFLOW-3758] Fix circular import in WasbTaskHandler (#4601)</p></li> |
| <li><p>[AIRFLOW-3706] Fix tooltip max-width by correcting ordering of CSS files (#4947)</p></li> |
| <li><p>[AIRFLOW-4100] Correctly JSON escape data for tree/graph views (#4921)</p></li> |
| <li><p>[AIRFLOW-3636] Fix a test introduced in #4425 (#4446)</p></li> |
| <li><p>[AIRFLOW-3977] Add examples of trigger rules in doc (#4805)</p></li> |
| <li><p>[AIRFLOW-2511] Fix improper failed session commit handling causing deadlocks (#4769)</p></li> |
| <li><p>[AIRFLOW-3962] Added graceful handling for creation of dag_run of a dag which doesn’t have any task (#4781)</p></li> |
| <li><p>[AIRFLOW-3881] Correct to_csv row number (#4699)</p></li> |
| <li><p>[AIRFLOW-3875] Simplify SlackWebhookHook code and change docstring (#4696)</p></li> |
| <li><p>[AIRFLOW-3733] Don’t raise NameError in HQL hook to_csv when no rows returned (#4560)</p></li> |
| <li><p>[AIRFLOW-3734] Fix hql not run when partition is None (#4561)</p></li> |
| <li><p>[AIRFLOW-3767] Correct bulk insert function (#4773)</p></li> |
| <li><p>[AIRFLOW-4087] remove sudo in basetaskrunner on_finish (#4916)</p></li> |
| <li><p>[AIRFLOW-3768] Escape search parameter in pagination controls (#4911)</p></li> |
| <li><p>[AIRFLOW-4045] Fix hard-coded URLs in FAB-based UI (#4914)</p></li> |
| <li><p>[AIRFLOW-3123] Use a stack for DAG context management (#3956)</p></li> |
| <li><p>[AIRFLOW-3060] DAG context manager fails to exit properly in certain circumstances</p></li> |
| <li><p>[AIRFLOW-3924] Fix try number in alert emails (#4741)</p></li> |
| <li><p>[AIRFLOW-4083] Add tests for link generation utils (#4912)</p></li> |
| <li><p>[AIRFLOW-2190] Send correct HTTP status for base_url not found (#4910)</p></li> |
| <li><p>[AIRFLOW-4015] Add get_dag_runs GET endpoint to “classic” API (#4884)</p></li> |
| <li><p>[AIRFLOW-3239] Enable existing CI tests (#4131)</p></li> |
| <li><p>[AIRFLOW-1390] Update Alembic to 0.9 (#3935)</p></li> |
| <li><p>[AIRFLOW-3885] Fix race condition in scheduler test (#4737)</p></li> |
| <li><p>[AIRFLOW-3885] ~10x speed-up of SchedulerJobTest suite (#4730)</p></li> |
| <li><p>[AIRFLOW-3780] Fix some incorrect when base_url is used (#4643)</p></li> |
| <li><p>[AIRFLOW-3807] Fix Graph View Highlighting of Tasks (#4653)</p></li> |
| <li><p>[AIRFLOW-3009] Import Hashable from collection.abc to fix Python 3.7 deprecation warning (#3849)</p></li> |
| <li><p>[AIRFLOW-2231] Fix relativedelta DAG schedule_interval (#3174)</p></li> |
| <li><p>[AIRFLOW-2641] Fix MySqlToHiveTransfer to handle MySQL DECIMAL correctly</p></li> |
| <li><p>[AIRFLOW-3751] Option to allow malformed schemas for LDAP authentication (#4574)</p></li> |
| <li><p>[AIRFLOW-2888] Add deprecation path for task_runner config change (#4851)</p></li> |
| <li><p>[AIRFLOW-2930] Fix celery excecutor scheduler crash (#3784)</p></li> |
| <li><p>[AIRFLOW-2888] Remove shell=True and bash from task launch (#3740)</p></li> |
| <li><p>[AIRFLOW-3885] ~2.5x speed-up for backfill tests (#4731)</p></li> |
| <li><p>[AIRFLOW-3885] ~20x speed-up of slowest unit test (#4726)</p></li> |
| <li><p>[AIRFLOW-2508] Handle non string types in Operators templatized fields (#4292)</p></li> |
| <li><p>[AIRFLOW-3792] Fix validation in BQ for useLegacySQL & queryParameters (#4626)</p></li> |
| <li><p>[AIRFLOW-3749] Fix Edit Dag Run page when using RBAC (#4613)</p></li> |
| <li><p>[AIRFLOW-3801] Fix DagBag collect dags invocation to prevent examples to be loaded (#4677)</p></li> |
| <li><p>[AIRFLOW-3774] Register blueprints with RBAC web app (#4598)</p></li> |
| <li><p>[AIRFLOW-3719] Handle StopIteration in CloudWatch logs retrieval (#4516)</p></li> |
| <li><p>[AIRFLOW-3108] Define get_autocommit method for MsSqlHook (#4525)</p></li> |
| <li><p>[AIRFLOW-3074] Add relevant ECS options to ECS operator. (#3908)</p></li> |
| <li><p>[AIRFLOW-3353] Upgrade Redis client (#4834)</p></li> |
| <li><p>[AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls (#4090)</p></li> |
| <li><p>[AIRFLOW-2009] Fix dataflow hook connection-id (#4563)</p></li> |
| <li><p>[AIRFLOW-2190] Fix TypeError when returning 404 (#4596)</p></li> |
| <li><p>[AIRFLOW-2876] Update Tenacity to 4.12 (#3723)</p></li> |
| <li><p>[AIRFLOW-3923] Update flask-admin dependency to 1.5.3 to resolve security vulnerabilities from safety (#4739)</p></li> |
| <li><p>[AIRFLOW-3683] Fix formatting of error message for invalid TriggerRule (#4490)</p></li> |
| <li><p>[AIRFLOW-2787] Allow is_backfill to handle NULL DagRun.run_id (#3629)</p></li> |
| <li><p>[AIRFLOW-3780] Fix some incorrect when base_url is used (#4643)</p></li> |
| <li><p>[AIRFLOW-3639] Fix request creation in Jenkins Operator (#4450)</p></li> |
| <li><p>[AIRFLOW-3779] Don’t install enum34 backport when not needed (#4620)</p></li> |
| <li><p>[AIRFLOW-3079] Improve migration scripts to support MSSQL Server (#3964)</p></li> |
| <li><p>[AIRFLOW-2735] Use equality, not identity, check for detecting AWS Batch failures[]</p></li> |
| <li><p>[AIRFLOW-2706] AWS Batch Operator should use top-level job state to determine status</p></li> |
| <li><p>[AIRFLOW-XXX] Fix typo in http_operator.py</p></li> |
| <li><p>[AIRFLOW-XXX] Solve lodash security warning (#4820)</p></li> |
| <li><p>[AIRFLOW-XXX] Pin version of tornado pulled in by Celery. (#4815)</p></li> |
| <li><p>[AIRFLOW-XXX] Upgrade FAB to 1.12.3 (#4694)</p></li> |
| <li><p>[AIRFLOW-XXX] Pin pinodb dependency (#4704)</p></li> |
| <li><p>[AIRFLOW-XXX] Pin version of Pip in tests to work around pypa/pip#6163 (#4576)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix spark submit hook KeyError (#4578)</p></li> |
| <li><p>[AIRFLOW-XXX] Pin psycopg2 due to breaking change (#5036)</p></li> |
| <li><p>[AIRFLOW-XXX] Pin Sendgrid dep. (#5031)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix flaky test - test_execution_unlimited_parallelism (#4988)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id3"> |
| <h3>Misc/Internal<a class="headerlink" href="#id3" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-4016] Clear runs for BackfillJobTest (#4839)</p></li> |
| <li><p>[AIRFLOW-4177] Check types in tests</p></li> |
| <li><p>[AIRFLOW-4144] add description of is_delete_operator_pod (#4943)</p></li> |
| <li><p>[AIRFLOW-3476][AIRFLOW-3477] Move Kube classes out of models.py (#4443)</p></li> |
| <li><p>[AIRFLOW-3464] Move SkipMixin out of models.py (#4386)</p></li> |
| <li><p>[AIRFLOW-3463] Move Log out of models.py (#4639)</p></li> |
| <li><p>[AIRFLOW-3458] Move connection tests (#4680)</p></li> |
| <li><p>[AIRFLOW-3461] Move TaskFail out of models.py (#4630)</p></li> |
| <li><p>[AIRFLOW-3462] Move TaskReschedule out of models.py (#4618)</p></li> |
| <li><p>[AIRFLOW-3474] Move SlaMiss out of models.py (#4608)</p></li> |
| <li><p>[AIRFLOW-3475] Move ImportError out of models.py (#4383)</p></li> |
| <li><p>[AIRFLOW-3459] Move DagPickle to separate file (#4374)</p></li> |
| <li><p>[AIRFLOW-3925] Don’t pull docker-images on pretest (#4740)</p></li> |
| <li><p>[AIRFLOW-4154] Correct string formatting in jobs.py (#4972)</p></li> |
| <li><p>[AIRFLOW-3458] Deprecation path for moving models.Connection</p></li> |
| <li><p>[AIRFLOW-3458] Move models.Connection into separate file (#4335)</p></li> |
| <li><p>[AIRFLOW-XXX] Remove old/non-test files that nose ignores (#4930)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id4"> |
| <h3>Doc-only changes<a class="headerlink" href="#id4" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-3996] Add view source link to included fragments</p></li> |
| <li><p>[AIRFLOW-3811] automatic generation of API Reference in docs (#4788)</p></li> |
| <li><p>[AIRFLOW-3810] Remove duplicate autoclass directive (#4656)</p></li> |
| <li><p>[AIRFLOW-XXX] Mention that statsd must be installed to gather metrics (#5038)</p></li> |
| <li><p>[AIRFLOW-XXX] Add contents to cli (#4825)</p></li> |
| <li><p>[AIRFLOW-XXX] fix check docs failure on CI (#4998)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix syntax docs errors (#4789)</p></li> |
| <li><p>[AIRFLOW-XXX] Docs rendering improvement (#4684)</p></li> |
| <li><p>[AIRFLOW-XXX] Automatically link Jira/GH on doc’s changelog page (#4587)</p></li> |
| <li><p>[AIRFLOW-XXX] Mention Oracle in the Extra Packages documentation (#4987)</p></li> |
| <li><p>[AIRFLOW-XXX] Drop deprecated sudo option; use default docker compose on Travis. (#4732)</p></li> |
| <li><p>[AIRFLOW-XXX] Update kubernetes.rst docs (#3875)</p></li> |
| <li><p>[AIRFLOW-XXX] Improvements to formatted content in documentation (#4835)</p></li> |
| <li><p>[AIRFLOW-XXX] Add Daniel to committer list (#4961)</p></li> |
| <li><p>[AIRFLOW-XXX] Add Xiaodong Deng to committers list</p></li> |
| <li><p>[AIRFLOW-XXX] Add history become ASF top level project (#4757)</p></li> |
| <li><p>[AIRFLOW-XXX] Move out the examples from integration.rst (#4672)</p></li> |
| <li><p>[AIRFLOW-XXX] Extract reverse proxy info to a separate file (#4657)</p></li> |
| <li><p>[AIRFLOW-XXX] Reduction of the number of warnings in the documentation (#4585)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix GCS Operator docstrings (#4054)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix Docstrings in Hooks, Sensors & Operators (#4137)</p></li> |
| <li><p>[AIRFLOW-XXX] Split guide for operators to multiple files (#4814)</p></li> |
| <li><p>[AIRFLOW-XXX] Split connection guide to multiple files (#4824)</p></li> |
| <li><p>[AIRFLOW-XXX] Remove almost all warnings from building docs (#4588)</p></li> |
| <li><p>[AIRFLOW-XXX] Add backreference in docs between operator and integration (#4671)</p></li> |
| <li><p>[AIRFLOW-XXX] Improve linking to classes (#4655)</p></li> |
| <li><p>[AIRFLOW-XXX] Mock optional modules when building docs (#4586)</p></li> |
| <li><p>[AIRFLOW-XXX] Update plugin macros documentation (#4971)</p></li> |
| <li><p>[AIRFLOW-XXX] Add missing docstring for ‘autodetect’ in GCS to BQ Operator (#4979)</p></li> |
| <li><p>[AIRFLOW-XXX] Add missing GCP operators to Docs (#4260)</p></li> |
| <li><p>[AIRFLOW-XXX] Fixing the issue in Documentation (#3756)</p></li> |
| <li><p>[AIRFLOW-XXX] Add Hint at user defined macros (#4885)</p></li> |
| <li><p>[AIRFLOW-XXX] Correct schedule_interval in Scheduler docs (#4157)</p></li> |
| <li><p>[AIRFLOW-XXX] Improve airflow-jira script to make RelManager’s life easier (#4857)</p></li> |
| <li><p>[AIRFLOW-XXX] Add missing class references to docs (#4644)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix typo (#4564)</p></li> |
| <li><p>[AIRFLOW-XXX] Add a doc about fab security (#4595)</p></li> |
| <li><p>[AIRFLOW-XXX] Speed up DagBagTest cases (#3974)</p></li> |
| <li><p>[AIRFLOW-XXX] Reduction of the number of warnings in the documentation (#4585)</p></li> |
| </ul> |
| </div> |
| </div> |
| <div class="section" id="airflow-1-10-2-2019-01-19"> |
| <h2>Airflow 1.10.2, 2019-01-19<a class="headerlink" href="#airflow-1-10-2-2019-01-19" title="Permalink to this headline">¶</a></h2> |
| <div class="section" id="id5"> |
| <h3>New features<a class="headerlink" href="#id5" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-2658] Add GCP specific k8s pod operator (#3532)</p></li> |
| <li><p>[AIRFLOW-2440] Google Cloud SQL import/export operator (#4251)</p></li> |
| <li><p>[AIRFLOW-3212] Add AwsGlueCatalogPartitionSensor (#4112)</p></li> |
| <li><p>[AIRFLOW-2750] Add subcommands to delete and list users</p></li> |
| <li><p>[AIRFLOW-3480] Add GCP Spanner Database Operators (#4353)</p></li> |
| <li><p>[AIRFLOW-3560] Add DayOfWeek Sensor (#4363)</p></li> |
| <li><p>[AIRFLOW-3371] BigQueryHook’s Ability to Create View (#4213)</p></li> |
| <li><p>[AIRFLOW-3332] Add method to allow inserting rows into BQ table (#4179)</p></li> |
| <li><p>[AIRFLOW-3055] add get_dataset and get_datasets_list to bigquery_hook (#3894)</p></li> |
| <li><p>[AIRFLOW-2887] Added BigQueryCreateEmptyDatasetOperator and create_emty_dataset to bigquery_hook (#3876)</p></li> |
| <li><p>[AIRFLOW-2758] Add a sensor for MongoDB</p></li> |
| <li><p>[AIRFLOW-2640] Add Cassandra table sensor</p></li> |
| <li><p>[AIRFLOW-3398] Google Cloud Spanner instance database query operator (#4314)</p></li> |
| <li><p>[AIRFLOW-3310] Google Cloud Spanner deploy / delete operators (#4286)</p></li> |
| <li><p>[AIRFLOW-3406] Implement an Azure CosmosDB operator (#4265)</p></li> |
| <li><p>[AIRFLOW-3434] Allows creating intermediate dirs in SFTPOperator (#4270)</p></li> |
| <li><p>[AIRFLOW-3345] Add Google Cloud Storage (GCS) operators for ACL (#4192)</p></li> |
| <li><p>[AIRFLOW-3266] Add AWS Athena Hook and Operator (#4111)</p></li> |
| <li><p>[AIRFLOW-3346] Add hook and operator for GCP transfer service (#4189)</p></li> |
| <li><p>[AIRFLOW-2983] Add prev_ds_nodash and next_ds_nodash macro (#3821)</p></li> |
| <li><p>[AIRFLOW-3403] Add AWS Athena Sensor (#4244)</p></li> |
| <li><p>[AIRFLOW-3323] Support HTTP basic authentication for Airflow Flower (#4166)</p></li> |
| <li><p>[AIRFLOW-3410] Add feature to allow Host Key Change for SSH Op (#4249)</p></li> |
| <li><p>[AIRFLOW-3275] Add Google Cloud SQL Query operator (#4170)</p></li> |
| <li><p>[AIRFLOW-2691] Manage JS dependencies via npm</p></li> |
| <li><p>[AIRFLOW-2795] Oracle to Oracle Transfer Operator (#3639)</p></li> |
| <li><p>[AIRFLOW-2596] Add Oracle to Azure Datalake Transfer Operator</p></li> |
| <li><p>[AIRFLOW-3220] Add Instance Group Manager Operators for GCE (#4167)</p></li> |
| <li><p>[AIRFLOW-2882] Add import and export for pool cli using JSON</p></li> |
| <li><p>[AIRFLOW-2965] CLI tool to show the next execution datetime (#3834)</p></li> |
| <li><p>[AIRFLOW-2874] Enables FAB’s theme support (#3719)</p></li> |
| <li><p>[AIRFLOW-3336] Add new TriggerRule for 0 upstream failures (#4182)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="improvements"> |
| <h3>Improvements<a class="headerlink" href="#improvements" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-3680] Consistency update in tests for All GCP-related operators (#4493)</p></li> |
| <li><p>[AIRFLOW-3675] Use googlapiclient for google apis (#4484)</p></li> |
| <li><p>[AIRFLOW-3205] Support multipart uploads to GCS (#4084)</p></li> |
| <li><p>[AIRFLOW-2826] Add GoogleCloudKMSHook (#3677)</p></li> |
| <li><p>[AIRFLOW-3676] Add required permission to CloudSQL export/import example (#4489)</p></li> |
| <li><p>[AIRFLOW-3679] Added Google Cloud Base Hook to documentation (#4487)</p></li> |
| <li><p>[AIRFLOW-3594] Unify different License Header</p></li> |
| <li><p>[AIRFLOW-3197] Remove invalid parameter KeepJobFlowAliveWhenNoSteps in example DAG (#4404)</p></li> |
| <li><p>[AIRFLOW-3504] Refine the functionality of “/health” endpoint (#4309)</p></li> |
| <li><p>[AIRFLOW-3103][AIRFLOW-3147] Update flask-appbuilder (#3937)</p></li> |
| <li><p>[AIRFLOW-3168] More resillient database use in CI (#4014)</p></li> |
| <li><p>[AIRFLOW-3076] Remove preloading of MySQL testdata (#3911)</p></li> |
| <li><p>[AIRFLOW-3035] Allow custom ‘job_error_states’ in dataproc ops (#3884)</p></li> |
| <li><p>[AIRFLOW-3246] Make hmsclient optional in airflow.hooks.hive_hooks (#4080)</p></li> |
| <li><p>[AIRFLOW-3059] Log how many rows are read from Postgres (#3905)</p></li> |
| <li><p>[AIRFLOW-2463] Make task instance context available for hive queries</p></li> |
| <li><p>[AIRFLOW-3190] Make flake8 compliant (#4035)</p></li> |
| <li><p>[AIRFLOW-1998] Implemented DatabricksRunNowOperator for jobs/run-now … (#3813)</p></li> |
| <li><p>[AIRFLOW-2267] Airflow DAG level access (#3197)</p></li> |
| <li><p>[AIRFLOW-2359] Add set failed for DagRun and task in tree view (#3255)</p></li> |
| <li><p>[AIRFLOW-3008] Move Kubernetes example DAGs to contrib</p></li> |
| <li><p>[AIRFLOW-3402] Support global k8s affinity and toleration configs (#4247)</p></li> |
| <li><p>[AIRFLOW-3610] Add region param for EMR jobflow creation (#4418)</p></li> |
| <li><p>[AIRFLOW-3531] Fix test for GCS to GCS Transfer Hook (#4452)</p></li> |
| <li><p>[AIRFLOW-3531] Add gcs to gcs transfer operator. (#4331)</p></li> |
| <li><p>[AIRFLOW-3034]: Readme updates : Add Slack & Twitter, remove Gitter</p></li> |
| <li><p>[AIRFLOW-3028] Update Text & Images in Readme.md</p></li> |
| <li><p>[AIRFLOW-208] Add badge to show supported Python versions (#3839)</p></li> |
| <li><p>[AIRFLOW-2238] Update PR tool to push directly to Github</p></li> |
| <li><p>[AIRFLOW-2238] Flake8 fixes on dev/airflow-pr</p></li> |
| <li><p>[AIRFLOW-2238] Update PR tool to remove outdated info (#3978)</p></li> |
| <li><p>[AIRFLOW-3005] Replace ‘Airbnb Airflow’ with ‘Apache Airflow’ (#3845)</p></li> |
| <li><p>[AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator (#4359)</p></li> |
| <li><p>[AIRFLOW-1196][AIRFLOW-2399] Add templated field in TriggerDagRunOperator (#4228)</p></li> |
| <li><p>[AIRFLOW-3340] Placeholder support in connections form (#4185)</p></li> |
| <li><p>[AIRFLOW-3446] Add Google Cloud BigTable operators (#4354)</p></li> |
| <li><p>[AIRFLOW-1921] Add support for https and user auth (#2879)</p></li> |
| <li><p>[AIRFLOW-2770] Read <cite>dags_in_image</cite> config value as a boolean (#4319)</p></li> |
| <li><p>[AIRFLOW-3022] Add volume mount to KubernetesExecutorConfig (#3855)</p></li> |
| <li><p>[AIRFLOW-2917] Set AIRFLOW__CORE__SQL_ALCHEMY_CONN only when needed (#3766)</p></li> |
| <li><p>[AIRFLOW-2712] Pass annotations to KubernetesExecutorConfig</p></li> |
| <li><p>[AIRFLOW-461] Support autodetected schemas in BigQuery run_load (#3880)</p></li> |
| <li><p>[AIRFLOW-2997] Support cluster fields in bigquery (#3838)</p></li> |
| <li><p>[AIRFLOW-2916] Arg <cite>verify</cite> for AwsHook() & S3 sensors/operators (#3764)</p></li> |
| <li><p>[AIRFLOW-491] Add feature to pass extra api configs to BQ Hook (#3733)</p></li> |
| <li><p>[AIRFLOW-2889] Fix typos detected by github.com/client9/misspell (#3732)</p></li> |
| <li><p>[AIRFLOW-850] Add a PythonSensor (#4349)</p></li> |
| <li><p>[AIRFLOW-2747] Explicit re-schedule of sensors (#3596)</p></li> |
| <li><p>[AIRFLOW-3392] Add index on dag_id in sla_miss table (#4235)</p></li> |
| <li><p>[AIRFLOW-3001] Add index ‘ti_dag_date’ to taskinstance (#3885)</p></li> |
| <li><p>[AIRFLOW-2861] Add index on log table (#3709)</p></li> |
| <li><p>[AIRFLOW-3518] Performance fixes for topological_sort of Tasks (#4322)</p></li> |
| <li><p>[AIRFLOW-3521] Fetch more than 50 items in <cite>airflow-jira compare</cite> script (#4300)</p></li> |
| <li><p>[AIRFLOW-1919] Add option to query for DAG runs given a DAG ID</p></li> |
| <li><p>[AIRFLOW-3444] Explicitly set transfer operator description. (#4279)</p></li> |
| <li><p>[AIRFLOW-3411] Add OpenFaaS hook (#4267)</p></li> |
| <li><p>[AIRFLOW-2785] Add context manager entry points to mongoHook</p></li> |
| <li><p>[AIRFLOW-2524] Add SageMaker doc to AWS integration section (#4278)</p></li> |
| <li><p>[AIRFLOW-3479] Keeps records in Log Table when DAG is deleted (#4287)</p></li> |
| <li><p>[AIRFLOW-2948] Arg check & better doc - SSHOperator & SFTPOperator (#3793)</p></li> |
| <li><p>[AIRFLOW-2245] Add remote_host of SSH/SFTP operator as templated field (#3765)</p></li> |
| <li><p>[AIRFLOW-2670] Update SSH Operator’s Hook to respect timeout (#3666)</p></li> |
| <li><p>[AIRFLOW-3380] Add metrics documentation (#4219)</p></li> |
| <li><p>[AIRFLOW-3361] Log the task_id in the PendingDeprecationWarning from BaseOperator (#4030)</p></li> |
| <li><p>[AIRFLOW-3213] Create ADLS to GCS operator (#4134)</p></li> |
| <li><p>[AIRFLOW-3395] added the REST API endpoints to the doc (#4236)</p></li> |
| <li><p>[AIRFLOW-3294] Update connections form and integration docs (#4129)</p></li> |
| <li><p>[AIRFLOW-3236] Create AzureDataLakeStorageListOperator (#4094)</p></li> |
| <li><p>[AIRFLOW-3062] Add Qubole in integration docs (#3946)</p></li> |
| <li><p>[AIRFLOW-3306] Disable flask-sqlalchemy modification tracking. (#4146)</p></li> |
| <li><p>[AIRFLOW-2867] Refactor Code to conform standards (#3714)</p></li> |
| <li><p>[AIRFLOW-2753] Add dataproc_job_id instance var holding actual DP jobId</p></li> |
| <li><p>[AIRFLOW-3132] Enable specifying auto_remove option for DockerOperator (#3977)</p></li> |
| <li><p>[AIRFLOW-2731] Raise psutil restriction to <6.0.0</p></li> |
| <li><p>[AIRFLOW-3384] Allow higher versions of Sqlalchemy and Jinja2 (#4227)</p></li> |
| <li><p>[Airflow-2760] Decouple DAG parsing loop from scheduler loop (#3873)</p></li> |
| <li><p>[AIRFLOW-3004] Add config disabling scheduler cron (#3899)</p></li> |
| <li><p>[AIRFLOW-3175] Fix docstring format in airflow/jobs.py (#4025)</p></li> |
| <li><p>[AIRFLOW-3589] Visualize reschedule state in all views (#4408)</p></li> |
| <li><p>[AIRFLOW-2698] Simplify Kerberos code (#3563)</p></li> |
| <li><p>[AIRFLOW-2499] Dockerise CI pipeline (#3393)</p></li> |
| <li><p>[AIRFLOW-3432] Add test for feature “Delete DAG in UI” (#4266)</p></li> |
| <li><p>[AIRFLOW-3301] Update DockerOperator CI test for PR #3977 (#4138)</p></li> |
| <li><p>[AIRFLOW-3478] Make sure that the session is closed</p></li> |
| <li><p>[AIRFLOW-3687] Add missing @apply_defaults decorators (#4498)</p></li> |
| <li><p>[AIRFLOW-3691] Update notice to 2019 (#4503)</p></li> |
| <li><p>[AIRFLOW-3689] Update pop-up message when deleting DAG in RBAC UI (#4505)</p></li> |
| <li><p>[AIRFLOW-2801] Skip test_mark_success_no_kill in PostgreSQL on CI (#3642)</p></li> |
| <li><p>[AIRFLOW-3693] Replace psycopg2-binary by psycopg2 (#4508)</p></li> |
| <li><p>[AIRFLOW-3700] Change the lowest allowed version of “requests” (#4517)</p></li> |
| <li><p>[AIRFLOW-3704] Support SSL Protection When Redis is Used as Broker for CeleryExecutor (#4521)</p></li> |
| <li><p>[AIRFLOW-3681] All GCP operators have now optional GCP Project ID (#4500)</p></li> |
| <li><p>[Airflow 2782] Upgrades Dagre D3 version to latest possible</p></li> |
| <li><p>[Airflow 2783] Implement eslint for JS code check (#3641)</p></li> |
| <li><p>[AIRFLOW-2805] Display multiple timezones on UI (#3687)</p></li> |
| <li><p>[AIRFLOW-3302] Small CSS fixes (#4140)</p></li> |
| <li><p>[Airflow-2766] Respect shared datetime across tabs</p></li> |
| <li><p>[AIRFLOW-2776] Compress tree view JSON</p></li> |
| <li><p>[AIRFLOW-2407] Use feature detection for reload() (#3298)</p></li> |
| <li><p>[AIRFLOW-3452] Removed an unused/dangerous display-none (#4295)</p></li> |
| <li><p>[AIRFLOW-3348] Update run statistics on dag refresh (#4197)</p></li> |
| <li><p>[AIRFLOW-3125] Monitor Task Instances creation rates (#3966)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id6"> |
| <h3>Bug fixes<a class="headerlink" href="#id6" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-3191] Fix not being able to specify execution_date when creating dagrun (#4037)</p></li> |
| <li><p>[AIRFLOW-3657] Fix zendesk integration (#4466)</p></li> |
| <li><p>[AIRFLOW-3605] Load plugins from entry_points (#4412)</p></li> |
| <li><p>[AIRFLOW-3646] Rename plugins_manager.py to test_xx to trigger tests (#4464)</p></li> |
| <li><p>[AIRFLOW-3655] Escape links generated in model views (#4463)</p></li> |
| <li><p>[AIRFLOW-3662] Add dependency for Enum (#4468)</p></li> |
| <li><p>[AIRFLOW-3630] Cleanup of GCP Cloud SQL Connection (#4451)</p></li> |
| <li><p>[AIRFLOW-1837] Respect task start_date when different from dag’s (#4010)</p></li> |
| <li><p>[AIRFLOW-2829] Brush up the CI script for minikube</p></li> |
| <li><p>[AIRFLOW-3519] Fix example http operator (#4455)</p></li> |
| <li><p>[AIRFLOW-2811] Fix scheduler_ops_metrics.py to work (#3653)</p></li> |
| <li><p>[AIRFLOW-2751] add job properties update in hive to druid operator.</p></li> |
| <li><p>[AIRFLOW-2918] Remove unused imports</p></li> |
| <li><p>[AIRFLOW-2918] Fix Flake8 violations (#3931)</p></li> |
| <li><p>[AIRFLOW-2771] Add except type to broad S3Hook try catch clauses</p></li> |
| <li><p>[AIRFLOW-2918] Fix Flake8 violations (#3772)</p></li> |
| <li><p>[AIRFLOW-2099] Handle getsource() calls gracefully</p></li> |
| <li><p>[AIRFLOW-3397] Fix integrety error in rbac AirflowSecurityManager (#4305)</p></li> |
| <li><p>[AIRFLOW-3281] Fix Kubernetes operator with git-sync (#3770)</p></li> |
| <li><p>[AIRFLOW-2615] Limit DAGs parsing to once only</p></li> |
| <li><p>[AIRFLOW-2952] Fix Kubernetes CI (#3922)</p></li> |
| <li><p>[AIRFLOW-2933] Enable Codecov on Docker-CI Build (#3780)</p></li> |
| <li><p>[AIRFLOW-2082] Resolve a bug in adding password_auth to api as auth method (#4343)</p></li> |
| <li><p>[AIRFLOW-3612] Remove incubation/incubator mention (#4419)</p></li> |
| <li><p>[AIRFLOW-3581] Fix next_ds/prev_ds semantics for manual runs (#4385)</p></li> |
| <li><p>[AIRFLOW-3527] Update Cloud SQL Proxy to have shorter path for UNIX socket (#4350)</p></li> |
| <li><p>[AIRFLOW-3316] For gcs_to_bq: add missing init of schema_fields var (#4430)</p></li> |
| <li><p>[AIRFLOW-3583] Fix AirflowException import (#4389)</p></li> |
| <li><p>[AIRFLOW-3578] Fix Type Error for BigQueryOperator (#4384)</p></li> |
| <li><p>[AIRFLOW-2755] Added <cite>kubernetes.worker_dags_folder</cite> configuration (#3612)</p></li> |
| <li><p>[AIRFLOW-2655] Fix inconsistency of default config of kubernetes worker</p></li> |
| <li><p>[AIRFLOW-2645][AIRFLOW-2617] Add worker_container_image_pull_policy</p></li> |
| <li><p>[AIRFLOW-2661] fix config dags_volume_subpath and logs_volume_subpath</p></li> |
| <li><p>[AIRFLOW-3550] Standardize GKE hook (#4364)</p></li> |
| <li><p>[AIRFLOW-2863] Fix GKEClusterHook catching wrong exception (#3711)</p></li> |
| <li><p>[AIRFLOW-2939][AIRFLOW-3568] Fix TypeError in GCSToS3Op & S3ToGCSOp (#4371)</p></li> |
| <li><p>[AIRFLOW-3327] Add support for location in BigQueryHook (#4324)</p></li> |
| <li><p>[AIRFLOW-3438] Fix default values in BigQuery Hook & BigQueryOperator (…</p></li> |
| <li><p>[AIRFLOW-3355] Fix BigQueryCursor.execute to work with Python3 (#4198)</p></li> |
| <li><p>[AIRFLOW-3447] Add 2 options for ts_nodash Macro (#4323)</p></li> |
| <li><p>[AIRFLOW-1552] Airflow Filter_by_owner not working with password_auth (#4276)</p></li> |
| <li><p>[AIRFLOW-3484] Fix Over-logging in the k8s executor (#4296)</p></li> |
| <li><p>[AIRFLOW-3309] Add MongoDB connection (#4154)</p></li> |
| <li><p>[AIRFLOW-3414] Fix reload_module in DagFileProcessorAgent (#4253)</p></li> |
| <li><p>[AIRFLOW-1252] API accept JSON when invoking a trigger dag (#2334)</p></li> |
| <li><p>[AIRFLOW-3425] Fix setting default scope in hook (#4261)</p></li> |
| <li><p>[AIRFLOW-3416] Fixes Python 3 compatibility with CloudSqlQueryOperator (#4254)</p></li> |
| <li><p>[AIRFLOW-3263] Ignore exception when ‘run’ kills already killed job (#4108)</p></li> |
| <li><p>[AIRFLOW-3264] URL decoding when parsing URI for connection (#4109)</p></li> |
| <li><p>[AIRFLOW-3365][AIRFLOW-3366] Allow celery_broker_transport_options to be set with environment variables (#4211)</p></li> |
| <li><p>[AIRFLOW-2642] fix wrong value git-sync initcontainer env GIT_SYNC_ROOT (#3519)</p></li> |
| <li><p>[AIRFLOW-3353] Pin redis version (#4195)</p></li> |
| <li><p>[AIRFLOW-3251] KubernetesPodOperator now uses ‘image_pull_secrets’ argument when creating Pods (#4188)</p></li> |
| <li><p>[AIRFLOW-2705] Move class-level moto decorator to method-level</p></li> |
| <li><p>[AIRFLOW-3233] Fix deletion of DAGs in the UI (#4069)</p></li> |
| <li><p>[AIRFLOW-2908] Allow retries with KubernetesExecutor. (#3758)</p></li> |
| <li><p>[AIRFLOW-1561] Fix scheduler to pick up example DAGs without other DAGs (#2635)</p></li> |
| <li><p>[AIRFLOW-3352] Fix expose_config not honoured on RBAC UI (#4194)</p></li> |
| <li><p>[AIRFLOW-3592] Fix logs when task is in rescheduled state (#4492)</p></li> |
| <li><p>[AIRFLOW-3634] Fix GCP Spanner Test (#4440)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix PythonVirtualenvOperator tests (#3968)</p></li> |
| <li><p>[AIRFLOW-3239] Fix/refine tests for api/common/experimental/ (#4255)</p></li> |
| <li><p>[AIRFLOW-2951] Update dag_run table end_date when state change (#3798)</p></li> |
| <li><p>[AIRFLOW-2756] Fix bug in set DAG run state workflow (#3606)</p></li> |
| <li><p>[AIRFLOW-3690] Fix bug to set state of a task for manually-triggered DAGs (#4504)</p></li> |
| <li><p>[AIRFLOW-3319] KubernetsExecutor: Need in try_number in labels if getting them later (#4163)</p></li> |
| <li><p>[AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI</p></li> |
| <li><p>[AIRFLOW-3732] Fix issue when trying to edit connection in RBAC UI</p></li> |
| <li><p>[AIRFLOW-2866] Fix missing CSRF token head when using RBAC UI (#3804)</p></li> |
| <li><p>[AIRFLOW-3259] Fix internal server error when displaying charts (#4114)</p></li> |
| <li><p>[AIRFLOW-3271] Fix issue with persistence of RBAC Permissions modified via UI (#4118)</p></li> |
| <li><p>[AIRFLOW-3141] Handle duration View for missing dag (#3984)</p></li> |
| <li><p>[AIRFLOW-2766] Respect shared datetime across tabs</p></li> |
| <li><p>[AIRFLOW-1413] Fix FTPSensor failing on error message with unexpected (#2450)</p></li> |
| <li><p>[AIRFLOW-3378] KubernetesPodOperator does not delete on timeout failure (#4218)</p></li> |
| <li><p>[AIRFLOW-3245] Fix list processing in resolve_template_files (#4086)</p></li> |
| <li><p>[AIRFLOW-2703] Catch transient DB exceptions from scheduler’s heartbeat it does not crash (#3650)</p></li> |
| <li><p>[AIRFLOW-1298] Clear UPSTREAM_FAILED using the clean cli (#3886)</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id7"> |
| <h3>Doc-only changes<a class="headerlink" href="#id7" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-XXX] GCP operators documentation clarifications (#4273)</p></li> |
| <li><p>[AIRFLOW-XXX] Docs: Fix paths to GCS transfer operator (#4479)</p></li> |
| <li><p>[AIRFLOW-XXX] Add missing GCP operators to Docs (#4260)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix Docstrings for Operators (#3820)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix inconsistent comment in example_python_operator.py (#4337)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix incorrect parameter in SFTPOperator example (#4344)</p></li> |
| <li><p>[AIRFLOW-XXX] Add missing remote logging field (#4333)</p></li> |
| <li><p>[AIRFLOW-XXX] Revise template variables documentation (#4172)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix typo in docstring of gcs_to_bq (#3833)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix display of SageMaker operators/hook docs (#4263)</p></li> |
| <li><p>[AIRFLOW-XXX] Better instructions for airflow flower (#4214)</p></li> |
| <li><p>[AIRFLOW-XXX] Make pip install commands consistent (#3752)</p></li> |
| <li><p>[AIRFLOW-XXX] Add <cite>BigQueryGetDataOperator</cite> to Integration Docs (#4063)</p></li> |
| <li><p>[AIRFLOW-XXX] Don’t spam test logs with “bad cron expression” messages (#3973)</p></li> |
| <li><p>[AIRFLOW-XXX] Update committer list based on latest TLP discussion (#4427)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix incorrect statement in contributing guide (#4104)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix Broken Link in CONTRIBUTING.md</p></li> |
| <li><p>[AIRFLOW-XXX] Update Contributing Guide - Git Hooks (#4120)</p></li> |
| <li><p>[AIRFLOW-3426] Correct Python Version Documentation Reference (#4259)</p></li> |
| <li><p>[AIRFLOW-2663] Add instructions to install SSH dependencies</p></li> |
| <li><p>[AIRFLOW-XXX] Clean up installation extra packages table (#3750)</p></li> |
| <li><p>[AIRFLOW-XXX] Remove redundant space in Kerberos (#3866)</p></li> |
| <li><p>[AIRFLOW-3086] Add extras group for google auth to setup.py (#3917)</p></li> |
| <li><p>[AIRFLOW-XXX] Add Kubernetes Dependency in Extra Packages Doc (#4281)</p></li> |
| <li><p>[AIRFLOW-3696] Add Version info to Airflow Documentation (#4512)</p></li> |
| <li><p>[AIRFLOW-XXX] Correct Typo in sensor’s exception (#4545)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix a typo of config (#4544)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix BashOperator Docstring (#4052)</p></li> |
| <li><p>[AIRFLOW-3018] Fix Minor issues in Documentation</p></li> |
| <li><p>[AIRFLOW-XXX] Fix Minor issues with Azure Cosmos Operator (#4289)</p></li> |
| <li><p>[AIRFLOW-3382] Fix incorrect docstring in DatastoreHook (#4222)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix copy&paste mistake (#4212)</p></li> |
| <li><p>[AIRFLOW-3260] Correct misleading BigQuery error (#4098)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix Typo in SFTPOperator docstring (#4016)</p></li> |
| <li><p>[AIRFLOW-XXX] Fixing the issue in Documentation (#3998)</p></li> |
| <li><p>[AIRFLOW-XXX] Fix undocumented params in S3_hook</p></li> |
| <li><p>[AIRFLOW-XXX] Fix SlackWebhookOperator execute method comment (#3963)</p></li> |
| <li><p>[AIRFLOW-3070] Refine web UI authentication-related docs (#3863)</p></li> |
| </ul> |
| </div> |
| </div> |
| <div class="section" id="airflow-1-10-1-2018-11-13"> |
| <h2>Airflow 1.10.1, 2018-11-13<a class="headerlink" href="#airflow-1-10-1-2018-11-13" title="Permalink to this headline">¶</a></h2> |
| <div class="section" id="id8"> |
| <h3>New features<a class="headerlink" href="#id8" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-2524] Airflow integration with AWS Sagemaker</p></li> |
| <li><p>[AIRFLOW-2657] Add ability to delete DAG from web ui</p></li> |
| <li><p>[AIRFLOW-2780] Adds IMAP Hook to interact with a mail server</p></li> |
| <li><p>[AIRFLOW-2794] Add delete support for Azure blob</p></li> |
| <li><p>[AIRFLOW-2912] Add operators for Google Cloud Functions</p></li> |
| <li><p>[AIRFLOW-2974] Add Start/Restart/Terminate methods Databricks Hook</p></li> |
| <li><p>[AIRFLOW-2989] No Parameter to change bootDiskType for DataprocClusterCreateOperator</p></li> |
| <li><p>[AIRFLOW-3078] Basic operators for Google Compute Engine</p></li> |
| <li><p>[AIRFLOW-3147] Update Flask-AppBuilder version</p></li> |
| <li><p>[AIRFLOW-3231] Basic operators for Google Cloud SQL (deploy / patch / delete)</p></li> |
| <li><p>[AIRFLOW-3276] Google Cloud SQL database create / patch / delete operators</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id9"> |
| <h3>Improvements<a class="headerlink" href="#id9" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-393] Add progress callbacks for FTP downloads</p></li> |
| <li><p>[AIRFLOW-520] Show Airflow version on web page</p></li> |
| <li><p>[AIRFLOW-843] Exceptions now available in context during on_failure_callback</p></li> |
| <li><p>[AIRFLOW-2476] Update tabulate dependency to v0.8.2</p></li> |
| <li><p>[AIRFLOW-2592] Bump Bleach dependency</p></li> |
| <li><p>[AIRFLOW-2622] Add “confirm=False” option to SFTPOperator</p></li> |
| <li><p>[AIRFLOW-2662] support affinity & nodeSelector policies for kubernetes executor/operator</p></li> |
| <li><p>[AIRFLOW-2709] Improve error handling in Databricks hook</p></li> |
| <li><p>[AIRFLOW-2723] Update lxml dependency to >= 4.0.</p></li> |
| <li><p>[AIRFLOW-2763] No precheck mechanism in place during worker initialisation for the connection to metadata database</p></li> |
| <li><p>[AIRFLOW-2789] Add ability to create single node cluster to DataprocClusterCreateOperator</p></li> |
| <li><p>[AIRFLOW-2797] Add ability to create Google Dataproc cluster with custom image</p></li> |
| <li><p>[AIRFLOW-2854] kubernetes_pod_operator add more configuration items</p></li> |
| <li><p>[AIRFLOW-2855] Need to Check Validity of Cron Expression When Process DAG File/Zip File</p></li> |
| <li><p>[AIRFLOW-2904] Clean an unnecessary line in airflow/executors/celery_executor.py</p></li> |
| <li><p>[AIRFLOW-2921] A trivial incorrectness in CeleryExecutor()</p></li> |
| <li><p>[AIRFLOW-2922] Potential deal-lock bug in CeleryExecutor()</p></li> |
| <li><p>[AIRFLOW-2932] GoogleCloudStorageHook - allow compression of file</p></li> |
| <li><p>[AIRFLOW-2949] Syntax Highlight for Single Quote</p></li> |
| <li><p>[AIRFLOW-2951] dag_run end_date Null after a dag is finished</p></li> |
| <li><p>[AIRFLOW-2956] Kubernetes tolerations for pod operator</p></li> |
| <li><p>[AIRFLOW-2997] Support for clustered tables in Bigquery hooks/operators</p></li> |
| <li><p>[AIRFLOW-3006] Fix error when schedule_interval=”None”</p></li> |
| <li><p>[AIRFLOW-3008] Move Kubernetes related example DAGs to contrib/example_dags</p></li> |
| <li><p>[AIRFLOW-3025] Allow to specify dns and dns-search parameters for DockerOperator</p></li> |
| <li><p>[AIRFLOW-3067] (www_rbac) Flask flash messages are not displayed properly (no background color)</p></li> |
| <li><p>[AIRFLOW-3069] Decode output of S3 file transform operator</p></li> |
| <li><p>[AIRFLOW-3072] Assign permission get_logs_with_metadata to viewer role</p></li> |
| <li><p>[AIRFLOW-3090] INFO logs are too verbose</p></li> |
| <li><p>[AIRFLOW-3103] Update Flask-Login</p></li> |
| <li><p>[AIRFLOW-3112] Align SFTP hook with SSH hook</p></li> |
| <li><p>[AIRFLOW-3119] Enable loglevel on celery worker and inherit from airflow.cfg</p></li> |
| <li><p>[AIRFLOW-3137] Make ProxyFix middleware optional</p></li> |
| <li><p>[AIRFLOW-3173] Add _cmd options for more password config options</p></li> |
| <li><p>[AIRFLOW-3177] Change scheduler_heartbeat metric from gauge to counter</p></li> |
| <li><p>[AIRFLOW-3193] Pin docker requirement version to v3</p></li> |
| <li><p>[AIRFLOW-3195] Druid Hook: Log ingestion spec and task id</p></li> |
| <li><p>[AIRFLOW-3197] EMR Hook is missing some parameters to valid on the AWS API</p></li> |
| <li><p>[AIRFLOW-3232] Make documentation for GCF Functions operator more readable</p></li> |
| <li><p>[AIRFLOW-3262] Can’t get log containing Response when using SimpleHttpOperator</p></li> |
| <li><p>[AIRFLOW-3265] Add support for “unix_socket” in connection extra for Mysql Hook</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id10"> |
| <h3>Doc-only changes<a class="headerlink" href="#id10" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-1441] Tutorial Inconsistencies Between Example Pipeline Definition and Recap</p></li> |
| <li><p>[AIRFLOW-2682] Add how-to guide(s) for how to use basic operators like BashOperator and PythonOperator</p></li> |
| <li><p>[AIRFLOW-3104] .airflowignore feature is not mentioned at all in documentation</p></li> |
| <li><p>[AIRFLOW-3237] Refactor example DAGs</p></li> |
| <li><p>[AIRFLOW-3187] Update airflow.gif file with a slower version</p></li> |
| <li><p>[AIRFLOW-3159] Update Airflow documentation on GCP Logging</p></li> |
| <li><p>[AIRFLOW-3030] Command Line docs incorrect subdir</p></li> |
| <li><p>[AIRFLOW-2990] Docstrings for Hooks/Operators are in incorrect format</p></li> |
| <li><p>[AIRFLOW-3127] Celery SSL Documentation is out-dated</p></li> |
| <li><p>[AIRFLOW-2779] Add license headers to doc files</p></li> |
| <li><p>[AIRFLOW-2779] Add project version to license</p></li> |
| </ul> |
| </div> |
| <div class="section" id="id11"> |
| <h3>Bug fixes<a class="headerlink" href="#id11" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-839] docker_operator.py attempts to log status key without first checking existence</p></li> |
| <li><p>[AIRFLOW-1104] Concurrency check in scheduler should count queued tasks as well as running</p></li> |
| <li><p>[AIRFLOW-1163] Add support for x-forwarded-* headers to support access behind AWS ELB</p></li> |
| <li><p>[AIRFLOW-1195] Cleared tasks in SubDagOperator do not trigger Parent dag_runs</p></li> |
| <li><p>[AIRFLOW-1508] Skipped state not part of State.task_states</p></li> |
| <li><p>[AIRFLOW-1762] Use key_file in SSHHook.create_tunnel()</p></li> |
| <li><p>[AIRFLOW-1837] Differing start_dates on tasks not respected by scheduler.</p></li> |
| <li><p>[AIRFLOW-1874] Support standard SQL in Check, ValueCheck and IntervalCheck BigQuery operators</p></li> |
| <li><p>[AIRFLOW-1917] print() from python operators end up with extra new line</p></li> |
| <li><p>[AIRFLOW-1970] Database cannot be initialized if an invalid fernet key is provided</p></li> |
| <li><p>[AIRFLOW-2145] Deadlock after clearing a running task</p></li> |
| <li><p>[AIRFLOW-2216] Cannot specify a profile for AWS Hook to load with s3 config file</p></li> |
| <li><p>[AIRFLOW-2574] initdb fails when mysql password contains percent sign</p></li> |
| <li><p>[AIRFLOW-2707] Error accessing log files from web UI</p></li> |
| <li><p>[AIRFLOW-2716] Replace new Python 3.7 keywords</p></li> |
| <li><p>[AIRFLOW-2744] RBAC app doesn’t integrate plugins (blueprints etc)</p></li> |
| <li><p>[AIRFLOW-2772] BigQuery hook does not allow specifying both the partition field name and table name at the same time</p></li> |
| <li><p>[AIRFLOW-2778] Bad Import in collect_dag in DagBag</p></li> |
| <li><p>[AIRFLOW-2786] Variables view fails to render if a variable has an empty key</p></li> |
| <li><p>[AIRFLOW-2799] Filtering UI objects by datetime is broken</p></li> |
| <li><p>[AIRFLOW-2800] Remove airflow/ low-hanging linting errors</p></li> |
| <li><p>[AIRFLOW-2825] S3ToHiveTransfer operator may not may able to handle GZIP file with uppercase ext in S3</p></li> |
| <li><p>[AIRFLOW-2848] dag_id is missing in metadata table “job” for LocalTaskJob</p></li> |
| <li><p>[AIRFLOW-2860] DruidHook: time variable is not updated correctly when checking for timeout</p></li> |
| <li><p>[AIRFLOW-2865] Race condition between on_success_callback and LocalTaskJob’s cleanup</p></li> |
| <li><p>[AIRFLOW-2893] Stuck dataflow job due to jobName mismatch.</p></li> |
| <li><p>[AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs</p></li> |
| <li><p>[AIRFLOW-2900] Code not visible for Packaged DAGs</p></li> |
| <li><p>[AIRFLOW-2905] Switch to regional dataflow job service.</p></li> |
| <li><p>[AIRFLOW-2907] Sendgrid - Attachments - ERROR - Object of type ‘bytes’ is not JSON serializable</p></li> |
| <li><p>[AIRFLOW-2938] Invalid ‘extra’ field in connection can raise an AttributeError when attempting to edit</p></li> |
| <li><p>[AIRFLOW-2979] Deprecated Celery Option not in Options list</p></li> |
| <li><p>[AIRFLOW-2981] TypeError in dataflow operators when using GCS jar or py_file</p></li> |
| <li><p>[AIRFLOW-2984] Cannot convert naive_datetime when task has a naive start_date/end_date</p></li> |
| <li><p>[AIRFLOW-2994] flatten_results in BigQueryOperator/BigQueryHook should default to None</p></li> |
| <li><p>[AIRFLOW-3002] ValueError in dataflow operators when using GCS jar or py_file</p></li> |
| <li><p>[AIRFLOW-3012] Email on sla miss is send only to first address on the list</p></li> |
| <li><p>[AIRFLOW-3046] ECS Operator mistakenly reports success when task is killed due to EC2 host termination</p></li> |
| <li><p>[AIRFLOW-3064] No output from <cite>airflow test</cite> due to default logging config</p></li> |
| <li><p>[AIRFLOW-3072] Only admin can view logs in RBAC UI</p></li> |
| <li><p>[AIRFLOW-3079] Improve initdb to support MSSQL Server</p></li> |
| <li><p>[AIRFLOW-3089] Google auth doesn’t work under http</p></li> |
| <li><p>[AIRFLOW-3099] Errors raised when some blocs are missing in airflow.cfg</p></li> |
| <li><p>[AIRFLOW-3109] Default user permission should contain ‘can_clear’</p></li> |
| <li><p>[AIRFLOW-3111] Confusing comments and instructions for log templates in UPDATING.md and default_airflow.cfg</p></li> |
| <li><p>[AIRFLOW-3124] Broken webserver debug mode (RBAC)</p></li> |
| <li><p>[AIRFLOW-3136] Scheduler Failing the Task retries run while processing Executor Events</p></li> |
| <li><p>[AIRFLOW-3138] Migration cc1e65623dc7 creates issues with postgres</p></li> |
| <li><p>[AIRFLOW-3161] Log Url link does not link to task instance logs in RBAC UI</p></li> |
| <li><p>[AIRFLOW-3162] HttpHook fails to parse URL when port is specified</p></li> |
| <li><p>[AIRFLOW-3183] Potential Bug in utils/dag_processing/DagFileProcessorManager.max_runs_reached()</p></li> |
| <li><p>[AIRFLOW-3203] Bugs in DockerOperator & Some operator test scripts were named incorrectly</p></li> |
| <li><p>[AIRFLOW-3238] Dags, removed from the filesystem, are not deactivated on initdb</p></li> |
| <li><p>[AIRFLOW-3268] Cannot pass SSL dictionary to mysql connection via URL</p></li> |
| <li><p>[AIRFLOW-3277] Invalid timezone transition handling for cron schedules</p></li> |
| <li><p>[AIRFLOW-3295] Require encryption in DaskExecutor when certificates are configured.</p></li> |
| <li><p>[AIRFLOW-3297] EmrStepSensor marks cancelled step as successful</p></li> |
| </ul> |
| </div> |
| </div> |
| <div class="section" id="airflow-1-10-0-2018-08-03"> |
| <h2>Airflow 1.10.0, 2018-08-03<a class="headerlink" href="#airflow-1-10-0-2018-08-03" title="Permalink to this headline">¶</a></h2> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-2870] Use abstract TaskInstance for migration</p></li> |
| <li><p>[AIRFLOW-2859] Implement own UtcDateTime (#3708)</p></li> |
| <li><p>[AIRFLOW-2140] Don’t require kubernetes for the SparkSubmit hook</p></li> |
| <li><p>[AIRFLOW-2869] Remove smart quote from default config</p></li> |
| <li><p>[AIRFLOW-2857] Fix Read the Docs env</p></li> |
| <li><p>[AIRFLOW-2817] Force explicit choice on GPL dependency</p></li> |
| <li><p>[AIRFLOW-2716] Replace async and await py3.7 keywords</p></li> |
| <li><p>[AIRFLOW-2810] Fix typo in Xcom model timestamp</p></li> |
| <li><p>[AIRFLOW-2710] Clarify fernet key value in documentation</p></li> |
| <li><p>[AIRFLOW-2606] Fix DB schema and SQLAlchemy model</p></li> |
| <li><p>[AIRFLOW-2646] Fix setup.py not to install snakebite on Python3</p></li> |
| <li><p>[AIRFLOW-2604] Add index to task_fail</p></li> |
| <li><p>[AIRFLOW-2650] Mark SchedulerJob as succeed when hitting Ctrl-c</p></li> |
| <li><p>[AIRFLOW-2678] Fix db schema unit test to remove checking fab models</p></li> |
| <li><p>[AIRFLOW-2624] Fix webserver login as anonymous</p></li> |
| <li><p>[AIRFLOW-2654] Fix incorret URL on refresh in Graph View of FAB UI</p></li> |
| <li><p>[AIRFLOW-2668] Handle missing optional cryptography dependency</p></li> |
| <li><p>[AIRFLOW-2681] Include last dag run of externally triggered DAGs in UI.</p></li> |
| <li><p>[AIRFLOW-1840] Support back-compat on old celery config</p></li> |
| <li><p>[AIRFLOW-2612][AIRFLOW-2534] Clean up Hive-related tests</p></li> |
| <li><p>[AIRFLOW-2608] Implements/Standardize custom exceptions for experimental APIs</p></li> |
| <li><p>[AIRFLOW-2607] Fix failing TestLocalClient</p></li> |
| <li><p>[AIRFLOW-2638] dbapi_hook: support REPLACE INTO</p></li> |
| <li><p>[AIRFLOW-2542][AIRFLOW-1790] Rename AWS Batch Operator queue to job_queue</p></li> |
| <li><p>[AIRFLOW-2567] Extract result from the kubernetes pod as Xcom</p></li> |
| <li><p>[AIRFLOW-XXX] Adding REA Group to readme</p></li> |
| <li><p>[AIRFLOW-2601] Allow user to specify k8s config</p></li> |
| <li><p>[AIRFLOW-2559] Azure Fileshare hook</p></li> |
| <li><p>[AIRFLOW-1786] Enforce correct behavior for soft-fail sensors</p></li> |
| <li><p>[AIRFLOW-2355] Airflow trigger tag parameters in subdag</p></li> |
| <li><p>[AIRFLOW-2613] Fix Airflow searching .zip bug</p></li> |
| <li><p>[AIRFLOW-2627] Add a sensor for Cassandra</p></li> |
| <li><p>[AIRFLOW-2634][AIRFLOW-2534] Remove dependency for impyla</p></li> |
| <li><p>[AIRFLOW-2611] Fix wrong dag volume mount path for kubernetes executor</p></li> |
| <li><p>[AIRFLOW-2562] Add Google Kubernetes Engine Operators</p></li> |
| <li><p>[AIRFLOW-2630] Fix classname in test_sql_sensor.py</p></li> |
| <li><p>[AIRFLOW-2534] Fix bug in HiveServer2Hook</p></li> |
| <li><p>[AIRFLOW-2586] Stop getting AIRFLOW_HOME value from config file in bash operator</p></li> |
| <li><p>[AIRFLOW-2605] Fix autocommit for MySqlHook</p></li> |
| <li><p>[AIRFLOW-2539][AIRFLOW-2359] Move remaining log config to configuration file</p></li> |
| <li><p>[AIRFLOW-1656] Tree view dags query changed</p></li> |
| <li><p>[AIRFLOW-2617] add imagePullPolicy config for kubernetes executor</p></li> |
| <li><p>[AIRFLOW-2429] Fix security/task/sensors/ti_deps folders flake8 error</p></li> |
| <li><p>[AIRFLOW-2550] Implements API endpoint to list DAG runs</p></li> |
| <li><p>[AIRFLOW-2512][AIRFLOW-2522] Use google-auth instead of oauth2client</p></li> |
| <li><p>[AIRFLOW-2429] Fix operators folder flake8 error</p></li> |
| <li><p>[AIRFLOW-2585] Fix several bugs in CassandraHook and CassandraToGCSOperator</p></li> |
| <li><p>[AIRFLOW-2597] Restore original dbapi.run() behavior</p></li> |
| <li><p>[AIRFLOW-2590] Fix commit in DbApiHook.run() for no-autocommit DB</p></li> |
| <li><p>[AIRFLOW-1115] fix github oauth api URL</p></li> |
| <li><p>[AIRFLOW-2587] Add TIMESTAMP type mapping to MySqlToHiveTransfer</p></li> |
| <li><p>[AIRFLOW-2591][AIRFLOW-2581] Set default value of autocommit to False in DbApiHook.run()</p></li> |
| <li><p>[AIRFLOW-59] Implement bulk_dump and bulk_load for the Postgres hook</p></li> |
| <li><p>[AIRFLOW-2533] Fix path to DAG’s on kubernetes executor workers</p></li> |
| <li><p>[AIRFLOW-2581] RFLOW-2581] Fix DbApiHook autocommit</p></li> |
| <li><p>[AIRFLOW-2578] Add option to use proxies in JiraHook</p></li> |
| <li><p>[AIRFLOW-2575] Make gcs to gcs operator work with large files</p></li> |
| <li><p>[AIRFLOW-437] Send TI context in kill zombies</p></li> |
| <li><p>[AIRFLOW-2566] Change backfill to rerun failed tasks</p></li> |
| <li><p>[AIRFLOW-1021] Fix double login for new users with LDAP</p></li> |
| <li><p>[AIRFLOW-XXX] Typo fix</p></li> |
| <li><p>[AIRFLOW-2561] Fix typo in EmailOperator</p></li> |
| <li><p>[AIRFLOW-2573] Cast BigQuery TIMESTAMP field to float</p></li> |
| <li><p>[AIRFLOW-2560] Adding support for internalIpOnly to DataprocClusterCreateOperator</p></li> |
| <li><p>[AIRFLOW-2565] templatize cluster_label</p></li> |
| <li><p>[AIRFLOW-83] add mongo hook and operator</p></li> |
| <li><p>[AIRFLOW-2558] Clear task/dag is clearing all executions</p></li> |
| <li><p>[AIRFLOW-XXX] Fix doc typos</p></li> |
| <li><p>[AIRFLOW-2513] Change <cite>bql</cite> to <cite>sql</cite> for BigQuery Hooks & Ops</p></li> |
| <li><p>[AIRFLOW-2557] Fix pagination for s3</p></li> |
| <li><p>[AIRFLOW-2545] Eliminate DeprecationWarning</p></li> |
| <li><p>[AIRFLOW-2500] Fix MySqlToHiveTransfer to transfer unsigned type properly</p></li> |
| <li><p>[AIRFLOW-2462] Change PasswordUser setter to correct syntax</p></li> |
| <li><p>[AIRFLOW-2525] Fix a bug introduced by commit dabf1b9</p></li> |
| <li><p>[AIRFLOW-2553] Add webserver.pid to .gitignore</p></li> |
| <li><p>[AIRFLOW-1863][AIRFLOW-2529] Add dag run selection widgets to gantt view</p></li> |
| <li><p>[AIRFLOW-2504] Log username correctly and add extra to search columns</p></li> |
| <li><p>[AIRFLOW-2551] Encode binary data with base64 standard rather than base64 url</p></li> |
| <li><p>[AIRFLOW-2537] Add reset-dagrun option to backfill command</p></li> |
| <li><p>[AIRFLOW-2526] dag_run.conf can override params</p></li> |
| <li><p>[AIRFLOW-2544][AIRFLOW-1967] Guard against next major release of Celery, Flower</p></li> |
| <li><p>[AIRFLOW-XXX] Add Yieldr to who is using airflow</p></li> |
| <li><p>[AIRFLOW-2547] Describe how to run tests using Docker</p></li> |
| <li><p>[AIRFLOW-2538] Update faq doc on how to reduce airflow scheduler latency</p></li> |
| <li><p>[AIRFLOW-2529] Improve graph view performance and usability</p></li> |
| <li><p>[AIRFLOW-2517] backfill support passing key values through CLI</p></li> |
| <li><p>[AIRFLOW-2532] Support logs_volume_subpath for KubernetesExecutor</p></li> |
| <li><p>[AIRFLOW-2466] consider task_id in _change_state_for_tis_without_dagrun</p></li> |
| <li><p>[AIRFLOW-2519] Fix CeleryExecutor with SQLAlchemy</p></li> |
| <li><p>[AIRFLOW-2402] Fix RBAC task log</p></li> |
| <li><p>[AIRFLOW-XXX] Add M4U to user list</p></li> |
| <li><p>[AIRFLOW-2536] docs about how to deal with airflow initdb failure</p></li> |
| <li><p>[AIRFLOW-2530] KubernetesOperator supports multiple clusters</p></li> |
| <li><p>[AIRFLOW-1499] Eliminate duplicate and unneeded code</p></li> |
| <li><p>[AIRFLOW-2521] backfill - make variable name and logging messages more acurate</p></li> |
| <li><p>[AIRFLOW-2429] Fix hook, macros folder flake8 error</p></li> |
| <li><p>[Airflow-XXX] add Prime to company list</p></li> |
| <li><p>[AIRFLOW-2525] Fix PostgresHook.copy_expert to work with “COPY FROM”</p></li> |
| <li><p>[AIRFLOW-2515] Add dependency on thrift_sasl to hive extra</p></li> |
| <li><p>[AIRFLOW-2523] Add how-to for managing GCP connections</p></li> |
| <li><p>[AIRFLOW-2510] Introduce new macros: prev_ds and next_ds</p></li> |
| <li><p>[AIRFLOW-1730] Unpickle value of XCom queried from DB</p></li> |
| <li><p>[AIRFLOW-2518] Fix broken ToC links in integration.rst</p></li> |
| <li><p>[AIRFLOW-1472] Fix SLA misses triggering on skipped tasks.</p></li> |
| <li><p>[AIRFLOW-2520] CLI - make backfill less verbose</p></li> |
| <li><p>[AIRFLOW-2107] add time_partitioning to run_query on BigQueryBaseCursor</p></li> |
| <li><p>[AIRFLOW-1057][AIRFLOW-1380][AIRFLOW-2362][2362] AIRFLOW Update DockerOperator to new API</p></li> |
| <li><p>[AIRFLOW-2415] Make airflow DAG templating render numbers</p></li> |
| <li><p>[AIRFLOW-2473] Fix wrong skip condition for TransferTests</p></li> |
| <li><p>[AIRFLOW-2472] Implement MySqlHook.bulk_dump</p></li> |
| <li><p>[AIRFLOW-2419] Use default view for subdag operator</p></li> |
| <li><p>[AIRFLOW-2498] Fix Unexpected argument in SFTP Sensor</p></li> |
| <li><p>[AIRFLOW-2509] Separate config docs into how-to guides</p></li> |
| <li><p>[AIRFLOW-2429] Add BaseExecutor back</p></li> |
| <li><p>[AIRFLOW-2429] Fix dag, example_dags, executors flake8 error</p></li> |
| <li><p>[AIRFLOW-2502] Change Single triple quotes to double for docstrings</p></li> |
| <li><p>[AIRFLOW-2503] Fix broken links in CONTRIBUTING.md</p></li> |
| <li><p>[AIRFLOW-2501] Refer to devel instructions in docs contrib guide</p></li> |
| <li><p>[AIRFLOW-2429] Fix contrib folder’s flake8 errors</p></li> |
| <li><p>[AIRFLOW-2471] Fix HiveCliHook.load_df to use unused parameters</p></li> |
| <li><p>[AIRFLOW-2495] Update celery to 4.1.1</p></li> |
| <li><p>[AIRFLOW-2429] Fix api, bin, config_templates folders flake8 error</p></li> |
| <li><p>[AIRFLOW-2493] Mark template_fields of all Operators in the API document as “templated”</p></li> |
| <li><p>[AIRFLOW-2489] Update FlaskAppBuilder to 1.11.1</p></li> |
| <li><p>[AIRFLOW-2448] Enhance HiveCliHook.load_df to work with datetime</p></li> |
| <li><p>[AIRFLOW-2487] Enhance druid ingestion hook</p></li> |
| <li><p>[AIRFLOW-2397] Support affinity policies for Kubernetes executor/operator</p></li> |
| <li><p>[AIRFLOW-2482] Add test for rewrite method in GCS Hook</p></li> |
| <li><p>[AIRFLOW-2481] Fix flaky Kubernetes test</p></li> |
| <li><p>[AIRFLOW-2479] Improve doc FAQ section</p></li> |
| <li><p>[AIRFLOW-2485] Fix Incorrect logging for Qubole Sensor</p></li> |
| <li><p>[AIRFLOW-2486] Remove unnecessary slash after port</p></li> |
| <li><p>[AIRFLOW-2429] Make Airflow flake8 compliant</p></li> |
| <li><p>[AIRFLOW-2491] Resolve flask version conflict</p></li> |
| <li><p>[AIRFLOW-2484] Remove duplicate key in MySQL to GCS Op</p></li> |
| <li><p>[AIRFLOW-2458] Add cassandra-to-gcs operator</p></li> |
| <li><p>[AIRFLOW-2477] Improve time units for task duration and landing times charts for RBAC UI</p></li> |
| <li><p>[AIRFLOW-2474] Only import snakebite if using py2</p></li> |
| <li><p>[AIRFLOW-48] Parse connection uri querystring</p></li> |
| <li><p>[AIRFLOW-2467][AIRFLOW-2] Update import direct warn message to use the module name</p></li> |
| <li><p>[AIRFLOW-XXX] Fix order of companies</p></li> |
| <li><p>[AIRFLOW-2452] Document field_dict must be OrderedDict</p></li> |
| <li><p>[AIRFLOW-2420] Azure Data Lake Hook</p></li> |
| <li><p>[AIRFLOW-2213] Add Quoble check operator</p></li> |
| <li><p>[AIRFLOW-2465] Fix wrong module names in the doc</p></li> |
| <li><p>[AIRFLOW-1929] Modifying TriggerDagRunOperator to accept execution_date</p></li> |
| <li><p>[AIRFLOW-2460] Users can now use volume mounts and volumes</p></li> |
| <li><p>[AIRFLOW-2110][AIRFLOW-2122] Enhance Http Hook</p></li> |
| <li><p>[AIRFLOW-XXX] Updated contributors list</p></li> |
| <li><p>[AIRFLOW-2435] Add launch_type to ECSOperator to allow FARGATE</p></li> |
| <li><p>[AIRFLOW-2451] Remove extra slash (‘/’) char when using wildcard in gcs_to_gcs operator</p></li> |
| <li><p>[AIRFLOW-2461] Add support for cluster scaling on dataproc operator</p></li> |
| <li><p>[AIRFLOW-2376] Fix no hive section error</p></li> |
| <li><p>[AIRFLOW-2425] Add lineage support</p></li> |
| <li><p>[AIRFLOW-2430] Extend query batching to additional slow queries</p></li> |
| <li><p>[AIRFLOW-2453] Add default nil value for kubernetes/git_subpath</p></li> |
| <li><p>[AIRFLOW-2396] Add support for resources in kubernetes operator</p></li> |
| <li><p>[AIRFLOW-2169] Encode binary data with base64 before importing to BigQuery</p></li> |
| <li><p>[AIRFLOW-XXX] Add spotahome in user list</p></li> |
| <li><p>[AIRFLOW-2457] Update FAB version requirement</p></li> |
| <li><p>[AIRFLOW-2454][Airflow 2454] Support imagePullPolicy for k8s</p></li> |
| <li><p>[AIRFLOW-2450] update supported k8s versions to 1.9 and 1.10</p></li> |
| <li><p>[AIRFLOW-2333] Add Segment Hook and TrackEventOperator</p></li> |
| <li><p>[AIRFLOW-2442][AIRFLOW-2] Airflow run command leaves database connections open</p></li> |
| <li><p>[AIRFLOW-2016] assign template_fields for Dataproc Workflow Template sub-classes, not base class</p></li> |
| <li><p>[AIRFLOW-2446] Add S3ToRedshiftTransfer into the “Integration” doc</p></li> |
| <li><p>[AIRFLOW-2449] Fix operators.py to run all test cases</p></li> |
| <li><p>[AIRFLOW-2424] Add dagrun status endpoint and increased k8s test coverage</p></li> |
| <li><p>[AIRFLOW-2441] Fix bugs in HiveCliHook.load_df</p></li> |
| <li><p>[AIRFLOW-2358][AIRFLOW-201804] Make the Kubernetes example optional</p></li> |
| <li><p>[AIRFLOW-2436] Remove cli_logger in initdb</p></li> |
| <li><p>[AIRFLOW-2444] Remove unused option(include_adhoc) in cli backfill command</p></li> |
| <li><p>[AIRFLOW-2447] Fix TestHiveMetastoreHook to run all cases</p></li> |
| <li><p>[AIRFLOW-2445] Allow templating in kubernetes operator</p></li> |
| <li><p>[AIRFLOW-2086][AIRFLOW-2393] Customize default dagrun number in tree view</p></li> |
| <li><p>[AIRFLOW-2437] Add PubNub to list of current airflow users</p></li> |
| <li><p>[AIRFLOW-XXX] Add Quantopian to list of Airflow users</p></li> |
| <li><p>[AIRFLOW-1978] Add WinRM windows operator and hook</p></li> |
| <li><p>[AIRFLOW-2427] Add tests to named hive sensor</p></li> |
| <li><p>[AIRFLOW-2412] Fix HiveCliHook.load_file to address HIVE-10541</p></li> |
| <li><p>[AIRFLOW-2431] Add the navigation bar color parameter for RBAC UI</p></li> |
| <li><p>[AIRFLOW-2407] Resolve Python undefined names</p></li> |
| <li><p>[AIRFLOW-1952] Add the navigation bar color parameter</p></li> |
| <li><p>[AIRFLOW-2222] Implement GoogleCloudStorageHook.rewrite</p></li> |
| <li><p>[AIRFLOW-2426] Add Google Cloud Storage Hook tests</p></li> |
| <li><p>[AIRFLOW-2418] Bump Flask-WTF</p></li> |
| <li><p>[AIRFLOW-2417] Wait for pod is not running to end task</p></li> |
| <li><p>[AIRFLOW-1914] Add other charset support to email utils</p></li> |
| <li><p>[AIRFLOW-XXX] Update README.md with <a class="reference external" href="mailto:Craig%40Work">Craig<span>@</span>Work</a></p></li> |
| <li><p>[AIRFLOW-1899] Fix Kubernetes tests</p></li> |
| <li><p>[AIRFLOW-1812] Update logging example</p></li> |
| <li><p>[AIRFLOW-2313] Add TTL parameters for Dataproc</p></li> |
| <li><p>[AIRFLOW-2411] add dataproc_jars to templated_fields</p></li> |
| <li><p>[AIRFLOW-XXX] Add Reddit to Airflow users</p></li> |
| <li><p>[AIRFLOW-XXX] Fix wrong table header in scheduler.rst</p></li> |
| <li><p>[AIRFLOW-2409] Supply password as a parameter</p></li> |
| <li><p>[AIRFLOW-2410][AIRFLOW-75] Set the timezone in the RBAC Web UI</p></li> |
| <li><p>[AIRFLOW-2394] default cmds and arguments in kubernetes operator</p></li> |
| <li><p>[AIRFLOW-2406] Add Apache2 License Shield to Readme</p></li> |
| <li><p>[AIRFLOW-2404] Add additional documentation for unqueued task</p></li> |
| <li><p>[AIRFLOW-2400] Add Ability to set Environment Variables for K8s</p></li> |
| <li><p>[AIRFLOW-XXX] Add Twine Labs as an Airflow user</p></li> |
| <li><p>[AIRFLOW-1853] Show only the desired number of runs in tree view</p></li> |
| <li><p>[AIRFLOW-2401] Document the use of variables in Jinja template</p></li> |
| <li><p>[AIRFLOW-2403] Fix License Headers</p></li> |
| <li><p>[AIRFLOW-1313] Fix license header</p></li> |
| <li><p>[AIRFLOW-2398] Add BounceX to list of current airflow users</p></li> |
| <li><p>[AIRFLOW-2363] Fix return type bug in TaskHandler</p></li> |
| <li><p>[AIRFLOW-2389] Create a pinot db api hook</p></li> |
| <li><p>[AIRFLOW-2390] Resolve FlaskWTFDeprecationWarning</p></li> |
| <li><p>[AIRFLOW-1933] Fix some typos</p></li> |
| <li><p>[AIRFLOW-1960] Add support for secrets in kubernetes operator</p></li> |
| <li><p>[AIRFLOW-1313] Add vertica_to_mysql operator</p></li> |
| <li><p>[AIRFLOW-1575] Add AWS Kinesis Firehose Hook for inserting batch records</p></li> |
| <li><p>[AIRFLOW-2266][AIRFLOW-2343] Remove google-cloud-dataflow dependency</p></li> |
| <li><p>[AIRFLOW-2370] Implement –use_random_password in create_user</p></li> |
| <li><p>[AIRFLOW-2348] Strip path prefix from the destination_object when source_object contains a wildcard[]</p></li> |
| <li><p>[AIRFLOW-2391] Fix to Flask 0.12.2</p></li> |
| <li><p>[AIRFLOW-2381] Fix the flaky ApiPasswordTests test</p></li> |
| <li><p>[AIRFLOW-2378] Add Groupon to list of current users</p></li> |
| <li><p>[AIRFLOW-2382] Fix wrong description for delimiter</p></li> |
| <li><p>[AIRFLOW-2380] Add support for environment variables in Spark submit operator.</p></li> |
| <li><p>[AIRFLOW-2377] Improve Sendgrid sender support</p></li> |
| <li><p>[AIRFLOW-2331] Support init action timeout on dataproc cluster create</p></li> |
| <li><p>[AIRFLOW-1835] Update docs: Variable file is json</p></li> |
| <li><p>[AIRFLOW-1781] Make search case-insensitive in LDAP group</p></li> |
| <li><p>[AIRFLOW-2042] Fix browser menu appearing over the autocomplete menu</p></li> |
| <li><p>[AIRFLOW-XXX] Remove wheelhouse files from travis not owned by travis</p></li> |
| <li><p>[AIRFLOW-2336] Use hmsclient in hive_hook</p></li> |
| <li><p>[AIRFLOW-2041] Correct Syntax in python examples</p></li> |
| <li><p>[AIRFLOW-74] SubdagOperators can consume all celeryd worker processes</p></li> |
| <li><p>[AIRFLOW-2369] Fix gcs tests</p></li> |
| <li><p>[AIRFLOW-2365] Fix autocommit attribute check</p></li> |
| <li><p>[AIRFLOW-2068] MesosExecutor allows optional Docker image</p></li> |
| <li><p>[AIRFLOW-1652] Push DatabricksRunSubmitOperator metadata into XCOM</p></li> |
| <li><p>[AIRFLOW-2234] Enable insert_rows for PrestoHook</p></li> |
| <li><p>[AIRFLOW-2208][Airflow-22208] Link to same DagRun graph from TaskInstance view</p></li> |
| <li><p>[AIRFLOW-1153] Allow HiveOperators to take hiveconfs</p></li> |
| <li><p>[AIRFLOW-775] Fix autocommit settings with Jdbc hook</p></li> |
| <li><p>[AIRFLOW-2364] Warn when setting autocommit on a connection which does not support it</p></li> |
| <li><p>[AIRFLOW-2357] Add persistent volume for the logs</p></li> |
| <li><p>[AIRFLOW-766] Skip conn.commit() when in Auto-commit</p></li> |
| <li><p>[AIRFLOW-2351] Check for valid default_args start_date</p></li> |
| <li><p>[AIRFLOW-1433] Set default rbac to initdb</p></li> |
| <li><p>[AIRFLOW-2270] Handle removed tasks in backfill</p></li> |
| <li><p>[AIRFLOW-2344] Fix <cite>connections -l</cite> to work with pipe/redirect</p></li> |
| <li><p>[AIRFLOW-2300] Add S3 Select functionarity to S3ToHiveTransfer</p></li> |
| <li><p>[AIRFLOW-1314] Cleanup the config</p></li> |
| <li><p>[AIRFLOW-1314] Polish some of the Kubernetes docs/config</p></li> |
| <li><p>[AIRFLOW-1314] Improve error handling</p></li> |
| <li><p>[AIRFLOW-1999] Add per-task GCP service account support</p></li> |
| <li><p>[AIRFLOW-1314] Rebasing against master</p></li> |
| <li><p>[AIRFLOW-1314] Small cleanup to address PR comments (#24)</p></li> |
| <li><p>[AIRFLOW-1314] Add executor_config and tests</p></li> |
| <li><p>[AIRFLOW-1314] Improve k8s support</p></li> |
| <li><p>[AIRFLOW-1314] Use VolumeClaim for transporting DAGs</p></li> |
| <li><p>[AIRFLOW-1314] Create integration testing environment</p></li> |
| <li><p>[AIRFLOW-1314] Git Mode to pull in DAGs for Kubernetes Executor</p></li> |
| <li><p>[AIRFLOW-1314] Add support for volume mounts & Secrets in Kubernetes Executor</p></li> |
| <li><p>[AIRFLOW=1314] Basic Kubernetes Mode</p></li> |
| <li><p>[AIRFLOW-2326][AIRFLOW-2222] remove contrib.gcs_copy_operator</p></li> |
| <li><p>[AIRFLOW-2328] Fix empty GCS blob in S3ToGoogleCloudStorageOperator</p></li> |
| <li><p>[AIRFLOW-2350] Fix grammar in UPDATING.md</p></li> |
| <li><p>[AIRFLOW-2302] Fix documentation</p></li> |
| <li><p>[AIRFLOW-2345] pip is not used in this setup.py</p></li> |
| <li><p>[AIRFLOW-2347] Add Banco de Formaturas to Readme</p></li> |
| <li><p>[AIRFLOW-2346] Add Investorise as official user of Airflow</p></li> |
| <li><p>[AIRFLOW-2330] Do not append destination prefix if not given</p></li> |
| <li><p>[AIRFLOW-2240][DASK] Added TLS/SSL support for the dask-distributed scheduler.</p></li> |
| <li><p>[AIRFLOW-2309] Fix duration calculation on TaskFail</p></li> |
| <li><p>[AIRFLOW-2335] fix issue with jdk8 download for ci</p></li> |
| <li><p>[AIRFLOW-2184] Add druid_checker_operator</p></li> |
| <li><p>[AIRFLOW-2299] Add S3 Select functionarity to S3FileTransformOperator</p></li> |
| <li><p>[AIRFLOW-2254] Put header as first row in unload</p></li> |
| <li><p>[AIRFLOW-610] Respect _cmd option in config before defaults</p></li> |
| <li><p>[AIRFLOW-2287] Fix incorrect ASF headers</p></li> |
| <li><p>[AIRFLOW-XXX] Add Zego as an Apache Airflow user</p></li> |
| <li><p>[AIRFLOW-952] fix save empty extra field in UI</p></li> |
| <li><p>[AIRFLOW-1325] Add ElasticSearch log handler and reader</p></li> |
| <li><p>[AIRFLOW-2301] Sync files of an S3 key with a GCS path</p></li> |
| <li><p>[AIRFLOW-2293] Fix S3FileTransformOperator to work with boto3</p></li> |
| <li><p>[AIRFLOW-3212][AIRFLOW-2314] Remove only leading slash in GCS path</p></li> |
| <li><p>[AIRFLOW-1509][AIRFLOW-442] SFTP Sensor</p></li> |
| <li><p>[AIRFLOW-2291] Add optional params to ML Engine</p></li> |
| <li><p>[AIRFLOW-1774] Allow consistent templating of arguments in MLEngineBatchPredictionOperator</p></li> |
| <li><p>[AIRFLOW-2302] Add missing operators and hooks</p></li> |
| <li><p>[AIRFLOW-2312] Docs Typo Correction: Corresponding</p></li> |
| <li><p>[AIRFLOW-1623] Trigger on_kill method in operators</p></li> |
| <li><p>[AIRFLOW-2162] When impersonating another user, pass env variables to sudo</p></li> |
| <li><p>[AIRFLOW-2304] Update quickstart doc to mention scheduler part</p></li> |
| <li><p>[AIRFLOW-1633] docker_operator needs a way to set shm_size</p></li> |
| <li><p>[AIRFLOW-1340] Add S3 to Redshift transfer operator</p></li> |
| <li><p>[AIRFLOW-2303] Lists the keys inside an S3 bucket</p></li> |
| <li><p>[AIRFLOW-2209] restore flask_login imports</p></li> |
| <li><p>[AIRFLOW-2306] Add Bonnier Broadcasting to list of current users</p></li> |
| <li><p>[AIRFLOW-2305][AIRFLOW-2027] Fix CI failure caused by []</p></li> |
| <li><p>[AIRFLOW-2281] Add support for Sendgrid categories</p></li> |
| <li><p>[AIRFLOW-2027] Only trigger sleep in scheduler after all files have parsed</p></li> |
| <li><p>[AIRFLOW-2256] SparkOperator: Add Client Standalone mode and retry mechanism</p></li> |
| <li><p>[AIRFLOW-2284] GCS to S3 operator</p></li> |
| <li><p>[AIRFLOW-2287] Update license notices</p></li> |
| <li><p>[AIRFLOW-2296] Add Cinimex DataLab to Readme</p></li> |
| <li><p>[AIRFLOW-2298] Add Kalibrr to who uses airflow</p></li> |
| <li><p>[AIRFLOW-2292] Fix docstring for S3Hook.get_wildcard_key</p></li> |
| <li><p>[AIRFLOW-XXX] Update PR template</p></li> |
| <li><p>[AIRFLOW-XXX] Remove outdated migrations.sql</p></li> |
| <li><p>[AIRFLOW-2287] Add license header to docs/Makefile</p></li> |
| <li><p>[AIRFLOW-2286] Add tokopedia to the readme</p></li> |
| <li><p>[AIRFLOW-2273] Add Discord webhook operator/hook</p></li> |
| <li><p>[AIRFLOW-2282] Fix grammar in UPDATING.md</p></li> |
| <li><p>[AIRFLOW-2200] Add snowflake operator with tests</p></li> |
| <li><p>[AIRFLOW-2178] Add handling on SLA miss errors</p></li> |
| <li><p>[AIRFLOW-2169] Fix type ‘bytes’ is not JSON serializable in python3</p></li> |
| <li><p>[AIRFLOW-2215] Pass environment to subproces.Popen in base_task_runner</p></li> |
| <li><p>[AIRFLOW-2253] Add Airflow CLI instrumentation</p></li> |
| <li><p>[AIRFLOW-2274] Fix Dataflow tests</p></li> |
| <li><p>[AIRFLOW-2269] Add Custom Ink as an Airflow user</p></li> |
| <li><p>[AIRFLOW-2259] Dataflow Hook Index out of range</p></li> |
| <li><p>[AIRFLOW-2233] Update updating.md to include the info of hdfs_sensors renaming</p></li> |
| <li><p>[AIRFLOW-2217] Add Slack webhook operator</p></li> |
| <li><p>[AIRFLOW-1729] improve dagBag time</p></li> |
| <li><p>[AIRFLOW-2264] Improve create_user cli help message</p></li> |
| <li><p>[AIRFLOW-2260] [AIRFLOW-2260] SSHOperator add command template .sh files</p></li> |
| <li><p>[AIRFLOW-2261] Check config/env for remote base log folder</p></li> |
| <li><p>[AIRFLOW-2258] Allow import of Parquet-format files into BigQuery</p></li> |
| <li><p>[AIRFLOW-1430] Include INSTALL instructions to avoid GPL</p></li> |
| <li><p>[AIRFLOW-1430] Solve GPL dependency</p></li> |
| <li><p>[AIRFLOW-2251] Add Thinknear as an Airflow user</p></li> |
| <li><p>[AIRFLOW-2244] bugfix: remove legacy LongText code from models.py</p></li> |
| <li><p>[AIRFLOW-2247] Fix RedshiftToS3Transfer not to fail with ValueError</p></li> |
| <li><p>[AIRFLOW-2249] Add side-loading support for Zendesk Hook</p></li> |
| <li><p>[AIRFLOW-XXX] Add Qplum to Airflow users</p></li> |
| <li><p>[AIRFLOW-2228] Enhancements in ValueCheckOperator</p></li> |
| <li><p>[AIRFLOW-1206] Typos</p></li> |
| <li><p>[AIRFLOW-2060] Update pendulum version to 1.4.4</p></li> |
| <li><p>[AIRFLOW-2248] Fix wrong param name in RedshiftToS3Transfer doc</p></li> |
| <li><p>[AIRFLOW-1433][AIRFLOW-85] New Airflow Webserver UI with RBAC support</p></li> |
| <li><p>[AIRFLOW-1235] Fix webserver’s odd behaviour</p></li> |
| <li><p>[AIRFLOW-1460] Allow restoration of REMOVED TI’s</p></li> |
| <li><p>[airflow-2235] Fix wrong docstrings in two operators</p></li> |
| <li><p>[AIRFLOW-XXX] Fix chronological order for companies using Airflow</p></li> |
| <li><p>[AIRFLOW-2124] Upload Python file to a bucket for Dataproc</p></li> |
| <li><p>[AIRFLOW-2212] Fix ungenerated sensor API reference</p></li> |
| <li><p>[AIRFLOW-2226] Rename google_cloud_storage_default to google_cloud_default</p></li> |
| <li><p>[AIRFLOW-2211] Rename hdfs_sensors.py to hdfs_sensor.py for consistency</p></li> |
| <li><p>[AIRFLOW-2225] Update document to include DruidDbApiHook</p></li> |
| <li><p>[Airflow-2202] Add filter support in HiveMetastoreHook().max_partition()</p></li> |
| <li><p>[AIRFLOW-2220] Remove duplicate numeric list entry in security.rst</p></li> |
| <li><p>[AIRFLOW-XXX] Update tutorial documentation</p></li> |
| <li><p>[AIRFLOW-2215] Update celery task to preserve environment variables and improve logging on exception</p></li> |
| <li><p>[AIRFLOW-2185] Use state instead of query param</p></li> |
| <li><p>[AIRFLOW-2183] Refactor DruidHook to enable sql</p></li> |
| <li><p>[AIRFLOW-2203] Defer cycle detection</p></li> |
| <li><p>[AIRFLOW-2203] Remove Useless Commands.</p></li> |
| <li><p>[AIRFLOW-2203] Cache signature in apply_defaults</p></li> |
| <li><p>[AIRFLOW-2203] Speed up Operator Resources</p></li> |
| <li><p>[AIRFLOW-2203] Cache static rules (trigger/weight)</p></li> |
| <li><p>[AIRFLOW-2203] Store task ids as sets not lists</p></li> |
| <li><p>[AIRFLOW-2205] Remove unsupported args from JdbcHook doc</p></li> |
| <li><p>[AIRFLOW-2207] Fix flaky test that uses app.cached_app()</p></li> |
| <li><p>[AIRFLOW-2206] Remove unsupported args from JdbcOperator doc</p></li> |
| <li><p>[AIRFLOW-2140] Add Kubernetes scheduler to SparkSubmitOperator</p></li> |
| <li><p>[AIRFLOW-XXX] Add Xero to list of users</p></li> |
| <li><p>[AIRFLOW-2204] Fix webserver debug mode</p></li> |
| <li><p>[AIRFLOW-102] Fix test_complex_template always succeeds</p></li> |
| <li><p>[AIRFLOW-442] Add SFTPHook</p></li> |
| <li><p>[AIRFLOW-2169] Add schema to MySqlToGoogleCloudStorageOperator</p></li> |
| <li><p>[AIRFLOW-2184][AIRFLOW-2138] Google Cloud Storage allow wildcards</p></li> |
| <li><p>[AIRFLOW-1588] Cast Variable value to string</p></li> |
| <li><p>[AIRFLOW-2199] Fix invalid reference to logger</p></li> |
| <li><p>[AIRFLOW-2191] Change scheduler heartbeat logs from info to debug</p></li> |
| <li><p>[AIRFLOW-2106] SalesForce hook sandbox option</p></li> |
| <li><p>[AIRFLOW-2197] Silence hostname_callable config error message</p></li> |
| <li><p>[AIRFLOW-2150] Use lighter call in HiveMetastoreHook().max_partition()</p></li> |
| <li><p>[AIRFLOW-2186] Change the way logging is carried out in few ops</p></li> |
| <li><p>[AIRFLOW-2181] Convert password_auth and test_password_endpoints from DOS to UNIX</p></li> |
| <li><p>[AIRFLOW-2187] Fix Broken Travis CI due to AIRFLOW-2123</p></li> |
| <li><p>[AIRFLOW-2175] Check that filepath is not None</p></li> |
| <li><p>[AIRFLOW-2173] Don’t check task IDs for concurrency reached check</p></li> |
| <li><p>[AIRFLOW-2168] Remote logging for Azure Blob Storage</p></li> |
| <li><p>[AIRFLOW-XXX] Add DocuTAP to list of users</p></li> |
| <li><p>[AIRFLOW-2176] Change the way logging is carried out in BQ Get Data Operator</p></li> |
| <li><p>[AIRFLOW-2177] Add mock test for GCS Download op</p></li> |
| <li><p>[AIRFLOW-2123] Install CI dependencies from setup.py</p></li> |
| <li><p>[AIRFLOW-2129] Presto hook calls _parse_exception_message but defines _get_pretty_exception_message</p></li> |
| <li><p>[AIRFLOW-2174] Fix typos and wrongly rendered documents</p></li> |
| <li><p>[AIRFLOW-2171] Store delegated credentials</p></li> |
| <li><p>[AIRFLOW-2166] Restore BQ run_query dialect param</p></li> |
| <li><p>[AIRFLOW-2163] Add HBC Digital to users of airflow</p></li> |
| <li><p>[AIRFLOW-2065] Fix race-conditions when creating loggers</p></li> |
| <li><p>[AIRFLOW-2147] Plugin manager: added ‘sensors’ attribute</p></li> |
| <li><p>[AIRFLOW-2059] taskinstance query is awful, un-indexed, and does not scale</p></li> |
| <li><p>[AIRFLOW-2159] Fix a few typos in salesforce_hook</p></li> |
| <li><p>[AIRFLOW-2132] Add step to initialize database</p></li> |
| <li><p>[AIRFLOW-2160] Fix bad rowid deserialization</p></li> |
| <li><p>[AIRFLOW-2161] Add Vevo to list of companies using Airflow</p></li> |
| <li><p>[AIRFLOW-2149] Add link to apache Beam documentation to create self executing Jar</p></li> |
| <li><p>[AIRFLOW-2151] Allow getting the session from AwsHook</p></li> |
| <li><p>[AIRFLOW-2097] tz referenced before assignment</p></li> |
| <li><p>[AIRFLOW-2152] Add Multiply to list of companies using Airflow</p></li> |
| <li><p>[AIRFLOW-1551] Add operator to trigger Jenkins job</p></li> |
| <li><p>[AIRFLOW-2034] Fix mixup between %s and {} when using str.format Convention is to use .format for string formating oustide logging, else use lazy format See comment in related issue <a class="reference external" href="https://github.com/apache/airflow/pull/2823/files">https://github.com/apache/airflow/pull/2823/files</a> Identified problematic case using following command line .git/COMMIT_EDITMSG:<cite>grep -r ‘%s’./* | grep ‘.format(‘</cite></p></li> |
| <li><p>[AIRFLOW-2102] Add custom_args to Sendgrid personalizations</p></li> |
| <li><p>[AIRFLOW-1035][AIRFLOW-1053] import unicode_literals to parse Unicode in HQL</p></li> |
| <li><p>[AIRFLOW-2127] Keep loggers during DB migrations</p></li> |
| <li><p>[AIRFLOW-2146] Resolve issues with BQ using DbApiHook methods</p></li> |
| <li><p>[AIRFLOW-2087] Scheduler Report shows incorrect Total task number</p></li> |
| <li><p>[AIRFLOW-2139] Remove unncecessary boilerplate to get DataFrame using pandas_gbq</p></li> |
| <li><p>[AIRFLOW-2125] Using binary package psycopg2-binary</p></li> |
| <li><p>[AIRFLOW-2142] Include message on mkdir failure</p></li> |
| <li><p>[AIRFLOW-1615] SSHHook: use port specified by Connection</p></li> |
| <li><p>[AIRFLOW-2122] Handle boolean values in sshHook</p></li> |
| <li><p>[AIRFLOW-XXX] Add Tile to the list of users</p></li> |
| <li><p>[AIRFLOW-2130] Add missing Operators to API Reference docs</p></li> |
| <li><p>[AIRFLOW-XXX] Add timeout units (seconds)</p></li> |
| <li><p>[AIRFLOW-2134] Add Alan to the list of companies that use Airflow</p></li> |
| <li><p>[AIRFLOW-2133] Remove references to GitHub issues in CONTRIBUTING</p></li> |
| <li><p>[AIRFLOW-2131] Remove confusing AirflowImport docs</p></li> |
| <li><p>[AIRFLOW-1852] Allow hostname to be overridable.</p></li> |
| <li><p>[AIRFLOW-2126] Add Bluecore to active users</p></li> |
| <li><p>[AIRFLOW-1618] Add feature to create GCS bucket</p></li> |
| <li><p>[AIRFLOW-2108] Fix log indentation in BashOperator</p></li> |
| <li><p>[AIRFLOW-2115] Fix doc links to PythonHosted</p></li> |
| <li><p>[AIRFLOW-XXX] Add contributor from Easy company</p></li> |
| <li><p>[AIRFLOW-1882] Add ignoreUnknownValues option to gcs_to_bq operator</p></li> |
| <li><p>[AIRFLOW-2089] Add on kill for SparkSubmit in Standalone Cluster</p></li> |
| <li><p>[AIRFLOW-2113] Address missing DagRun callbacks Given that the handle_callback method belongs to the DAG object, we are able to get the list of task directly with get_task and reduce the communication with the database, making airflow more lightweight.</p></li> |
| <li><p>[AIRFLOW-2112] Fix svg width for Recent Tasks on UI.</p></li> |
| <li><p>[AIRFLOW-2116] Set CI Cloudant version to <2.0</p></li> |
| <li><p>[AIRFLOW-XXX] Add PMC to list of companies using Airflow</p></li> |
| <li><p>[AIRFLOW-2100] Fix Broken Documentation Links</p></li> |
| <li><p>[AIRFLOW-1404] Add ‘flatten_results’ & ‘maximum_bytes_billed’ to BQ Operator</p></li> |
| <li><p>[AIRFLOW-800] Initialize valid Google BigQuery Connection</p></li> |
| <li><p>[AIRFLOW-1319] Fix misleading SparkSubmitOperator and SparkSubmitHook docstring</p></li> |
| <li><p>[AIRFLOW-1983] Parse environment parameter as template</p></li> |
| <li><p>[AIRFLOW-2095] Add operator to create External BigQuery Table</p></li> |
| <li><p>[AIRFLOW-2085] Add SparkJdbc operator</p></li> |
| <li><p>[AIRFLOW-1002] Add ability to clean all dependencies of removed DAG</p></li> |
| <li><p>[AIRFLOW-2094] Jinjafied project_id, region & zone in DataProc{*} Operators</p></li> |
| <li><p>[AIRFLOW-2092] Fixed incorrect parameter in docstring for FTPHook</p></li> |
| <li><p>[AIRFLOW-XXX] Add SocialCops to Airflow users</p></li> |
| <li><p>[AIRFLOW-2088] Fix duplicate keys in MySQL to GCS Helper function</p></li> |
| <li><p>[AIRFLOW-2091] Fix incorrect docstring parameter in BigQuery Hook</p></li> |
| <li><p>[AIRFLOW-2090] Fix typo in DataStore Hook</p></li> |
| <li><p>[AIRFLOW-1157] Fix missing pools crashing the scheduler</p></li> |
| <li><p>[AIRFLOW-713] Jinjafy {EmrCreateJobFlow,EmrAddSteps}Operator attributes</p></li> |
| <li><p>[AIRFLOW-2083] Docs: Use “its” instead of “it’s” where appropriate</p></li> |
| <li><p>[AIRFLOW-2066] Add operator to create empty BQ table</p></li> |
| <li><p>[AIRFLOW-XXX] add Karmic to list of companies</p></li> |
| <li><p>[AIRFLOW-2073] Make FileSensor fail when the file doesn’t exist</p></li> |
| <li><p>[AIRFLOW-2078] Improve task_stats and dag_stats performance</p></li> |
| <li><p>[AIRFLOW-2080] Use a log-out icon instead of a power button</p></li> |
| <li><p>[AIRFLOW-2077] Fetch all pages of list_objects_v2 response</p></li> |
| <li><p>[AIRFLOW-XXX] Add TM to list of companies</p></li> |
| <li><p>[AIRFLOW-1985] Impersonation fixes for using <cite>run_as_user</cite></p></li> |
| <li><p>[AIRFLOW-2018][AIRFLOW-2] Make Sensors backward compatible</p></li> |
| <li><p>[AIRFLOW-XXX] Fix typo in concepts doc (dag_md)</p></li> |
| <li><p>[AIRFLOW-2069] Allow Bytes to be uploaded to S3</p></li> |
| <li><p>[AIRFLOW-2074] Fix log var name in GHE auth</p></li> |
| <li><p>[AIRFLOW-1927] Convert naive datetimes for TaskInstances</p></li> |
| <li><p>[AIRFLOW-1760] Password auth for experimental API</p></li> |
| <li><p>[AIRFLOW-2038] Add missing kubernetes dependency for dev</p></li> |
| <li><p>[AIRFLOW-2040] Escape special chars in task instance logs URL</p></li> |
| <li><p>[AIRFLOW-1968][AIRFLOW-1520] Add role_arn and aws_account_id/aws_iam_role support back to aws hook</p></li> |
| <li><p>[AIRFLOW-2048] Fix task instance failure string formatting</p></li> |
| <li><p>[AIRFLOW-2046] Fix kerberos error to work with python 3.x</p></li> |
| <li><p>[AIRFLOW-2063] Add missing docs for GCP</p></li> |
| <li><p>[AIRFLOW-XXX] Fix typo in docs</p></li> |
| <li><p>[AIRFLOW-1793] Use docker_url instead of invalid base_url</p></li> |
| <li><p>[AIRFLOW-2055] Elaborate on slightly ambiguous documentation</p></li> |
| <li><p>[AIRFLOW-2039] BigQueryOperator supports priority property</p></li> |
| <li><p>[AIRFLOW-2053] Fix quote character bug in BQ hook</p></li> |
| <li><p>[AIRFLOW-2057] Add Overstock to list of companies</p></li> |
| <li><p>[AIRFLOW-XXX] Add Plaid to Airflow users</p></li> |
| <li><p>[AIRFLOW-2044] Add SparkSubmitOperator to documentation</p></li> |
| <li><p>[AIRFLOW-2037] Add methods to get Hash values of a GCS object</p></li> |
| <li><p>[AIRFLOW-2050] Fix Travis permission problem</p></li> |
| <li><p>[AIRFLOW-2043] Add Intercom to list of companies</p></li> |
| <li><p>[AIRFLOW-2023] Add debug logging around number of queued files</p></li> |
| <li><p>[AIRFLOW-XXX] Add Pernod-ricard as a airflow user</p></li> |
| <li><p>[AIRFLOW-1453] Add ‘steps’ into template_fields in EmrAddSteps</p></li> |
| <li><p>[AIRFLOW-2015] Add flag for interactive runs</p></li> |
| <li><p>[AIRFLOW-1895] Fix primary key integrity for mysql</p></li> |
| <li><p>[AIRFLOW-2030] Fix KeyError:<cite>i</cite> in DbApiHook for insert</p></li> |
| <li><p>[AIRFLOW-1943] Add External BigQuery Table feature</p></li> |
| <li><p>[AIRFLOW-2033] Add Google Cloud Storage List Operator</p></li> |
| <li><p>[AIRFLOW-2006] Add local log catching to kubernetes operator</p></li> |
| <li><p>[AIRFLOW-2031] Add missing gcp_conn_id in the example in DataFlow docstrings</p></li> |
| <li><p>[AIRFLOW-2029] Fix AttributeError in BigQueryPandasConnector</p></li> |
| <li><p>[AIRFLOW-2028] Add JobTeaser to official users list</p></li> |
| <li><p>[AIRFLOW-2016] Add support for Dataproc Workflow Templates</p></li> |
| <li><p>[AIRFLOW-2025] Reduced Logging verbosity</p></li> |
| <li><p>[AIRFLOW-1267][AIRFLOW-1874] Add dialect parameter to BigQueryHook</p></li> |
| <li><p>[AIRFLOW-XXX] Fixed a typo</p></li> |
| <li><p>[AIRFLOW-XXX] Typo node to nodes</p></li> |
| <li><p>[AIRFLOW-2019] Update DataflowHook for updating Streaming type job</p></li> |
| <li><p>[AIRFLOW-2017][Airflow 2017] adding query output to PostgresOperator</p></li> |
| <li><p>[AIRFLOW-1889] Split sensors into separate files</p></li> |
| <li><p>[AIRFLOW-1950] Optionally pass xcom_pull task_ids</p></li> |
| <li><p>[AIRFLOW-1755] Allow mount below root</p></li> |
| <li><p>[AIRFLOW-511][Airflow 511] add success/failure callbacks on dag level</p></li> |
| <li><p>[AIRFLOW-192] Add weight_rule param to BaseOperator</p></li> |
| <li><p>[AIRFLOW-2008] Use callable for python column defaults</p></li> |
| <li><p>[AIRFLOW-1984] Fix to AWS Batch operator</p></li> |
| <li><p>[AIRFLOW-2000] Support non-main dataflow job class</p></li> |
| <li><p>[AIRFLOW-2003] Use flask-caching instead of flask-cache</p></li> |
| <li><p>[AIRFLOW-2002] Do not swallow exception on logging import</p></li> |
| <li><p>[AIRFLOW-2004] Import flash from flask not flask.login</p></li> |
| <li><p>[AIRFLOW-1997] Fix GCP operator doc strings</p></li> |
| <li><p>[AIRFLOW-1996] Update DataflowHook waitfordone for Streaming type job[]</p></li> |
| <li><p>[AIRFLOW-1995][Airflow 1995] add on_kill method to SqoopOperator</p></li> |
| <li><p>[AIRFLOW-1770] Allow HiveOperator to take in a file</p></li> |
| <li><p>[AIRFLOW-1994] Change background color of Scheduled state Task Instances</p></li> |
| <li><p>[AIRFLOW-1436][AIRFLOW-1475] EmrJobFlowSensor considers Cancelled step as Successful</p></li> |
| <li><p>[AIRFLOW-1517] Kubernetes operator PR fixes</p></li> |
| <li><p>[AIRFLOW-1517] addressed PR comments</p></li> |
| <li><p>[AIRFLOW-1517] started documentation of k8s operator</p></li> |
| <li><p>[AIRFLOW-1517] Restore authorship of resources</p></li> |
| <li><p>[AIRFLOW-1517] Remove authorship of resources</p></li> |
| <li><p>[AIRFLOW-1517] Add minikube for kubernetes integration tests</p></li> |
| <li><p>[AIRFLOW-1517] Restore authorship of resources</p></li> |
| <li><p>[AIRFLOW-1517] fixed license issues</p></li> |
| <li><p>[AIRFLOW-1517] Created more accurate failures for kube cluster issues</p></li> |
| <li><p>[AIRFLOW-1517] Remove authorship of resources</p></li> |
| <li><p>[AIRFLOW-1517] Add minikube for kubernetes integration tests</p></li> |
| <li><p>[AIRFLOW-1988] Change BG color of None state TIs</p></li> |
| <li><p>[AIRFLOW-790] Clean up TaskInstances without DagRuns</p></li> |
| <li><p>[AIRFLOW-1949] Fix var upload, str() produces “b’…’” which is not json</p></li> |
| <li><p>[AIRFLOW-1930] Convert func.now() to timezone.utcnow()</p></li> |
| <li><p>[AIRFLOW-1688] Support load.time_partitioning in bigquery_hook</p></li> |
| <li><p>[AIRFLOW-1975] Make TriggerDagRunOperator callback optional</p></li> |
| <li><p>[AIRFLOW-1480] Render template attributes for ExternalTaskSensor fields</p></li> |
| <li><p>[AIRFLOW-1958] Add kwargs to send_email</p></li> |
| <li><p>[AIRFLOW-1976] Fix for missing log/logger attribute FileProcessHandler</p></li> |
| <li><p>[AIRFLOW-1982] Fix Executor event log formatting</p></li> |
| <li><p>[AIRFLOW-1971] Propagate hive config on impersonation</p></li> |
| <li><p>[AIRFLOW-1969] Always use HTTPS URIs for Google OAuth2</p></li> |
| <li><p>[AIRFLOW-1954] Add DataFlowTemplateOperator</p></li> |
| <li><p>[AIRFLOW-1963] Add config for HiveOperator mapred_queue</p></li> |
| <li><p>[AIRFLOW-1946][AIRFLOW-1855] Create a BigQuery Get Data Operator</p></li> |
| <li><p>[AIRFLOW-1953] Add labels to dataflow operators</p></li> |
| <li><p>[AIRFLOW-1967] Update Celery to 4.0.2</p></li> |
| <li><p>[AIRFLOW-1964] Add Upsight to list of Airflow users</p></li> |
| <li><p>[AIRFLOW-XXX] Changelog for 1.9.0</p></li> |
| <li><p>[AIRFLOW-1470] Implement BashSensor operator</p></li> |
| <li><p>[AIRFLOW-XXX] Pin sqlalchemy dependency</p></li> |
| <li><p>[AIRFLOW-1955] Do not reference unassigned variable</p></li> |
| <li><p>[AIRFLOW-1957] Add contributor to BalanceHero in Readme</p></li> |
| <li><p>[AIRFLOW-1517] Restore authorship of secrets and init container</p></li> |
| <li><p>[AIRFLOW-1517] Remove authorship of secrets and init container</p></li> |
| <li><p>[AIRFLOW-1935] Add BalanceHero to readme</p></li> |
| <li><p>[AIRFLOW-1939] add astronomer contributors</p></li> |
| <li><p>[AIRFLOW-1517] Kubernetes Operator</p></li> |
| <li><p>[AIRFLOW-1928] Fix @once with catchup=False</p></li> |
| <li><p>[AIRFLOW-1937] Speed up scheduling by committing in batch</p></li> |
| <li><p>[AIRFLOW-1821] Enhance default logging config by removing extra loggers</p></li> |
| <li><p>[AIRFLOW-1904] Correct DAG fileloc to the right filepath</p></li> |
| <li><p>[AIRFLOW-1909] Update docs with supported versions of MySQL server</p></li> |
| <li><p>[AIRFLOW-1915] Relax flask-wtf dependency specification</p></li> |
| <li><p>[AIRFLOW-1920] Update CONTRIBUTING.md to reflect enforced linting rules</p></li> |
| <li><p>[AIRFLOW-1942] Update Sphinx docs to remove deprecated import structure</p></li> |
| <li><p>[AIRFLOW-1846][AIRFLOW-1697] Hide Ad Hoc Query behind secure_mode config</p></li> |
| <li><p>[AIRFLOW-1948] Include details for on_kill failure</p></li> |
| <li><p>[AIRFLOW-1938] Clean up unused exception</p></li> |
| <li><p>[AIRFLOW-1932] Add GCP Pub/Sub Pull and Ack</p></li> |
| <li><p>[AIRFLOW-XXX] Purge coveralls</p></li> |
| <li><p>[AIRFLOW-XXX] Remove unused coveralls token</p></li> |
| <li><p>[AIRFLOW-1938] Remove tag version check in setup.py</p></li> |
| <li><p>[AIRFLOW-1916] Don’t upload logs to remote from <cite>run –raw</cite></p></li> |
| <li><p>[AIRFLOW-XXX] Fix failing PubSub tests on Python3</p></li> |
| <li><p>[AIRFLOW-XXX] Upgrade to python 3.5 and disable dask tests</p></li> |
| <li><p>[AIRFLOW-1913] Add new GCP PubSub operators</p></li> |
| <li><p>[AIRFLOW-1525] Fix minor LICENSE and NOTICE issues</p></li> |
| <li><p>[AIRFLOW-1687] fix fernet error without encryption</p></li> |
| <li><p>[AIRFLOW-1912] airflow.processor should not propagate logging</p></li> |
| <li><p>[AIRFLOW-1911] Rename celeryd_concurrency</p></li> |
| <li><p>[AIRFLOW-1885] Fix IndexError in ready_prefix_on_cmdline</p></li> |
| <li><p>[AIRFLOW-1854] Improve Spark Submit operator for standalone cluster mode</p></li> |
| <li><p>[AIRFLOW-1908] Fix celery broker options config load</p></li> |
| <li><p>[AIRFLOW-1907] Pass max_ingestion_time to Druid hook</p></li> |
| <li><p>[AIRFLOW-1909] Add away to list of users</p></li> |
| <li><p>[AIRFLOW-1893][AIRFLOW-1901] Propagate PYTHONPATH when using impersonation</p></li> |
| <li><p>[AIRFLOW-1892] Modify BQ hook to extract data filtered by column</p></li> |
| <li><p>[AIRFLOW-1829] Support for schema updates in query jobs</p></li> |
| <li><p>[AIRFLOW-1840] Make celery configuration congruent with Celery 4</p></li> |
| <li><p>[AIRFLOW-1878] Fix stderr/stdout redirection for tasks</p></li> |
| <li><p>[AIRFLOW-1897][AIRFLOW-1873] Task Logs for running instance not visible in WebUI</p></li> |
| <li><p>[AIRFLOW-1896] FIX bleach <> html5lib incompatibility</p></li> |
| <li><p>[AIRFLOW-1884][AIRFLOW-1059] Reset orphaned task state for external dagruns</p></li> |
| <li><p>[AIRFLOW-XXX] Fix typo in comment</p></li> |
| <li><p>[AIRFLOW-1869] Do not emit spurious warning on missing logs</p></li> |
| <li><p>[AIRFLOW-1888] Add AWS Redshift Cluster Sensor</p></li> |
| <li><p>[AIRFLOW-1887] Renamed endpoint url variable</p></li> |
| <li><p>[AIRFLOW-1873] Set TI.try_number to right value depending TI state</p></li> |
| <li><p>[AIRFLOW-1891] Fix non-ascii typo in default configuration template</p></li> |
| <li><p>[AIRFLOW-1879] Handle ti log entirely within ti</p></li> |
| <li><p>[AIRFLOW-1869] Write more error messages into gcs and file logs</p></li> |
| <li><p>[AIRFLOW-1876] Write subtask id to task log header</p></li> |
| <li><p>[AIRFLOW-1554] Fix wrong DagFileProcessor termination method call</p></li> |
| <li><p>[AIRFLOW-342] Do not use amqp, rpc as result backend</p></li> |
| <li><p>[AIRFLOW-966] Make celery broker_transport_options configurable</p></li> |
| <li><p>[AIRFLOW-1881] Make operator log in task log</p></li> |
| <li><p>[AIRFLOW-XXX] Added DataReply to the list of Airflow Users</p></li> |
| <li><p>[AIRFLOW-1883] Get File Size for objects in Google Cloud Storage</p></li> |
| <li><p>[AIRFLOW-1872] Set context for all handlers including parents</p></li> |
| <li><p>[AIRFLOW-1855][AIRFLOW-1866] Add GCS Copy Operator to copy multiple files</p></li> |
| <li><p>[AIRFLOW-1870] Enable flake8 tests</p></li> |
| <li><p>[AIRFLOW-1785] Enable Python 3 tests</p></li> |
| <li><p>[AIRFLOW-1850] Copy cmd before masking</p></li> |
| <li><p>[AIRFLOW-1665] Reconnect on database errors</p></li> |
| <li><p>[AIRFLOW-1559] Dispose SQLAlchemy engines on exit</p></li> |
| <li><p>[AIRFLOW-1559] Close file handles in subprocesses</p></li> |
| <li><p>[AIRFLOW-1559] Make database pooling optional</p></li> |
| <li><p>[AIRFLOW-1848][Airflow-1848] Fix DataFlowPythonOperator py_file extension doc comment</p></li> |
| <li><p>[AIRFLOW-1843] Add Google Cloud Storage Sensor with prefix</p></li> |
| <li><p>[AIRFLOW-1803] Time zone documentation</p></li> |
| <li><p>[AIRFLOW-1826] Update views to use timezone aware objects</p></li> |
| <li><p>[AIRFLOW-1827] Fix api endpoint date parsing</p></li> |
| <li><p>[AIRFLOW-1806] Use naive datetime when using cron</p></li> |
| <li><p>[AIRFLOW-1809] Update tests to use timezone aware objects</p></li> |
| <li><p>[AIRFLOW-1806] Use naive datetime for cron scheduling</p></li> |
| <li><p>[AIRFLOW-1807] Force use of time zone aware db fields</p></li> |
| <li><p>[AIRFLOW-1808] Convert all utcnow() to time zone aware</p></li> |
| <li><p>[AIRFLOW-1804] Add time zone configuration options</p></li> |
| <li><p>[AIRFLOW-1802] Convert database fields to timezone aware</p></li> |
| <li><p>[AIRFLOW-XXX] Add dask lock files to excludes</p></li> |
| <li><p>[AIRFLOW-1790] Add support for AWS Batch operator</p></li> |
| <li><p>[AIRFLOW-XXX] Update README.md</p></li> |
| <li><p>[AIRFLOW-1820] Remove timestamp from metric name</p></li> |
| <li><p>[AIRFLOW-1810] Remove unused mysql import in migrations.</p></li> |
| <li><p>[AIRFLOW-1838] Properly log collect_dags exception</p></li> |
| <li><p>[AIRFLOW-1842] Fixed Super class name for the gcs to gcs copy operator</p></li> |
| <li><p>[AIRFLOW-1845] Modal background now covers long or tall pages</p></li> |
| <li><p>[AIRFLOW-1229] Add link to Run Id, incl execution_date</p></li> |
| <li><p>[AIRFLOW-1842] Add gcs to gcs copy operator with renaming if required</p></li> |
| <li><p>[AIRFLOW-1841] change False to None in operator and hook</p></li> |
| <li><p>[AIRFLOW-1839] Fix more bugs in S3Hook boto -> boto3 migration</p></li> |
| <li><p>[AIRFLOW-1830] Support multiple domains in Google authentication backend</p></li> |
| <li><p>[AIRFLOW-1831] Add driver-classpath spark submit</p></li> |
| <li><p>[AIRFLOW-1795] Correctly call S3Hook after migration to boto3</p></li> |
| <li><p>[AIRFLOW-1811] Fix render Druid operator</p></li> |
| <li><p>[AIRFLOW-1819] Fix slack operator unittest bug</p></li> |
| <li><p>[AIRFLOW-1805] Allow Slack token to be passed through connection</p></li> |
| <li><p>[AIRFLOW-1816] Add region param to Dataproc operators</p></li> |
| <li><p>[AIRFLOW-868] Add postgres_to_gcs operator and unittests</p></li> |
| <li><p>[AIRFLOW-1613] make mysql_to_gcs_operator py3 compatible</p></li> |
| <li><p>[AIRFLOW-1817] use boto3 for s3 dependency</p></li> |
| <li><p>[AIRFLOW-1813] Bug SSH Operator empty buffer</p></li> |
| <li><p>[AIRFLOW-1801][AIRFLOW-288] Url encode execution dates</p></li> |
| <li><p>[AIRFLOW-1563] Catch OSError while symlinking the latest log directory</p></li> |
| <li><p>[AIRFLOW-1794] Remove uses of Exception.message for Python 3</p></li> |
| <li><p>[AIRFLOW-1799] Fix logging line which raises errors</p></li> |
| <li><p>[AIRFLOW-1102] Upgrade Gunicorn >=19.4.0</p></li> |
| <li><p>[AIRFLOW-1756] Fix S3TaskHandler to work with Boto3-based S3Hook</p></li> |
| <li><p>[AIRFLOW-1797] S3Hook.load_string didn’t work on Python3</p></li> |
| <li><p>[AIRFLOW-646] Add docutils to setup_requires</p></li> |
| <li><p>[AIRFLOW-1792] Missing intervals DruidOperator</p></li> |
| <li><p>[AIRFLOW-1789][AIRFLOW-1712] Log SSHOperator stderr to log.warning</p></li> |
| <li><p>[AIRFLOW-1787] Fix task instance batch clear and set state bugs</p></li> |
| <li><p>[AIRFLOW-1780] Fix long output lines with unicode from hanging parent</p></li> |
| <li><p>[AIRFLOW-387] Close SQLAlchemy sessions properly</p></li> |
| <li><p>[AIRFLOW-1779] Add keepalive packets to ssh hook</p></li> |
| <li><p>[AIRFLOW-1669] Fix Docker and pin Moto to 1.1.19</p></li> |
| <li><p>[AIRFLOW-71] Add support for private Docker images</p></li> |
| <li><p>[AIRFLOW-XXX] Give a clue what the ‘ds’ variable is</p></li> |
| <li><p>[AIRFLOW-XXX] Correct typos in the faq docs page</p></li> |
| <li><p>[AIRFLOW-1571] Add AWS Lambda Hook</p></li> |
| <li><p>[AIRFLOW-1675] Fix docstrings for API docs</p></li> |
| <li><p>[AIRFLOW-1712][AIRFLOW-756][AIRFLOW-751] Log SSHOperator output</p></li> |
| <li><p>[AIRFLOW-1776] Capture stdout and stderr for logging</p></li> |
| <li><p>[AIRFLOW-1765] Make experimental API securable without needing Kerberos.</p></li> |
| <li><p>[AIRFLOW-1764] The web interface should not use the experimental API</p></li> |
| <li><p>[AIRFLOW-1771] Rename heartbeat to avoid confusion</p></li> |
| <li><p>[AIRFLOW-1769] Add support for templates in VirtualenvOperator</p></li> |
| <li><p>[AIRFLOW-1763] Fix S3TaskHandler unit tests</p></li> |
| <li><p>[AIRFLOW-1315] Add Qubole File & Partition Sensors</p></li> |
| <li><p>[AIRFLOW-1018] Make processor use logging framework</p></li> |
| <li><p>[AIRFLOW-1695] Add RedshiftHook using boto3</p></li> |
| <li><p>[AIRFLOW-1706] Fix query error for MSSQL backend</p></li> |
| <li><p>[AIRFLOW-1711] Use ldap3 dict for group membership</p></li> |
| <li><p>[AIRFLOW-1723] Make sendgrid a plugin</p></li> |
| <li><p>[AIRFLOW-1757] Add missing options to SparkSubmitOperator</p></li> |
| <li><p>[AIRFLOW-1734][Airflow 1734] Sqoop hook/operator enhancements</p></li> |
| <li><p>[AIRFLOW-1761] Fix type in scheduler.rst</p></li> |
| <li><p>[AIRFLOW-1731] Set pythonpath for logging</p></li> |
| <li><p>[AIRFLOW-1641] Handle executor events in the scheduler</p></li> |
| <li><p>[AIRFLOW-1744] Make sure max_tries can be set</p></li> |
| <li><p>[AIRFLOW-1732] Improve dataflow hook logging</p></li> |
| <li><p>[AIRFLOW-1736] Add HotelQuickly to Who Uses Airflow</p></li> |
| <li><p>[AIRFLOW-1657] Handle failing qubole operator</p></li> |
| <li><p>[AIRFLOW-1677] Fix typo in example_qubole_operator</p></li> |
| <li><p>[AIRFLOW-926] Fix JDBC Hook</p></li> |
| <li><p>[AIRFLOW-1520] Boto3 S3Hook, S3Log</p></li> |
| <li><p>[AIRFLOW-1716] Fix multiple __init__ def in SimpleDag</p></li> |
| <li><p>[AIRFLOW-XXX] Fix DateTime in Tree View</p></li> |
| <li><p>[AIRFLOW-1719] Fix small typo</p></li> |
| <li><p>[AIRFLOW-1432] Charts label for Y axis not visible</p></li> |
| <li><p>[AIRFLOW-1743] Verify ldap filters correctly</p></li> |
| <li><p>[AIRFLOW-1745] Restore default signal disposition</p></li> |
| <li><p>[AIRFLOW-1741] Correctly hide second chart on task duration page</p></li> |
| <li><p>[AIRFLOW-1728] Add networkUri, subnet, tags to Dataproc operator</p></li> |
| <li><p>[AIRFLOW-1726] Add copy_expert psycopg2 method to PostgresHook</p></li> |
| <li><p>[AIRFLOW-1330] Add conn_type argument to CLI when adding connection</p></li> |
| <li><p>[AIRFLOW-1698] Remove SCHEDULER_RUNS env var in systemd</p></li> |
| <li><p>[AIRFLOW-1694] Stop using itertools.izip</p></li> |
| <li><p>[AIRFLOW-1692] Change test_views filename to support Windows</p></li> |
| <li><p>[AIRFLOW-1722] Fix typo in scheduler autorestart output filename</p></li> |
| <li><p>[AIRFLOW-1723] Support sendgrid in email backend</p></li> |
| <li><p>[AIRFLOW-1718] Set num_retries on Dataproc job request execution</p></li> |
| <li><p>[AIRFLOW-1727] Add unit tests for DataProcHook</p></li> |
| <li><p>[AIRFLOW-1631] Fix timing issue in unit test</p></li> |
| <li><p>[AIRFLOW-1631] Fix local executor unbound parallelism</p></li> |
| <li><p>[AIRFLOW-1724] Add Fundera to Who uses Airflow?</p></li> |
| <li><p>[AIRFLOW-1683] Cancel BigQuery job on timeout.</p></li> |
| <li><p>[AIRFLOW-1714] Fix misspelling: s/seperate/separate/</p></li> |
| <li><p>[AIRFLOW-1681] Add batch clear in task instance view</p></li> |
| <li><p>[AIRFLOW-1696] Fix dataproc version label error</p></li> |
| <li><p>[AIRFLOW-1613] Handle binary field in MySqlToGoogleCloudStorageOperator</p></li> |
| <li><p>[AIRFLOW-1697] Mode to disable charts endpoint</p></li> |
| <li><p>[AIRFLOW-1691] Add better Google cloud logging documentation</p></li> |
| <li><p>[AIRFLOW-1690] Add detail to gcs error messages</p></li> |
| <li><p>[AIRFLOW-1682] Make S3TaskHandler write to S3 on close</p></li> |
| <li><p>[AIRFLOW-1634] Adds task_concurrency feature</p></li> |
| <li><p>[AIRFLOW-1676] Make GCSTaskHandler write to GCS on close</p></li> |
| <li><p>[AIRFLOW-1678] Fix erroneously repeated word in function docstrings</p></li> |
| <li><p>[AIRFLOW-1323] Made Dataproc operator parameter names consistent</p></li> |
| <li><p>[AIRFLOW-1590] fix unused module and variable</p></li> |
| <li><p>[AIRFLOW-1671] Add @apply_defaults back to gcs download operator</p></li> |
| <li><p>[AIRFLOW-988] Fix repeating SLA miss callbacks</p></li> |
| <li><p>[AIRFLOW-1611] Customize logging</p></li> |
| <li><p>[AIRFLOW-1668] Expose keepalives_idle for Postgres connections</p></li> |
| <li><p>[AIRFLOW-1658] Kill Druid task on timeout</p></li> |
| <li><p>[AIRFLOW-1669][AIRFLOW-1368] Fix Docker import</p></li> |
| <li><p>[AIRFLOW-891] Make webserver clock include date</p></li> |
| <li><p>[AIRFLOW-1560] Add AWS DynamoDB hook and operator for inserting batch items</p></li> |
| <li><p>[AIRFLOW-1654] Show tooltips for link icons in DAGs view</p></li> |
| <li><p>[AIRFLOW-1660] Change webpage width to full-width</p></li> |
| <li><p>[AIRFLOW-1664] write file as binary instead of str</p></li> |
| <li><p>[AIRFLOW-1659] Fix invalid obj attribute bug in file_task_handler.py</p></li> |
| <li><p>[AIRFLOW-1635] Allow creating GCP connection without requiring a JSON file</p></li> |
| <li><p>[AIRFLOW-1650] Fix custom celery config loading</p></li> |
| <li><p>[AIRFLOW-1647] Fix Spark-sql hook</p></li> |
| <li><p>[AIRFLOW-1587] Fix CeleryExecutor import error</p></li> |
| <li><p>[Airflow-1640][AIRFLOW-1640] Add qubole default connection</p></li> |
| <li><p>[AIRFLOW-1576] Added region param to Dataproc{*}Operators</p></li> |
| <li><p>[AIRFLOW-1643] Add healthjump to officially using list</p></li> |
| <li><p>[AIRFLOW-1626] Add Azri Solutions to Airflow users</p></li> |
| <li><p>[AIRFLOW-1636] Add AWS and EMR connection type</p></li> |
| <li><p>[AIRFLOW-1527] Refactor celery config</p></li> |
| <li><p>[AIRFLOW-1639] Fix Fernet error handling</p></li> |
| <li><p>[AIRFLOW-1637] Fix Travis CI build status link</p></li> |
| <li><p>[AIRFLOW-1628] Fix docstring of sqlsensor</p></li> |
| <li><p>[AIRFLOW-1331] add SparkSubmitOperator option</p></li> |
| <li><p>[AIRFLOW-1627] Only query pool in SubDAG init when necessary</p></li> |
| <li><p>[AIRFLOW-1629] Make extra a textarea in edit connections form</p></li> |
| <li><p>[AIRFLOW-1368] Automatically remove Docker container on exit</p></li> |
| <li><p>[AIRFLOW-289] Make airflow timezone independent</p></li> |
| <li><p>[AIRFLOW-1356] Add <cite>–celery_hostname</cite> to <cite>airflow worker</cite></p></li> |
| <li><p>[AIRFLOW-1247] Fix ignore_all_dependencies argument ignored</p></li> |
| <li><p>[AIRFLOW-1621] Add tests for server side paging</p></li> |
| <li><p>[AIRFLOW-1591] Avoid attribute error when rendering logging filename</p></li> |
| <li><p>[AIRFLOW-1031] Replace hard-code to DagRun.ID_PREFIX</p></li> |
| <li><p>[AIRFLOW-1604] Rename logger to log</p></li> |
| <li><p>[AIRFLOW-1512] Add PythonVirtualenvOperator</p></li> |
| <li><p>[AIRFLOW-1617] Fix XSS vulnerability in Variable endpoint</p></li> |
| <li><p>[AIRFLOW-1497] Reset hidden fields when changing connection type</p></li> |
| <li><p>[AIRFLOW-1619] Add poll_sleep parameter to GCP dataflow operator</p></li> |
| <li><p>[AIRFLOW-XXX] Remove landscape.io config</p></li> |
| <li><p>[AIRFLOW-XXX] Remove non working service badges</p></li> |
| <li><p>[AIRFLOW-1177] Fix Variable.setdefault w/existing JSON</p></li> |
| <li><p>[AIRFLOW-1600] Fix exception handling in get_fernet</p></li> |
| <li><p>[AIRFLOW-1614] Replace inspect.stack() with sys._getframe()</p></li> |
| <li><p>[AIRFLOW-1519] Add server side paging in DAGs list</p></li> |
| <li><p>[AIRFLOW-1309] Allow hive_to_druid to take tblproperties</p></li> |
| <li><p>[AIRFLOW-1613] Make MySqlToGoogleCloudStorageOperator compaitible with python3</p></li> |
| <li><p>[AIRFLOW-1603] add PAYMILL to companies list</p></li> |
| <li><p>[AIRFLOW-1609] Fix gitignore to ignore all venvs</p></li> |
| <li><p>[AIRFLOW-1601] Add configurable task cleanup time</p></li> |
| </ul> |
| </div> |
| <div class="section" id="airflow-1-9-0-2018-01-02"> |
| <h2>Airflow 1.9.0, 2018-01-02<a class="headerlink" href="#airflow-1-9-0-2018-01-02" title="Permalink to this headline">¶</a></h2> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-1525] Fix minor LICENSE and NOTICE issues</p></li> |
| <li><p>[AIRFLOW-XXX] Bump version to 1.9.0</p></li> |
| <li><p>[AIRFLOW-1897][AIRFLOW-1873] Task Logs for running instance not visible in WebUI</p></li> |
| <li><p>[AIRFLOW-XXX] Make sure session is committed</p></li> |
| <li><p>[AIRFLOW-1896] FIX bleach <> html5lib incompatibility</p></li> |
| <li><p>[AIRFLOW-XXX] Fix log handler test</p></li> |
| <li><p>[AIRFLOW-1873] Set TI.try_number to right value depending TI state</p></li> |
| <li><p>[AIRFLOW-1554] Fix wrong DagFileProcessor termination method call</p></li> |
| <li><p>[AIRFLOW-1872] Set context for all handlers including parents</p></li> |
| <li><p>[AIRFLOW-XXX] Add dask lock files to excludes</p></li> |
| <li><p>[AIRFLOW-1839] Fix more bugs in S3Hook boto -> boto3 migration</p></li> |
| <li><p>[AIRFLOW-1795] Correctly call S3Hook after migration to boto3</p></li> |
| <li><p>[AIRFLOW-1813] Bug SSH Operator empty buffer</p></li> |
| <li><p>[AIRFLOW-1794] Remove uses of Exception.message for Python 3</p></li> |
| <li><p>[AIRFLOW-1799] Fix logging line which raises errors</p></li> |
| <li><p>[AIRFLOW-1102] Upgrade Gunicorn >=19.4.0</p></li> |
| <li><p>[AIRFLOW-1756] Fix S3TaskHandler to work with Boto3-based S3Hook</p></li> |
| <li><p>[AIRFLOW-1797] S3Hook.load_string didn’t work on Python3</p></li> |
| <li><p>[AIRFLOW-1792] Missing intervals DruidOperator</p></li> |
| <li><p>[AIRFLOW-1789][AIRFLOW-1712] Log SSHOperator stderr to log.warning</p></li> |
| <li><p>[AIRFLOW-1669] Fix Docker and pin Moto to 1.1.19</p></li> |
| <li><p>[AIRFLOW-71] Add support for private Docker images</p></li> |
| <li><p>[AIRFLOW-1779] Add keepalive packets to ssh hook</p></li> |
| <li><p>[AIRFLOW-XXX] Give a clue what the ‘ds’ variable is</p></li> |
| <li><p>[AIRFLOW-XXX] Correct typos in the faq docs page</p></li> |
| <li><p>[AIRFLOW-1571] Add AWS Lambda Hook</p></li> |
| <li><p>[AIRFLOW-1675] Fix docstrings for API docs</p></li> |
| <li><p>[AIRFLOW-1712][AIRFLOW-756][AIRFLOW-751] Log SSHOperator output</p></li> |
| <li><p>[AIRFLOW-1776] Capture stdout and stderr for logging</p></li> |
| <li><p>[AIRFLOW-1765] Make experimental API securable without needing Kerberos.</p></li> |
| <li><p>[AIRFLOW-1764] The web interface should not use the experimental API</p></li> |
| <li><p>[AIRFLOW-1634] Adds task_concurrency feature</p></li> |
| <li><p>[AIRFLOW-1018] Make processor use logging framework</p></li> |
| <li><p>[AIRFLOW-1695] Add RedshiftHook using boto3</p></li> |
| <li><p>[AIRFLOW-1706] Fix query error for MSSQL backend</p></li> |
| <li><p>[AIRFLOW-1711] Use ldap3 dict for group membership</p></li> |
| <li><p>[AIRFLOW-1757] Add missing options to SparkSubmitOperator</p></li> |
| <li><p>[AIRFLOW-1734][Airflow 1734] Sqoop hook/operator enhancements</p></li> |
| <li><p>[AIRFLOW-1731] Set pythonpath for logging</p></li> |
| <li><p>[AIRFLOW-1641] Handle executor events in the scheduler</p></li> |
| <li><p>[AIRFLOW-1744] Make sure max_tries can be set</p></li> |
| <li><p>[AIRFLOW-1330] Add conn_type argument to CLI when adding connection</p></li> |
| <li><p>[AIRFLOW-926] Fix JDBC Hook</p></li> |
| <li><p>[AIRFLOW-1520] Boto3 S3Hook, S3Log</p></li> |
| <li><p>[AIRFLOW-XXX] Fix DateTime in Tree View</p></li> |
| <li><p>[AIRFLOW-1432] Charts label for Y axis not visible</p></li> |
| <li><p>[AIRFLOW-1743] Verify ldap filters correctly</p></li> |
| <li><p>[AIRFLOW-1745] Restore default signal disposition</p></li> |
| <li><p>[AIRFLOW-1741] Correctly hide second chart on task duration page</p></li> |
| <li><p>[AIRFLOW-1726] Add copy_expert psycopg2 method to PostgresHook</p></li> |
| <li><p>[AIRFLOW-1698] Remove SCHEDULER_RUNS env var in systemd</p></li> |
| <li><p>[AIRFLOW-1694] Stop using itertools.izip</p></li> |
| <li><p>[AIRFLOW-1692] Change test_views filename to support Windows</p></li> |
| <li><p>[AIRFLOW-1722] Fix typo in scheduler autorestart output filename</p></li> |
| <li><p>[AIRFLOW-1691] Add better Google cloud logging documentation</p></li> |
| <li><p>[AIRFLOW-1690] Add detail to gcs error messages</p></li> |
| <li><p>[AIRFLOW-1682] Make S3TaskHandler write to S3 on close</p></li> |
| <li><p>[AIRFLOW-1676] Make GCSTaskHandler write to GCS on close</p></li> |
| <li><p>[AIRFLOW-1635] Allow creating GCP connection without requiring a JSON file</p></li> |
| <li><p>[AIRFLOW-1323] Made Dataproc operator parameter names consistent</p></li> |
| <li><p>[AIRFLOW-1590] fix unused module and variable</p></li> |
| <li><p>[AIRFLOW-988] Fix repeating SLA miss callbacks</p></li> |
| <li><p>[AIRFLOW-1611] Customize logging</p></li> |
| <li><p>[AIRFLOW-1668] Expose keepalives_idle for Postgres connections</p></li> |
| <li><p>[AIRFLOW-1658] Kill Druid task on timeout</p></li> |
| <li><p>[AIRFLOW-1669][AIRFLOW-1368] Fix Docker import</p></li> |
| <li><p>[AIRFLOW-1560] Add AWS DynamoDB hook and operator for inserting batch items</p></li> |
| <li><p>[AIRFLOW-1654] Show tooltips for link icons in DAGs view</p></li> |
| <li><p>[AIRFLOW-1660] Change webpage width to full-width</p></li> |
| <li><p>[AIRFLOW-1664] write file as binary instead of str</p></li> |
| <li><p>[AIRFLOW-1659] Fix invalid obj attribute bug in file_task_handler.py</p></li> |
| <li><p>[AIRFLOW-1650] Fix custom celery config loading</p></li> |
| <li><p>[AIRFLOW-1647] Fix Spark-sql hook</p></li> |
| <li><p>[AIRFLOW-1587] Fix CeleryExecutor import error</p></li> |
| <li><p>[AIRFLOW-1636] Add AWS and EMR connection type</p></li> |
| <li><p>[AIRFLOW-1527] Refactor celery config</p></li> |
| <li><p>[AIRFLOW-1639] Fix Fernet error handling</p></li> |
| <li><p>[AIRFLOW-1628] Fix docstring of sqlsensor</p></li> |
| <li><p>[AIRFLOW-1331] add SparkSubmitOperator option</p></li> |
| <li><p>[AIRFLOW-1627] Only query pool in SubDAG init when necessary</p></li> |
| <li><p>[AIRFLOW-1629] Make extra a textarea in edit connections form</p></li> |
| <li><p>[AIRFLOW-1621] Add tests for server side paging</p></li> |
| <li><p>[AIRFLOW-1519] Add server side paging in DAGs list</p></li> |
| <li><p>[AIRFLOW-289] Make airflow timezone independent</p></li> |
| <li><p>[AIRFLOW-1356] Add <cite>–celery_hostname</cite> to <cite>airflow worker</cite></p></li> |
| <li><p>[AIRFLOW-1591] Avoid attribute error when rendering logging filename</p></li> |
| <li><p>[AIRFLOW-1031] Replace hard-code to DagRun.ID_PREFIX</p></li> |
| <li><p>[AIRFLOW-1604] Rename logger to log</p></li> |
| <li><p>[AIRFLOW-1512] Add PythonVirtualenvOperator</p></li> |
| <li><p>[AIRFLOW-1617] Fix XSS vulnerability in Variable endpoint</p></li> |
| <li><p>[AIRFLOW-1497] Reset hidden fields when changing connection type</p></li> |
| <li><p>[AIRFLOW-1177] Fix Variable.setdefault w/existing JSON</p></li> |
| <li><p>[AIRFLOW-1600] Fix exception handling in get_fernet</p></li> |
| <li><p>[AIRFLOW-1614] Replace inspect.stack() with sys._getframe()</p></li> |
| <li><p>[AIRFLOW-1613] Make MySqlToGoogleCloudStorageOperator compaitible with python3</p></li> |
| <li><p>[AIRFLOW-1609] Fix gitignore to ignore all venvs</p></li> |
| <li><p>[AIRFLOW-1601] Add configurable task cleanup time</p></li> |
| <li><p>[AIRFLOW-XXX] Bumping Airflow 1.9.0alpha0 version</p></li> |
| <li><p>[AIRFLOW-1608] Handle pending job state in GCP Dataflow hook</p></li> |
| <li><p>[AIRFLOW-1606] Use non static DAG.sync_to_db</p></li> |
| <li><p>[AIRFLOW-1606][Airflow-1606][AIRFLOW-1605][AIRFLOW-160] DAG.sync_to_db is now a normal method</p></li> |
| <li><p>[AIRFLOW-1602] LoggingMixin in DAG class</p></li> |
| <li><p>[AIRFLOW-1593] expose load_string in WasbHook</p></li> |
| <li><p>[AIRFLOW-1597] Add GameWisp as Airflow user</p></li> |
| <li><p>[AIRFLOW-1594] Don’t install test packages into python root.[]</p></li> |
| <li><p>[AIRFLOW-1582] Improve logging within Airflow</p></li> |
| <li><p>[AIRFLOW-1476] add INSTALL instruction for source releases</p></li> |
| <li><p>[AIRFLOW-XXX] Save username and password in airflow-pr</p></li> |
| <li><p>[AIRFLOW-1522] Increase text size for var field in variables for MySQL</p></li> |
| <li><p>[AIRFLOW-950] Missing AWS integrations on documentation::integrations</p></li> |
| <li><p>[AIRFLOW-XXX] 1.8.2 release notes</p></li> |
| <li><p>[AIRFLOW-1573] Remove <cite>thrift < 0.10.0</cite> requirement</p></li> |
| <li><p>[AIRFLOW-1584] Remove insecure /headers endpoint</p></li> |
| <li><p>[AIRFLOW-1586] Add mapping for date type to mysql_to_gcs operator</p></li> |
| <li><p>[AIRFLOW-1579] Adds support for jagged rows in Bigquery hook for BQ load jobs</p></li> |
| <li><p>[AIRFLOW-1577] Add token support to DatabricksHook</p></li> |
| <li><p>[AIRFLOW-1580] Error in string formating</p></li> |
| <li><p>[AIRFLOW-1567] Updated docs for Google ML Engine operators/hooks</p></li> |
| <li><p>[AIRFLOW-1574] add ‘to’ attribute to templated vars of email operator</p></li> |
| <li><p>[AIRFLOW-1572] add carbonite to company list</p></li> |
| <li><p>[AIRFLOW-1568] Fix typo in BigQueryHook</p></li> |
| <li><p>[AIRFLOW-1493][AIRFLOW-XXXX][WIP] fixed dumb thing</p></li> |
| <li><p>[AIRFLOW-1567][Airflow-1567] Renamed cloudml hook and operator to mlengine</p></li> |
| <li><p>[AIRFLOW-1568] Add datastore export/import operators</p></li> |
| <li><p>[AIRFLOW-1564] Use Jinja2 to render logging filename</p></li> |
| <li><p>[AIRFLOW-1562] Spark-sql logging contains deadlock</p></li> |
| <li><p>[AIRFLOW-1556][Airflow 1556] Add support for SQL parameters in BigQueryBaseCursor</p></li> |
| <li><p>[AIRFLOW-108] Add CreditCards.com to companies list</p></li> |
| <li><p>[AIRFLOW-1541] Add channel to template fields of slack_operator</p></li> |
| <li><p>[AIRFLOW-1535] Add service account/scopes in dataproc</p></li> |
| <li><p>[AIRFLOW-1384] Add to README.md CaDC/ARGO</p></li> |
| <li><p>[AIRFLOW-1546] add Zymergen 80to org list in README</p></li> |
| <li><p>[AIRFLOW-1545] Add Nextdoor to companies list</p></li> |
| <li><p>[AIRFLOW-1544] Add DataFox to companies list</p></li> |
| <li><p>[AIRFLOW-1529] Add logic supporting quoted newlines in Google BigQuery load jobs</p></li> |
| <li><p>[AIRFLOW-1521] Fix emplate rendering for BigqueryTableDeleteOperator</p></li> |
| <li><p>[AIRFLOW-1324] Generalize Druid operator and hook</p></li> |
| <li><p>[AIRFLOW-1516] Fix error handling getting fernet</p></li> |
| <li><p>[AIRFLOW-1420][AIRFLOW-1473] Fix deadlock check</p></li> |
| <li><p>[AIRFLOW-1495] Fix migration on index on job_id</p></li> |
| <li><p>[AIRFLOW-1483] Making page size consistent in list</p></li> |
| <li><p>[AIRFLOW-1495] Add TaskInstance index on job_id</p></li> |
| <li><p>[AIRFLOW-855] Replace PickleType with LargeBinary in XCom</p></li> |
| <li><p>[AIRFLOW-1505] Document when Jinja substitution occurs</p></li> |
| <li><p>[AIRFLOW-1504] Log dataproc cluster name</p></li> |
| <li><p>[AIRFLOW-1239] Fix unicode error for logs in base_task_runner</p></li> |
| <li><p>[AIRFLOW-1280] Fix Gantt chart height</p></li> |
| <li><p>[AIRFLOW-1507] Template parameters in file_to_gcs operator</p></li> |
| <li><p>[AIRFLOW-1452] workaround lock on method</p></li> |
| <li><p>[AIRFLOW-1385] Make Airflow task logging configurable</p></li> |
| <li><p>[AIRFLOW-940] Handle error on variable decrypt</p></li> |
| <li><p>[AIRFLOW-1492] Add gauge for task successes/failures</p></li> |
| <li><p>[AIRFLOW-1443] Update Airflow configuration documentation</p></li> |
| <li><p>[AIRFLOW-1486] Unexpected S3 writing log error</p></li> |
| <li><p>[AIRFLOW-1487] Added links to all companies officially using Airflow</p></li> |
| <li><p>[AIRFLOW-1489] Fix typo in BigQueryCheckOperator</p></li> |
| <li><p>[AIRFLOW-1349] Fix backfill to respect limits</p></li> |
| <li><p>[AIRFLOW-1478] Chart owner column should be sortable</p></li> |
| <li><p>[AIRFLOW-1397][AIRFLOW-1] No Last Run column data displyed in Airflow UI 1.8.1</p></li> |
| <li><p>[AIRFLOW-1474] Add dag_id regex feature for <cite>airflow clear</cite> command</p></li> |
| <li><p>[AIRFLOW-1445] Changing HivePartitionSensor UI color to lighter shade</p></li> |
| <li><p>[AIRFLOW-1359] Use default_args in Cloud ML eval</p></li> |
| <li><p>[AIRFLOW-1389] Support createDisposition in BigQueryOperator</p></li> |
| <li><p>[AIRFLOW-1349] Refactor BackfillJob _execute</p></li> |
| <li><p>[AIRFLOW-1459] Fixed broken integration .rst formatting</p></li> |
| <li><p>[AIRFLOW-1448] Revert “Fix cli reading logfile in memory”</p></li> |
| <li><p>[AIRFLOW-1398] Allow ExternalTaskSensor to wait on multiple runs of a task</p></li> |
| <li><p>[AIRFLOW-1399] Fix cli reading logfile in memory</p></li> |
| <li><p>[AIRFLOW-1442] Remove extra space from ignore_all_deps generated command</p></li> |
| <li><p>[AIRFLOW-1438] Change batch size per query in scheduler</p></li> |
| <li><p>[AIRFLOW-1439] Add max billing tier for the BQ Hook and Operator</p></li> |
| <li><p>[AIRFLOW-1437] Modify BigQueryTableDeleteOperator</p></li> |
| <li><p>[Airflow 1332] Split logs based on try number</p></li> |
| <li><p>[AIRFLOW-1385] Create abstraction for Airflow task logging</p></li> |
| <li><p>[AIRFLOW-756][AIRFLOW-751] Replace ssh hook, operator & sftp operator with paramiko based</p></li> |
| <li><p>[AIRFLOW-1393][[AIRFLOW-1393] Enable Py3 tests in contrib/spark_submit_hook[</p></li> |
| <li><p>[AIRFLOW-1345] Dont expire TIs on each scheduler loop</p></li> |
| <li><p>[AIRFLOW-1059] Reset orphaned tasks in batch for scheduler</p></li> |
| <li><p>[AIRFLOW-1255] Fix SparkSubmitHook output deadlock</p></li> |
| <li><p>[AIRFLOW-1359] Add Google CloudML utils for model evaluation</p></li> |
| <li><p>[AIRFLOW-1247] Fix ignore all dependencies argument ignored</p></li> |
| <li><p>[AIRFLOW-1401] Standardize cloud ml operator arguments</p></li> |
| <li><p>[AIRFLOW-1394] Add quote_character param to GCS hook and operator</p></li> |
| <li><p>[AIRFLOW-1402] Cleanup SafeConfigParser DeprecationWarning</p></li> |
| <li><p>[AIRFLOW-1326][[AIRFLOW-1326][AIRFLOW-1184] Don’t split argument array – it’s already an array.[</p></li> |
| <li><p>[AIRFLOW-1384] Add ARGO/CaDC as a Airflow user</p></li> |
| <li><p>[AIRFLOW-1357] Fix scheduler zip file support</p></li> |
| <li><p>[AIRFLOW-1382] Add working dir option to DockerOperator</p></li> |
| <li><p>[AIRFLOW-1388] Add Cloud ML Engine operators to integration doc</p></li> |
| <li><p>[AIRFLOW-1387] Add unicode string prefix</p></li> |
| <li><p>[AIRFLOW-1366] Add max_tries to task instance</p></li> |
| <li><p>[AIRFLOW-1300] Enable table creation with TBLPROPERTIES</p></li> |
| <li><p>[AIRFLOW-1271] Add Google CloudML Training Operator</p></li> |
| <li><p>[AIRFLOW-300] Add Google Pubsub hook and operator</p></li> |
| <li><p>[AIRFLOW-1343] Fix dataproc label format</p></li> |
| <li><p>[AIRFLOW-1367] Pass Content-ID To reference inline images in an email, we need to be able to add <img src=”cid:{}”/> to the HTML. However currently the Content-ID (cid) is not passed, so we need to add it</p></li> |
| <li><p>[AIRFLOW-1265] Fix celery executor parsing CELERY_SSL_ACTIVE</p></li> |
| <li><p>[AIRFLOW-1272] Google Cloud ML Batch Prediction Operator</p></li> |
| <li><p>[AIRFLOW-1352][AIRFLOW-1335] Revert MemoryHandler change ()[]</p></li> |
| <li><p>[AIRFLOW-1350] Add query_uri param to Hive/SparkSQL DataProc operator</p></li> |
| <li><p>[AIRFLOW-1334] Check if tasks are backfill on scheduler in a join</p></li> |
| <li><p>[AIRFLOW-1343] Add Airflow default label to the dataproc operator</p></li> |
| <li><p>[AIRFLOW-1273] Add Google Cloud ML version and model operators</p></li> |
| <li><p>[AIRFLOW-1273]AIRFLOW-1273] Add Google Cloud ML version and model operators</p></li> |
| <li><p>[AIRFLOW-1321] Fix hidden field key to ignore case</p></li> |
| <li><p>[AIRFLOW-1337] Make log_format key names lowercase</p></li> |
| <li><p>[AIRFLOW-1338][AIRFLOW-782] Add GCP dataflow hook runner change to UPDATING.md</p></li> |
| <li><p>[AIRFLOW-801] Remove outdated docstring on BaseOperator</p></li> |
| <li><p>[AIRFLOW-1344] Fix text encoding bug when reading logs for Python 3.5</p></li> |
| <li><p>[AIRFLOW-1338] Fix incompatible GCP dataflow hook</p></li> |
| <li><p>[AIRFLOW-1333] Enable copy function for Google Cloud Storage Hook</p></li> |
| <li><p>[AIRFLOW-1337] Allow log format customization via airflow.cfg</p></li> |
| <li><p>[AIRFLOW-1320] Update LetsBonus users in README</p></li> |
| <li><p>[AIRFLOW-1335] Use MemoryHandler for buffered logging</p></li> |
| <li><p>[AIRFLOW-1339] Add Drivy to the list of users</p></li> |
| <li><p>[AIRFLOW-1275] Put ‘airflow pool’ into API</p></li> |
| <li><p>[AIRFLOW-1296] Propagate SKIPPED to all downstream tasks</p></li> |
| <li><p>[AIRFLOW-1317] Fix minor issues in API reference</p></li> |
| <li><p>[AIRFLOW-1308] Disable nanny usage for Dask</p></li> |
| <li><p>[AIRFLOW-1172] Support nth weekday of the month cron expression</p></li> |
| <li><p>[AIRFLOW-936] Add clear/mark success for DAG in the UI</p></li> |
| <li><p>[AIRFLOW-1294] Backfills can loose tasks to execute</p></li> |
| <li><p>[AIRFLOW-1299] Support imageVersion in Google Dataproc cluster</p></li> |
| <li><p>[AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF requirements</p></li> |
| <li><p>[AIRFLOW-1301] Add New Relic to list of companies</p></li> |
| <li><p>[AIRFLOW-1289] Removes restriction on number of scheduler threads</p></li> |
| <li><p>[AIRFLOW-1024] Ignore celery executor errors (#49)</p></li> |
| <li><p>[AIRFLOW-1265] Fix exception while loading celery configurations</p></li> |
| <li><p>[AIRFLOW-1290] set docs author to ‘Apache Airflow’</p></li> |
| <li><p>[AIRFLOW-1242] Allowing project_id to have a colon in it.</p></li> |
| <li><p>[AIRFLOW-1282] Fix known event column sorting</p></li> |
| <li><p>[AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun</p></li> |
| <li><p>[AIRFLOW-1208] Speed-up cli tests</p></li> |
| <li><p>[AIRFLOW-1192] Some enhancements to qubole_operator</p></li> |
| <li><p>[AIRFLOW-1281] Sort variables by key field by default</p></li> |
| <li><p>[AIRFLOW-1277] Forbid KE creation with empty fields</p></li> |
| <li><p>[AIRFLOW-1276] Forbid event creation with end_data earlier than start_date</p></li> |
| <li><p>[AIRFLOW-1263] Dynamic height for charts</p></li> |
| <li><p>[AIRFLOW-1266] Increase width of gantt y axis</p></li> |
| <li><p>[AIRFLOW-1244] Forbid creation of a pool with empty name</p></li> |
| <li><p>[AIRFLOW-1274][HTTPSENSOR] Rename parameter params to data</p></li> |
| <li><p>[AIRFLOW-654] Add SSL Config Option for CeleryExecutor w/ RabbitMQ - Add BROKER_USE_SSL config to give option to send AMQP messages over SSL - Can be set using usual airflow options (e.g. airflow.cfg, env vars, etc.)</p></li> |
| <li><p>[AIRFLOW-1256] Add United Airlines to readme</p></li> |
| <li><p>[AIRFLOW-1251] Add eRevalue to Airflow users</p></li> |
| <li><p>[AIRFLOW-908] Print hostname at the start of cli run</p></li> |
| <li><p>[AIRFLOW-1237] Fix IN-predicate sqlalchemy warning</p></li> |
| <li><p>[AIRFLOW-1243] DAGs table has no default entries to show</p></li> |
| <li><p>[AIRFLOW-1245] Fix random failure in test_trigger_dag_for_date</p></li> |
| <li><p>[AIRFLOW-1248] Fix wrong conf name for worker timeout</p></li> |
| <li><p>[AIRFLOW-1197] : SparkSubmitHook on_kill error</p></li> |
| <li><p>[AIRFLOW-1191] : SparkSubmitHook custom cmd</p></li> |
| <li><p>[AIRFLOW-1234] Cover utils.operator_helpers with UTs</p></li> |
| <li><p>[AIRFLOW-1217] Enable Sqoop logging</p></li> |
| <li><p>[AIRFLOW-645] Support HTTPS connections in HttpHook</p></li> |
| <li><p>[AIRFLOW-1231] Use flask_wtf.CSRFProtect</p></li> |
| <li><p>[AIRFLOW-1232] Remove deprecated readfp warning</p></li> |
| <li><p>[AIRFLOW-1233] Cover utils.json with unit tests</p></li> |
| <li><p>[AIRFLOW-1227] Remove empty column on the Logs view</p></li> |
| <li><p>[AIRFLOW-1226] Remove empty column on the Jobs view</p></li> |
| <li><p>[AIRFLOW-1221] Fix templating bug with DatabricksSubmitRunOperator</p></li> |
| <li><p>[AIRFLOW-1210] Enable DbApiHook unit tests</p></li> |
| <li><p>[AIRFLOW-1199] Fix create modal</p></li> |
| <li><p>[AIRFLOW-1200] Forbid creation of a variable with an empty key</p></li> |
| <li><p>[AIRFLOW-1207] Enable utils.helpers unit tests</p></li> |
| <li><p>[AIRFLOW-1213] Add hcatalog parameters to sqoop</p></li> |
| <li><p>[AIRFLOW-1201] Update deprecated ‘nose-parameterized’</p></li> |
| <li><p>[AIRFLOW-1186] Sort dag.get_task_instances by execution_date</p></li> |
| <li><p>[AIRFLOW-1203] Pin Google API client version to fix OAuth issue</p></li> |
| <li><p>[AIRFLOW-1145] Fix closest_date_partition function with before set to True If we’re looking for the closest date before, we should take the latest date in the list of date before.</p></li> |
| <li><p>[AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection</p></li> |
| <li><p>[AIRFLOW-993] Update date inference logic</p></li> |
| <li><p>[AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately</p></li> |
| <li><p>[AIRFLOW-1041] Do not shadow xcom_push method[]</p></li> |
| <li><p>[AIRFLOW-860][AIRFLOW-935] Fix plugin executor import cycle and executor selection</p></li> |
| <li><p>[AIRFLOW-1189] Fix get a DataFrame using BigQueryHook failing</p></li> |
| <li><p>[AIRFLOW-1184] SparkSubmitHook does not split args</p></li> |
| <li><p>[AIRFLOW-1182] SparkSubmitOperator template field</p></li> |
| <li><p>[AIRFLOW-823] Allow specifying execution date in task_info API</p></li> |
| <li><p>[AIRFLOW-1175] Add Pronto Tools to Airflow user list</p></li> |
| <li><p>[AIRFLOW-1150] Fix scripts execution in sparksql hook[]</p></li> |
| <li><p>[AIRFLOW-1141] remove crawl_for_tasks</p></li> |
| <li><p>[AIRFLOW-1193] Add Checkr to company using Airflow</p></li> |
| <li><p>[AIRFLOW-1168] Add closing() to all connections and cursors</p></li> |
| <li><p>[AIRFLOW-1188] Add max_bad_records param to GoogleCloudStorageToBigQueryOperator</p></li> |
| <li><p>[AIRFLOW-1187][AIRFLOW-1185] Fix PyPi package names in documents</p></li> |
| <li><p>[AIRFLOW-1185] Fix PyPi URL in templates</p></li> |
| <li><p>[AIRFLOW-XXX] Updating CHANGELOG, README, and UPDATING after 1.8.1 release</p></li> |
| <li><p>[AIRFLOW-1181] Add delete and list functionality to gcs_hook</p></li> |
| <li><p>[AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change</p></li> |
| <li><p>[AIRFLOW-1167] Support microseconds in FTPHook modification time</p></li> |
| <li><p>[AIRFLOW-1173] Add Robinhood to who uses Airflow</p></li> |
| <li><p>[AIRFLOW-945][AIRFLOW-941] Remove psycopg2 connection workaround</p></li> |
| <li><p>[AIRFLOW-1140] DatabricksSubmitRunOperator should template the “json” field.</p></li> |
| <li><p>[AIRFLOW-1160] Update Spark parameters for Mesos</p></li> |
| <li><p>[AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2 templates</p></li> |
| <li><p>[AIRFLOW-1036] Randomize exponential backoff</p></li> |
| <li><p>[AIRFLOW-1155] Add Tails.com to community</p></li> |
| <li><p>[AIRFLOW-1142] Do not reset orphaned state for backfills</p></li> |
| <li><p>[AIRFLOW-492] Make sure stat updates cannot fail a task</p></li> |
| <li><p>[AIRFLOW-1119] Fix unload query so headers are on first row[]</p></li> |
| <li><p>[AIRFLOW-1089] Add Spark application arguments</p></li> |
| <li><p>[AIRFLOW-1125] Document encrypted connections</p></li> |
| <li><p>[AIRFLOW-1122] Increase stroke width in UI</p></li> |
| <li><p>[AIRFLOW-1138] Add missing licenses to files in scripts directory</p></li> |
| <li><p>(AIRFLOW-11-38) [AIRFLOW-1136] Capture invalid arguments for Sqoop</p></li> |
| <li><p>[AIRFLOW-1127] Move license notices to LICENSE</p></li> |
| <li><p>[AIRFLOW-1118] Add evo.company to Airflow users</p></li> |
| <li><p>[AIRFLOW-1121][AIRFLOW-1004] Fix <cite>airflow webserver –pid</cite> to write out pid file</p></li> |
| <li><p>[AIRFLOW-1124] Do not set all tasks to scheduled in backfill</p></li> |
| <li><p>[AIRFLOW-1120] Update version view to include Apache prefix</p></li> |
| <li><p>[AIRFLOW-1091] Add script that can compare jira target against merges</p></li> |
| <li><p>[AIRFLOW-1107] Add support for ftps non-default port</p></li> |
| <li><p>[AIRFLOW-1000] Rebrand distribution to Apache Airflow</p></li> |
| <li><p>[AIRFLOW-1094] Run unit tests under contrib in Travis</p></li> |
| <li><p>[AIRFLOW-1112] Log which pool when pool is full in scheduler</p></li> |
| <li><p>[AIRFLOW-1106] Add Groupalia/Letsbonus to the ReadMe</p></li> |
| <li><p>[AIRFLOW-1109] Use kill signal to kill processes and log results</p></li> |
| <li><p>[AIRFLOW-1074] Don’t count queued tasks for concurrency limits</p></li> |
| <li><p>[AIRFLOW-1095] Make ldap_auth memberOf come from configuration</p></li> |
| <li><p>[AIRFLOW-1090] Add HBO</p></li> |
| <li><p>[AIRFLOW-1035] Use binary exponential backoff</p></li> |
| <li><p>[AIRFLOW-1081] Improve performance of duration chart</p></li> |
| <li><p>[AIRFLOW-1078] Fix latest_runs endpoint for old flask versions</p></li> |
| <li><p>[AIRFLOW-1085] Enhance the SparkSubmitOperator</p></li> |
| <li><p>[AIRFLOW-1050] Do not count up_for_retry as not ready</p></li> |
| <li><p>[AIRFLOW-1028] Databricks Operator for Airflow</p></li> |
| <li><p>[AIRFLOW-1075] Security docs cleanup</p></li> |
| <li><p>[AIRFLOW-1033][AIFRLOW-1033] Fix ti_deps for no schedule dags</p></li> |
| <li><p>[AIRFLOW-1016] Allow HTTP HEAD request method on HTTPSensor</p></li> |
| <li><p>[AIRFLOW-970] Load latest_runs on homepage async</p></li> |
| <li><p>[AIRFLOW-111] Include queued tasks in scheduler concurrency check</p></li> |
| <li><p>[AIRFLOW-1001] Fix landing times if there is no following schedule</p></li> |
| <li><p>[AIRFLOW-1065] Add functionality for Azure Blob Storage over wasb://</p></li> |
| <li><p>[AIRFLOW-947] Improve exceptions for unavailable Presto cluster</p></li> |
| <li><p>[AIRFLOW-1067] use example.com in examples</p></li> |
| <li><p>[AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView</p></li> |
| <li><p>[AIRFLOW-1030][AIRFLOW-1] Fix hook import for HttpSensor</p></li> |
| <li><p>[AIRFLOW-1051] Add a test for resetdb to CliTests</p></li> |
| <li><p>[AIRFLOW-1004][AIRFLOW-276] Fix <cite>airflow webserver -D</cite> to run in background</p></li> |
| <li><p>[AIRFLOW-1062] Fix DagRun#find to return correct result</p></li> |
| <li><p>[AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs</p></li> |
| <li><p>[AIRFLOW-1038] Specify celery serialization options explicitly</p></li> |
| <li><p>[AIRFLOW-1054] Fix broken import in test_dag</p></li> |
| <li><p>[AIRFLOW-1007] Use Jinja sandbox for chart_data endpoint</p></li> |
| <li><p>[AIRFLOW-719] Fix race condition in ShortCircuit, Branch and LatestOnly</p></li> |
| <li><p>[AIRFLOW-1043] Fix doc strings of operators</p></li> |
| <li><p>[AIRFLOW-840] Make ticket renewer python3 compatible</p></li> |
| <li><p>[AIRFLOW-985] Extend the sqoop operator and hook</p></li> |
| <li><p>[AIRFLOW-1034] Make it possible to connect to S3 in sigv4 regions</p></li> |
| <li><p>[AIRFLOW-1045] Make log level configurable via airflow.cfg</p></li> |
| <li><p>[AIRFLOW-1047] Sanitize strings passed to Markup</p></li> |
| <li><p>[AIRFLOW-1040] Fix some small typos in comments and docstrings</p></li> |
| <li><p>[AIRFLOW-1017] get_task_instance shouldn’t throw exception when no TI</p></li> |
| <li><p>[AIRFLOW-1006] Add config_templates to MANIFEST</p></li> |
| <li><p>[AIRFLOW-999] Add support for Redis database</p></li> |
| <li><p>[AIRFLOW-1009] Remove SQLOperator from Concepts page</p></li> |
| <li><p>[AIRFLOW-1006] Move config templates to separate files</p></li> |
| <li><p>[AIRFLOW-1005] Improve Airflow startup time</p></li> |
| <li><p>[AIRFLOW-1010] Add convenience script for signing releases</p></li> |
| <li><p>[AIRFLOW-995] Remove reference to actual Airflow issue</p></li> |
| <li><p>[AIRFLOW-681] homepage doc link should pointing to apache repo not airbnb repo</p></li> |
| <li><p>[AIRFLOW-705][AIRFLOW-706] Fix run_command bugs</p></li> |
| <li><p>[AIRFLOW-990] Fix Py27 unicode logging in DockerOperator</p></li> |
| <li><p>[AIRFLOW-963] Fix non-rendered code examples</p></li> |
| <li><p>[AIRFLOW-969] Catch bad python_callable argument</p></li> |
| <li><p>[AIRFLOW-984] Enable subclassing of SubDagOperator</p></li> |
| <li><p>[AIRFLOW-997] Update setup.cfg to point to Apache</p></li> |
| <li><p>[AIRFLOW-994] Add MiNODES to the official airflow user list</p></li> |
| <li><p>[AIRFLOW-995][AIRFLOW-1] Update GitHub PR Template</p></li> |
| <li><p>[AIRFLOW-989] Do not mark dag run successful if unfinished tasks</p></li> |
| <li><p>[AIRFLOW-903] New configuration setting for the default dag view</p></li> |
| <li><p>[AIRFLOW-979] Add GovTech GDS</p></li> |
| <li><p>[AIRFLOW-933] Replace eval with literal_eval to prevent RCE</p></li> |
| <li><p>[AIRFLOW-974] Fix mkdirs race condition</p></li> |
| <li><p>[AIRFLOW-917] Fix formatting of error message</p></li> |
| <li><p>[AIRFLOW-770] Refactor BaseHook so env vars are always read</p></li> |
| <li><p>[AIRFLOW-900] Double trigger should not kill original task instance</p></li> |
| <li><p>[AIRFLOW-900] Fixes bugs in LocalTaskJob for double run protection</p></li> |
| <li><p>[AIRFLOW-932][AIRFLOW-932][AIRFLOW-921][AIRFLOW-910] Do not mark tasks removed when backfilling[</p></li> |
| <li><p>[AIRFLOW-961] run onkill when SIGTERMed</p></li> |
| <li><p>[AIRFLOW-910] Use parallel task execution for backfills</p></li> |
| <li><p>[AIRFLOW-967] Wrap strings in native for py2 ldap compatibility</p></li> |
| <li><p>[AIRFLOW-958] Improve tooltip readability</p></li> |
| <li><p>AIRFLOW-959 Cleanup and reorganize .gitignore</p></li> |
| <li><p>AIRFLOW-960 Add .editorconfig file</p></li> |
| <li><p>[AIRFLOW-931] Do not set QUEUED in TaskInstances</p></li> |
| <li><p>[AIRFLOW-956] Get docs working on readthedocs.org</p></li> |
| <li><p>[AIRFLOW-954] Fix configparser ImportError</p></li> |
| <li><p>[AIRFLOW-941] Use defined parameters for psycopg2</p></li> |
| <li><p>[AIRFLOW-943] Update Digital First Media in users list</p></li> |
| <li><p>[AIRFLOW-942] Add mytaxi to Airflow users</p></li> |
| <li><p>[AIRFLOW-939] add .swp to gitginore</p></li> |
| <li><p>[AIRFLOW-719] Prevent DAGs from ending prematurely</p></li> |
| <li><p>[AIRFLOW-938] Use test for True in task_stats queries</p></li> |
| <li><p>[AIRFLOW-937] Improve performance of task_stats</p></li> |
| <li><p>[AIRFLOW-933] use ast.literal_eval rather eval because ast.literal_eval does not execute input.</p></li> |
| <li><p>[AIRFLOW-925] Revert airflow.hooks change that cherry-pick picked</p></li> |
| <li><p>[AIRFLOW-919] Running tasks with no start date shouldn’t break a DAGs UI</p></li> |
| <li><p>[AIRFLOW-802][AIRFLOW-1] Add spark-submit operator/hook</p></li> |
| <li><p>[AIRFLOW-725] Use keyring to store credentials for JIRA</p></li> |
| <li><p>[AIRFLOW-916] Remove deprecated readfp function</p></li> |
| <li><p>[AIRFLOW-911] Add coloring and timing to tests</p></li> |
| <li><p>[AIRFLOW-906] Update Code icon from lightning bolt to file</p></li> |
| <li><p>[AIRFLOW-897] Prevent dagruns from failing with unfinished tasks</p></li> |
| <li><p>[AIRFLOW-896] Remove unicode to 8-bit conversion in BigQueryOperator</p></li> |
| <li><p>[AIRFLOW-899] Tasks in SCHEDULED state should be white in the UI instead of black</p></li> |
| <li><p>[AIRFLOW-895] Address Apache release incompliancies</p></li> |
| <li><p>[AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a dagrun has no start date</p></li> |
| <li><p>[AIRFLOW-880] Make webserver serve logs in a sane way for remote logs</p></li> |
| <li><p>[AIRFLOW-889] Fix minor error in the docstrings for BaseOperator</p></li> |
| <li><p>[AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing Booleans</p></li> |
| <li><p>[AIRFLOW-875] Add template to HttpSensor params</p></li> |
| <li><p>[AIRFLOW-866] Add FTPSensor</p></li> |
| <li><p>[AIRFLOW-881] Check if SubDagOperator is in DAG context manager</p></li> |
| <li><p>[AIRFLOW-885] Add change.org to the users list</p></li> |
| <li><p>[AIRFLOW-836] Use POST and CSRF for state changing endpoints</p></li> |
| <li><p>[AIRFLOW-862] Fix Unit Tests for DaskExecutor</p></li> |
| <li><p>[AIRFLOW-887] Support future v0.16</p></li> |
| <li><p>[AIRFLOW-886] Pass result to post_execute() hook</p></li> |
| <li><p>[AIRFLOW-871] change logging.warn() into warning()</p></li> |
| <li><p>[AIRFLOW-882] Remove unnecessary dag>>op assignment in docs</p></li> |
| <li><p>[AIRFLOW-861] make pickle_info endpoint be login_required</p></li> |
| <li><p>[AIRFLOW-869] Refactor mark success functionality</p></li> |
| <li><p>[AIRFLOW-877] Remove .sql template extension from GCS download operator</p></li> |
| <li><p>[AIRFLOW-826] Add Zendesk hook</p></li> |
| <li><p>[AIRFLOW-842] do not query the DB with an empty IN clause</p></li> |
| <li><p>[AIRFLOW-834] change raise StopIteration into return</p></li> |
| <li><p>[AIRFLOW-832] Let debug server run without SSL</p></li> |
| <li><p>[AIRFLOW-862] Add DaskExecutor</p></li> |
| <li><p>[AIRFLOW-858] Configurable database name for DB operators</p></li> |
| <li><p>[AIRFLOW-863] Example DAGs should have recent start dates</p></li> |
| <li><p>[AIRFLOW-853] use utf8 encoding for stdout line decode</p></li> |
| <li><p>[AIRFLOW-857] Use library assert statements instead of conditionals</p></li> |
| <li><p>[AIRFLOW-856] Make sure execution date is set for local client</p></li> |
| <li><p>[AIRFLOW-854] Add OKI as Airflow user</p></li> |
| <li><p>[AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log verbosity</p></li> |
| <li><p>[AIRFLOW-814] Fix Presto*CheckOperator.__init__</p></li> |
| <li><p>[AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer</p></li> |
| <li><p>[AIRFLOW-844] Fix cgroups directory creation</p></li> |
| <li><p>[AIRFLOW-831] Restore import to fix broken tests</p></li> |
| <li><p>[AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN exclusively from settings</p></li> |
| <li><p>[AIRFLOW-694] Fix config behaviour for empty envvar</p></li> |
| <li><p>[AIRFLOW-365] Set dag.fileloc explicitly and use for Code view</p></li> |
| <li><p>[AIRFLOW-781] Allow DataFlowOperators to accept jobs stored in GCS</p></li> |
| </ul> |
| </div> |
| <div class="section" id="airflow-1-8-2-2017-09-04"> |
| <h2>Airflow 1.8.2, 2017-09-04<a class="headerlink" href="#airflow-1-8-2-2017-09-04" title="Permalink to this headline">¶</a></h2> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing Booleans</p></li> |
| <li><p>[AIRFLOW-1296] Propagate SKIPPED to all downstream tasks</p></li> |
| <li><p>Re-enable caching for hadoop components</p></li> |
| <li><p>Pin Hive and Hadoop to a specific version and create writable warehouse dir</p></li> |
| <li><p>[AIRFLOW-1308] Disable nanny usage for Dask</p></li> |
| <li><p>Updating CHANGELOG for 1.8.2rc1</p></li> |
| <li><p>[AIRFLOW-1294] Backfills can loose tasks to execute</p></li> |
| <li><p>[AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF requirements</p></li> |
| <li><p>[AIRFLOW-XXX] Set version to 1.8.2rc1</p></li> |
| <li><p>[AIRFLOW-1160] Update Spark parameters for Mesos</p></li> |
| <li><p>[AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2 templates</p></li> |
| <li><p>[AIRFLOW-1119] Fix unload query so headers are on first row[]</p></li> |
| <li><p>[AIRFLOW-1089] Add Spark application arguments</p></li> |
| <li><p>[AIRFLOW-1078] Fix latest_runs endpoint for old flask versions</p></li> |
| <li><p>[AIRFLOW-1074] Don’t count queued tasks for concurrency limits</p></li> |
| <li><p>[AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView</p></li> |
| <li><p>[AIRFLOW-1038] Specify celery serialization options explicitly</p></li> |
| <li><p>[AIRFLOW-1036] Randomize exponential backoff</p></li> |
| <li><p>[AIRFLOW-993] Update date inference logic</p></li> |
| <li><p>[AIRFLOW-1167] Support microseconds in FTPHook modification time</p></li> |
| <li><p>[AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change</p></li> |
| <li><p>[AIRFLOW-1263] Dynamic height for charts</p></li> |
| <li><p>[AIRFLOW-1266] Increase width of gantt y axis</p></li> |
| <li><p>[AIRFLOW-1290] set docs author to ‘Apache Airflow’</p></li> |
| <li><p>[AIRFLOW-1282] Fix known event column sorting</p></li> |
| <li><p>[AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun</p></li> |
| <li><p>[AIRFLOW-1192] Some enhancements to qubole_operator</p></li> |
| <li><p>[AIRFLOW-1281] Sort variables by key field by default</p></li> |
| <li><p>[AIRFLOW-1244] Forbid creation of a pool with empty name</p></li> |
| <li><p>[AIRFLOW-1243] DAGs table has no default entries to show</p></li> |
| <li><p>[AIRFLOW-1227] Remove empty column on the Logs view</p></li> |
| <li><p>[AIRFLOW-1226] Remove empty column on the Jobs view</p></li> |
| <li><p>[AIRFLOW-1199] Fix create modal</p></li> |
| <li><p>[AIRFLOW-1200] Forbid creation of a variable with an empty key</p></li> |
| <li><p>[AIRFLOW-1186] Sort dag.get_task_instances by execution_date</p></li> |
| <li><p>[AIRFLOW-1145] Fix closest_date_partition function with before set to True If we’re looking for the closest date before, we should take the latest date in the list of date before.</p></li> |
| <li><p>[AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection</p></li> |
| <li><p>[AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately</p></li> |
| <li><p>[AIRFLOW-1150] Fix scripts execution in sparksql hook[]</p></li> |
| <li><p>[AIRFLOW-1168] Add closing() to all connections and cursors</p></li> |
| <li><p>[AIRFLOW-XXX] Updating CHANGELOG, README, and UPDATING after 1.8.1 release</p></li> |
| </ul> |
| </div> |
| <div class="section" id="airflow-1-8-1-2017-05-09"> |
| <h2>Airflow 1.8.1, 2017-05-09<a class="headerlink" href="#airflow-1-8-1-2017-05-09" title="Permalink to this headline">¶</a></h2> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-1142] SubDAG Tasks Not Executed Even Though All Dependencies Met</p></li> |
| <li><p>[AIRFLOW-1138] Add licenses to files in scripts directory</p></li> |
| <li><p>[AIRFLOW-1127] Move license notices to LICENSE instead of NOTICE</p></li> |
| <li><p>[AIRFLOW-1124] Do not set all task instances to scheduled on backfill</p></li> |
| <li><p>[AIRFLOW-1120] Update version view to include Apache prefix</p></li> |
| <li><p>[AIRFLOW-1062] DagRun#find returns wrong result if external_trigger=False is specified</p></li> |
| <li><p>[AIRFLOW-1054] Fix broken import on test_dag</p></li> |
| <li><p>[AIRFLOW-1050] Retries ignored - regression</p></li> |
| <li><p>[AIRFLOW-1033] TypeError: can’t compare datetime.datetime to None</p></li> |
| <li><p>[AIRFLOW-1017] get_task_instance should return None instead of throw an exception for non-existent TIs</p></li> |
| <li><p>[AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs</p></li> |
| <li><p>[AIRFLOW-1004] <cite>airflow webserver -D</cite> runs in foreground</p></li> |
| <li><p>[AIRFLOW-1001] Landing Time shows “unsupported operand type(s) for -: ‘datetime.datetime’ and ‘NoneType’” on example_subdag_operator</p></li> |
| <li><p>[AIRFLOW-1000] Rebrand to Apache Airflow instead of Airflow</p></li> |
| <li><p>[AIRFLOW-989] Clear Task Regression</p></li> |
| <li><p>[AIRFLOW-974] airflow.util.file mkdir has a race condition</p></li> |
| <li><p>[AIRFLOW-906] Update Code icon from lightning bolt to file</p></li> |
| <li><p>[AIRFLOW-858] Configurable database name for DB operators</p></li> |
| <li><p>[AIRFLOW-853] ssh_execute_operator.py stdout decode default to ASCII</p></li> |
| <li><p>[AIRFLOW-832] Fix debug server</p></li> |
| <li><p>[AIRFLOW-817] Trigger dag fails when using CLI + API</p></li> |
| <li><p>[AIRFLOW-816] Make sure to pull nvd3 from local resources</p></li> |
| <li><p>[AIRFLOW-815] Add previous/next execution dates to available default variables.</p></li> |
| <li><p>[AIRFLOW-813] Fix unterminated unit tests in tests.job (tests/job.py)</p></li> |
| <li><p>[AIRFLOW-812] Scheduler job terminates when there is no dag file</p></li> |
| <li><p>[AIRFLOW-806] UI should properly ignore DAG doc when it is None</p></li> |
| <li><p>[AIRFLOW-794] Consistent access to DAGS_FOLDER and SQL_ALCHEMY_CONN</p></li> |
| <li><p>[AIRFLOW-785] ImportError if cgroupspy is not installed</p></li> |
| <li><p>[AIRFLOW-784] Cannot install with funcsigs > 1.0.0</p></li> |
| <li><p>[AIRFLOW-780] The UI no longer shows broken DAGs</p></li> |
| <li><p>[AIRFLOW-777] dag_is_running is initlialized to True instead of False</p></li> |
| <li><p>[AIRFLOW-719] Skipped operations make DAG finish prematurely</p></li> |
| <li><p>[AIRFLOW-694] Empty env vars do not overwrite non-empty config values</p></li> |
| <li><p>[AIRFLOW-492] Insert into dag_stats table results into failed task while task itself succeeded</p></li> |
| <li><p>[AIRFLOW-139] Executing VACUUM with PostgresOperator</p></li> |
| <li><p>[AIRFLOW-111] DAG concurrency is not honored</p></li> |
| <li><p>[AIRFLOW-88] Improve clarity Travis CI reports</p></li> |
| </ul> |
| </div> |
| <div class="section" id="airflow-1-8-0-2017-03-12"> |
| <h2>Airflow 1.8.0, 2017-03-12<a class="headerlink" href="#airflow-1-8-0-2017-03-12" title="Permalink to this headline">¶</a></h2> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-900] Double trigger should not kill original task instance</p></li> |
| <li><p>[AIRFLOW-900] Fixes bugs in LocalTaskJob for double run protection</p></li> |
| <li><p>[AIRFLOW-932] Do not mark tasks removed when backfilling</p></li> |
| <li><p>[AIRFLOW-961] run onkill when SIGTERMed</p></li> |
| <li><p>[AIRFLOW-910] Use parallel task execution for backfills</p></li> |
| <li><p>[AIRFLOW-967] Wrap strings in native for py2 ldap compatibility</p></li> |
| <li><p>[AIRFLOW-941] Use defined parameters for psycopg2</p></li> |
| <li><p>[AIRFLOW-719] Prevent DAGs from ending prematurely</p></li> |
| <li><p>[AIRFLOW-938] Use test for True in task_stats queries</p></li> |
| <li><p>[AIRFLOW-937] Improve performance of task_stats</p></li> |
| <li><p>[AIRFLOW-933] use ast.literal_eval rather eval because ast.literal_eval does not execute input.</p></li> |
| <li><p>[AIRFLOW-925] Revert airflow.hooks change that cherry-pick picked</p></li> |
| <li><p>[AIRFLOW-919] Running tasks with no start date shouldn’t break a DAGs UI</p></li> |
| <li><p>[AIRFLOW-802] Add spark-submit operator/hook</p></li> |
| <li><p>[AIRFLOW-897] Prevent dagruns from failing with unfinished tasks</p></li> |
| <li><p>[AIRFLOW-861] make pickle_info endpoint be login_required</p></li> |
| <li><p>[AIRFLOW-853] use utf8 encoding for stdout line decode</p></li> |
| <li><p>[AIRFLOW-856] Make sure execution date is set for local client</p></li> |
| <li><p>[AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log verbosity</p></li> |
| <li><p>[AIRFLOW-831] Restore import to fix broken tests</p></li> |
| <li><p>[AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN exclusively from settings</p></li> |
| <li><p>[AIRFLOW-694] Fix config behaviour for empty envvar</p></li> |
| <li><p>[AIRFLOW-365] Set dag.fileloc explicitly and use for Code view</p></li> |
| <li><p>[AIRFLOW-931] Do not set QUEUED in TaskInstances</p></li> |
| <li><p>[AIRFLOW-899] Tasks in SCHEDULED state should be white in the UI instead of black</p></li> |
| <li><p>[AIRFLOW-895] Address Apache release incompliancies</p></li> |
| <li><p>[AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a dagrun has no start date</p></li> |
| <li><p>[AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer</p></li> |
| <li><p>[AIRFLOW-863] Example DAGs should have recent start dates</p></li> |
| <li><p>[AIRFLOW-869] Refactor mark success functionality</p></li> |
| <li><p>[AIRFLOW-856] Make sure execution date is set for local client</p></li> |
| <li><p>[AIRFLOW-814] Fix Presto*CheckOperator.__init__</p></li> |
| <li><p>[AIRFLOW-844] Fix cgroups directory creation</p></li> |
| <li><p>[AIRFLOW-816] Use static nvd3 and d3</p></li> |
| <li><p>[AIRFLOW-821] Fix py3 compatibility</p></li> |
| <li><p>[AIRFLOW-817] Check for None value of execution_date in endpoint</p></li> |
| <li><p>[AIRFLOW-822] Close db before exception</p></li> |
| <li><p>[AIRFLOW-815] Add prev/next execution dates to template variables</p></li> |
| <li><p>[AIRFLOW-813] Fix unterminated unit tests in SchedulerJobTest</p></li> |
| <li><p>[AIRFLOW-813] Fix unterminated scheduler unit tests</p></li> |
| <li><p>[AIRFLOW-806] UI should properly ignore DAG doc when it is None</p></li> |
| <li><p>[AIRFLOW-812] Fix the scheduler termination bug.</p></li> |
| <li><p>[AIRFLOW-780] Fix dag import errors no longer working</p></li> |
| <li><p>[AIRFLOW-783] Fix py3 incompatibility in BaseTaskRunner</p></li> |
| <li><p>[AIRFLOW-810] Correct down_revision dag_id/state index creation</p></li> |
| <li><p>[AIRFLOW-807] Improve scheduler performance for large DAGs</p></li> |
| <li><p>[AIRFLOW-798] Check return_code before forcing termination</p></li> |
| <li><p>[AIRFLOW-139] Let psycopg2 handle autocommit for PostgresHook</p></li> |
| <li><p>[AIRFLOW-776] Add missing cgroups devel dependency</p></li> |
| <li><p>[AIRFLOW-777] Fix expression to check if a DagRun is in running state</p></li> |
| <li><p>[AIRFLOW-785] Don’t import CgroupTaskRunner at global scope</p></li> |
| <li><p>[AIRFLOW-784] Pin funcsigs to 1.0.0</p></li> |
| <li><p>[AIRFLOW-624] Fix setup.py to not import airflow.version as version</p></li> |
| <li><p>[AIRFLOW-779] Task should fail with specific message when deleted</p></li> |
| <li><p>[AIRFLOW-778] Fix completely broken MetastorePartitionSensor</p></li> |
| <li><p>[AIRFLOW-739] Set pickle_info log to debug</p></li> |
| <li><p>[AIRFLOW-771] Make S3 logs append instead of clobber</p></li> |
| <li><p>[AIRFLOW-773] Fix flaky datetime addition in api test</p></li> |
| <li><p>[AIRFLOW-219][AIRFLOW-398] Cgroups + impersonation</p></li> |
| <li><p>[AIRFLOW-683] Add jira hook, operator and sensor</p></li> |
| <li><p>[AIRFLOW-762] Add Google DataProc delete operator</p></li> |
| <li><p>[AIRFLOW-760] Update systemd config</p></li> |
| <li><p>[AIRFLOW-759] Use previous dag_run to verify depend_on_past</p></li> |
| <li><p>[AIRFLOW-757] Set child_process_log_directory default more sensible</p></li> |
| <li><p>[AIRFLOW-692] Open XCom page to super-admins only</p></li> |
| <li><p>[AIRFLOW-737] Fix HDFS Sensor directory.</p></li> |
| <li><p>[AIRFLOW-747] Fix retry_delay not honoured</p></li> |
| <li><p>[AIRFLOW-558] Add Support for dag.catchup=(True|False) Option</p></li> |
| <li><p>[AIRFLOW-489] Allow specifying execution date in trigger_dag API</p></li> |
| <li><p>[AIRFLOW-738] Commit deleted xcom items before insert</p></li> |
| <li><p>[AIRFLOW-729] Add Google Cloud Dataproc cluster creation operator</p></li> |
| <li><p>[AIRFLOW-728] Add Google BigQuery table sensor</p></li> |
| <li><p>[AIRFLOW-741] Log to debug instead of info for app.py</p></li> |
| <li><p>[AIRFLOW-731] Fix period bug for NamedHivePartitionSensor</p></li> |
| <li><p>[AIRFLOW-740] Pin jinja2 to < 2.9.0</p></li> |
| <li><p>[AIRFLOW-663] Improve time units for task performance charts</p></li> |
| <li><p>[AIRFLOW-665] Fix email attachments</p></li> |
| <li><p>[AIRFLOW-734] Fix SMTP auth regression when not using user/pass</p></li> |
| <li><p>[AIRFLOW-702] Fix LDAP Regex Bug</p></li> |
| <li><p>[AIRFLOW-717] Add Cloud Storage updated sensor</p></li> |
| <li><p>[AIRFLOW-695] Retries do not execute because dagrun is in FAILED state</p></li> |
| <li><p>[AIRFLOW-673] Add operational metrics test for SchedulerJob</p></li> |
| <li><p>[AIRFLOW-727] try_number is not increased</p></li> |
| <li><p>[AIRFLOW-715] A more efficient HDFS Sensor:</p></li> |
| <li><p>[AIRFLOW-716] Allow AVRO BigQuery load-job without schema</p></li> |
| <li><p>[AIRFLOW-718] Allow the query URI for DataProc Pig</p></li> |
| <li><p>Log needs to be part of try/catch block</p></li> |
| <li><p>[AIRFLOW-721] Descendant process can disappear before termination</p></li> |
| <li><p>[AIRFLOW-403] Bash operator’s kill method leaves underlying processes running</p></li> |
| <li><p>[AIRFLOW-657] Add AutoCommit Parameter for MSSQL</p></li> |
| <li><p>[AIRFLOW-641] Improve pull request instructions</p></li> |
| <li><p>[AIRFLOW-685] Add test for MySqlHook.bulk_load()</p></li> |
| <li><p>[AIRFLOW-686] Match auth backend config section</p></li> |
| <li><p>[AIRFLOW-691] Add SSH KeepAlive option to SSH_hook</p></li> |
| <li><p>[AIRFLOW-709] Use same engine for migrations and reflection</p></li> |
| <li><p>[AIRFLOW-700] Update to reference to web authentication documentation</p></li> |
| <li><p>[AIRFLOW-649] Support non-sched DAGs in LatestOnlyOp</p></li> |
| <li><p>[AIRFLOW-712] Fix AIRFLOW-667 to use proper HTTP error properties</p></li> |
| <li><p>[AIRFLOW-710] Add OneFineStay as official user</p></li> |
| <li><p>[AIRFLOW-703][AIRFLOW-1] Stop Xcom being cleared too early</p></li> |
| <li><p>[AIRFLOW-679] Stop concurrent task instances from running</p></li> |
| <li><p>[AIRFLOW-704][AIRFLOW-1] Fix invalid syntax in BQ hook</p></li> |
| <li><p>[AIRFLOW-667] Handle BigQuery 503 error</p></li> |
| <li><p>[AIRFLOW-680] Disable connection pool for commands</p></li> |
| <li><p>[AIRFLOW-678] Prevent scheduler from double triggering TIs</p></li> |
| <li><p>[AIRFLOW-677] Kill task if it fails to heartbeat</p></li> |
| <li><p>[AIRFLOW-674] Ability to add descriptions for DAGs</p></li> |
| <li><p>[AIRFLOW-682] Bump MAX_PERIODS to make mark_success work for large DAGs</p></li> |
| <li><p>Use jdk selector to set required jdk</p></li> |
| <li><p>[AIRFLOW-647] Restore dag.get_active_runs</p></li> |
| <li><p>[AIRFLOW-662] Change seasons to months in project description</p></li> |
| <li><p>[AIRFLOW-656] Add dag/task/date index to xcom table</p></li> |
| <li><p>[AIRFLOW-658] Improve schema_update_options in GCP</p></li> |
| <li><p>[AIRFLOW-41] Fix pool oversubscription</p></li> |
| <li><p>[AIRFLOW-489] Add API Framework</p></li> |
| <li><p>[AIRFLOW-653] Add some missing endpoint tests</p></li> |
| <li><p>[AIRFLOW-652] Remove obsolete endpoint</p></li> |
| <li><p>[AIRFLOW-345] Add contrib ECSOperator</p></li> |
| <li><p>[AIRFLOW-650] Adding Celect to user list</p></li> |
| <li><p>[AIRFLOW-510] Filter Paused Dags, show Last Run & Trigger Dag</p></li> |
| <li><p>[AIRFLOW-643] Improve date handling for sf_hook</p></li> |
| <li><p>[AIRFLOW-638] Add schema_update_options to GCP ops</p></li> |
| <li><p>[AIRFLOW-640] Install and enable nose-ignore-docstring</p></li> |
| <li><p>[AIRFLOW-639]AIRFLOW-639] Alphasort package names</p></li> |
| <li><p>[AIRFLOW-375] Fix pylint errors</p></li> |
| <li><p>[AIRFLOW-347] Show empty DAG runs in tree view</p></li> |
| <li><p>[AIRFLOW-628] Adding SalesforceHook to contrib/hooks</p></li> |
| <li><p>[AIRFLOW-514] hive hook loads data from pandas DataFrame into hive and infers types</p></li> |
| <li><p>[AIRFLOW-565] Fixes DockerOperator on Python3.x</p></li> |
| <li><p>[AIRFLOW-635] Encryption option for S3 hook</p></li> |
| <li><p>[AIRFLOW-137] Fix max_active_runs on clearing tasks</p></li> |
| <li><p>[AIRFLOW-343] Fix schema plumbing in HiveServer2Hook</p></li> |
| <li><p>[AIRFLOW-130] Fix ssh operator macosx</p></li> |
| <li><p>[AIRFLOW-633] Show TI attributes in TI view</p></li> |
| <li><p>[AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment</p></li> |
| <li><p>[AIRFLOW-533] Set autocommit via set_autocommit</p></li> |
| <li><p>[AIRFLOW-629] stop pinning lxml</p></li> |
| <li><p>[AIRFLOW-464] Add setdefault method to Variable</p></li> |
| <li><p>[AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment</p></li> |
| <li><p>[AIRFLOW-591] Add datadog hook & sensor</p></li> |
| <li><p>[AIRFLOW-561] Add RedshiftToS3Transfer operator</p></li> |
| <li><p>[AIRFLOW-570] Pass root to date form on gantt</p></li> |
| <li><p>[AIRFLOW-504] Store fractional seconds in MySQL tables</p></li> |
| <li><p>[AIRFLOW-623] LDAP attributes not always a list</p></li> |
| <li><p>[AIRFLOW-611] source_format in BigQueryBaseCursor</p></li> |
| <li><p>[AIRFLOW-619] Fix exception in Gannt chart</p></li> |
| <li><p>[AIRFLOW-618] Cast DateTimes to avoid sqllite errors</p></li> |
| <li><p>[AIRFLOW-422] Add JSON endpoint for task info</p></li> |
| <li><p>[AIRFLOW-616][AIRFLOW-617] Minor fixes to PR tool UX</p></li> |
| <li><p>[AIRFLOW-179] Fix DbApiHook with non-ASCII chars</p></li> |
| <li><p>[AIRFLOW-566] Add timeout while fetching logs</p></li> |
| <li><p>[AIRFLOW-615] Set graph glyphicon first</p></li> |
| <li><p>[AIRFLOW-609] Add application_name to PostgresHook</p></li> |
| <li><p>[AIRFLOW-604] Revert .first() to .one()</p></li> |
| <li><p>[AIRFLOW-370] Create AirflowConfigException in exceptions.py</p></li> |
| <li><p>[AIRFLOW-582] Fixes TI.get_dagrun filter (removes start_date)</p></li> |
| <li><p>[AIRFLOW-568] Fix double task_stats count if a DagRun is active</p></li> |
| <li><p>[AIRFLOW-585] Fix race condition in backfill execution loop</p></li> |
| <li><p>[AIRFLOW-580] Prevent landscape warning on .format</p></li> |
| <li><p>[AIRFLOW-597] Check if content is None, not false-equivalent</p></li> |
| <li><p>[AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.</p></li> |
| <li><p>[AIRFLOW-453] Add XCom Admin Page</p></li> |
| <li><p>[AIRFLOW-588] Add Google Cloud Storage Object sensor[]</p></li> |
| <li><p>[AIRFLOW-592] example_xcom import Error</p></li> |
| <li><p>[AIRFLOW-587] Fix incorrect scope for Google Auth[]</p></li> |
| <li><p>[AIRFLOW-589] Add templatable job_name[]</p></li> |
| <li><p>[AIRFLOW-227] Show running config in config view</p></li> |
| <li><p>[AIRFLOW-319]AIRFLOW-319] xcom push response in HTTP Operator</p></li> |
| <li><p>[AIRFLOW-385] Add symlink to latest scheduler log directory</p></li> |
| <li><p>[AIRFLOW-583] Fix decode error in gcs_to_bq</p></li> |
| <li><p>[AIRFLOW-96] s3_conn_id using environment variable</p></li> |
| <li><p>[AIRFLOW-575] Clarify tutorial and FAQ about <cite>schedule_interval</cite> always inheriting from DAG object</p></li> |
| <li><p>[AIRFLOW-577] Output BigQuery job for improved debugging</p></li> |
| <li><p>[AIRFLOW-560] Get URI & SQLA engine from Connection</p></li> |
| <li><p>[AIRFLOW-518] Require DataProfilingMixin for Variables CRUD</p></li> |
| <li><p>[AIRFLOW-553] Fix load path for filters.js</p></li> |
| <li><p>[AIRFLOW-554] Add Jinja support to Spark-sql</p></li> |
| <li><p>[AIRFLOW-550] Make ssl config check empty string safe</p></li> |
| <li><p>[AIRFLOW-500] Use id for github allowed teams</p></li> |
| <li><p>[AIRFLOW-556] Add UI PR guidelines</p></li> |
| <li><p>[AIRFLOW-358][AIRFLOW-430] Add <cite>connections</cite> cli</p></li> |
| <li><p>[AIRFLOW-548] Load DAGs immediately & continually</p></li> |
| <li><p>[AIRFLOW-539] Updated BQ hook and BQ operator to support Standard SQL.</p></li> |
| <li><p>[AIRFLOW-378] Add string casting to params of spark-sql operator</p></li> |
| <li><p>[AIRFLOW-544] Add Pause/Resume toggle button</p></li> |
| <li><p>[AIRFLOW-333][AIRFLOW-258] Fix non-module plugin components</p></li> |
| <li><p>[AIRFLOW-542] Add tooltip to DAGs links icons</p></li> |
| <li><p>[AIRFLOW-530] Update docs to reflect connection environment var has to be in uppercase</p></li> |
| <li><p>[AIRFLOW-525] Update template_fields in Qubole Op</p></li> |
| <li><p>[AIRFLOW-480] Support binary file download from GCS</p></li> |
| <li><p>[AIRFLOW-198] Implement latest_only_operator</p></li> |
| <li><p>[AIRFLOW-91] Add SSL config option for the webserver</p></li> |
| <li><p>[AIRFLOW-191] Fix connection leak with PostgreSQL backend</p></li> |
| <li><p>[AIRFLOW-512] Fix ‘bellow’ typo in docs & comments</p></li> |
| <li><p>[AIRFLOW-509][AIRFLOW-1] Create operator to delete tables in BigQuery</p></li> |
| <li><p>[AIRFLOW-498] Remove hard-coded gcp project id</p></li> |
| <li><p>[AIRFLOW-505] Support unicode characters in authors’ names</p></li> |
| <li><p>[AIRFLOW-494] Add per-operator success/failure metrics</p></li> |
| <li><p>[AIRFLOW-488] Fix test_simple fail</p></li> |
| <li><p>[AIRFLOW-468] Update Panda requirement to 0.17.1</p></li> |
| <li><p>[AIRFLOW-159] Add cloud integration section + GCP documentation</p></li> |
| <li><p>[AIRFLOW-477][AIRFLOW-478] Restructure security section for clarity</p></li> |
| <li><p>[AIRFLOW-467] Allow defining of project_id in BigQueryHook</p></li> |
| <li><p>[AIRFLOW-483] Change print to logging statement</p></li> |
| <li><p>[AIRFLOW-475] make the segment granularity in Druid hook configurable</p></li> |
| </ul> |
| </div> |
| <div class="section" id="airflow-1-7-2"> |
| <h2>Airflow 1.7.2<a class="headerlink" href="#airflow-1-7-2" title="Permalink to this headline">¶</a></h2> |
| <ul class="simple"> |
| <li><p>[AIRFLOW-463] Link Airflow icon to landing page</p></li> |
| <li><p>[AIRFLOW-149] Task Dependency Engine + Why Isn’t My Task Running View</p></li> |
| <li><p>[AIRFLOW-361] Add default failure handler for the Qubole Operator</p></li> |
| <li><p>[AIRFLOW-353] Fix dag run status update failure</p></li> |
| <li><p>[AIRFLOW-447] Store source URIs in Python 3 compatible list</p></li> |
| <li><p>[AIRFLOW-443] Make module names unique when importing</p></li> |
| <li><p>[AIRFLOW-444] Add Google authentication backend</p></li> |
| <li><p>[AIRFLOW-446][AIRFLOW-445] Adds missing dataproc submit options</p></li> |
| <li><p>[AIRFLOW-431] Add CLI for CRUD operations on pools</p></li> |
| <li><p>[AIRFLOW-329] Update Dag Overview Page with Better Status Columns</p></li> |
| <li><p>[AIRFLOW-360] Fix style warnings in models.py</p></li> |
| <li><p>[AIRFLOW-425] Add white fill for null state tasks in tree view.</p></li> |
| <li><p>[AIRFLOW-69] Use dag runs in backfill jobs</p></li> |
| <li><p>[AIRFLOW-415] Make dag_id not found error clearer</p></li> |
| <li><p>[AIRFLOW-416] Use ordinals in README’s company list</p></li> |
| <li><p>[AIRFLOW-369] Allow setting default DAG orientation</p></li> |
| <li><p>[AIRFLOW-410] Add 2 Q/A to the FAQ in the docs</p></li> |
| <li><p>[AIRFLOW-407] Add different colors for some sensors</p></li> |
| <li><p>[AIRFLOW-414] Improve error message for missing FERNET_KEY</p></li> |
| <li><p>[AIRFLOW-406] Sphinx/rst fixes</p></li> |
| <li><p>[AIRFLOW-412] Fix lxml dependency</p></li> |
| <li><p>[AIRFLOW-413] Fix unset path bug when backfilling via pickle</p></li> |
| <li><p>[AIRFLOW-78] Airflow clear leaves dag_runs</p></li> |
| <li><p>[AIRFLOW-402] Remove NamedHivePartitionSensor static check, add docs</p></li> |
| <li><p>[AIRFLOW-394] Add an option to the Task Duration graph to show cumulative times</p></li> |
| <li><p>[AIRFLOW-404] Retry download if unpacking fails for hive</p></li> |
| <li><p>[AIRFLOW-276] Gunicorn rolling restart</p></li> |
| <li><p>[AIRFLOW-399] Remove dags/testdruid.py</p></li> |
| <li><p>[AIRFLOW-400] models.py/DAG.set_dag_runs_state() does not correctly set state</p></li> |
| <li><p>[AIRFLOW-395] Fix colon/equal signs typo for resources in default config</p></li> |
| <li><p>[AIRFLOW-397] Documentation: Fix typo “instatiating” to “instantiating”</p></li> |
| <li><p>[AIRFLOW-395] Remove trailing commas from resources in config</p></li> |
| <li><p>[AIRFLOW-388] Add a new chart for Task_Tries for each DAG</p></li> |
| <li><p>[AIRFLOW-322] Fix typo in FAQ section</p></li> |
| <li><p>[AIRFLOW-375] Pylint fixes</p></li> |
| <li><p>limit scope to user email only AIRFLOW-386</p></li> |
| <li><p>[AIRFLOW-383] Cleanup example qubole operator dag</p></li> |
| <li><p>[AIRFLOW-160] Parse DAG files through child processes</p></li> |
| <li><p>[AIRFLOW-381] Manual UI Dag Run creation: require dag_id field</p></li> |
| <li><p>[AIRFLOW-373] Enhance CLI variables functionality</p></li> |
| <li><p>[AIRFLOW-379] Enhance Variables page functionality: import/export variables</p></li> |
| <li><p>[AIRFLOW-331] modify the LDAP authentication config lines in ‘Security’ sample codes</p></li> |
| <li><p>[AIRFLOW-356][AIRFLOW-355][AIRFLOW-354] Replace nobr, enable DAG only exists locally message, change edit DAG icon</p></li> |
| <li><p>[AIRFLOW-362] Import __future__ division</p></li> |
| <li><p>[AIRFLOW-359] Pin flask-login to 0.2.11</p></li> |
| <li><p>[AIRFLOW-261] Add bcc and cc fields to EmailOperator</p></li> |
| <li><p>[AIRFLOW-348] Fix code style warnings</p></li> |
| <li><p>[AIRFLOW-349] Add metric for number of zombies killed</p></li> |
| <li><p>[AIRFLOW-340] Remove unused dependency on Babel</p></li> |
| <li><p>[AIRFLOW-339]: Ability to pass a flower conf file</p></li> |
| <li><p>[AIRFLOW-341][operators] Add resource requirement attributes to operators</p></li> |
| <li><p>[AIRFLOW-335] Fix simple style errors/warnings</p></li> |
| <li><p>[AIRFLOW-337] Add __repr__ to VariableAccessor and VariableJsonAccessor</p></li> |
| <li><p>[AIRFLOW-334] Fix using undefined variable</p></li> |
| <li><p>[AIRFLOW-315] Fix blank lines code style warnings</p></li> |
| <li><p>[AIRFLOW-306] Add Spark-sql Hook and Operator</p></li> |
| <li><p>[AIRFLOW-327] Add rename method to the FTPHook</p></li> |
| <li><p>[AIRFLOW-321] Fix a wrong code example about tests/dags</p></li> |
| <li><p>[AIRFLOW-316] Always check DB state for Backfill Job execution</p></li> |
| <li><p>[AIRFLOW-264] Adding workload management for Hive</p></li> |
| <li><p>[AIRFLOW-297] support exponential backoff option for retry delay</p></li> |
| <li><p>[AIRFLOW-31][AIRFLOW-200] Add note to updating.md</p></li> |
| <li><p>[AIRFLOW-307] There is no __neq__ python magic method.</p></li> |
| <li><p>[AIRFLOW-309] Add requirements of develop dependencies to docs</p></li> |
| <li><p>[AIRFLOW-307] Rename __neq__ to __ne__ python magic method.</p></li> |
| <li><p>[AIRFLOW-313] Fix code style for sqoop_hook.py</p></li> |
| <li><p>[AIRFLOW-311] Fix wrong path in CONTRIBUTING.md</p></li> |
| <li><p>[AIRFLOW-24] DataFlow Java Operator</p></li> |
| <li><p>[AIRFLOW-308] Add link to refresh DAG within DAG view header</p></li> |
| <li><p>[AIRFLOW-314] Fix BigQuery cursor run_table_upsert method</p></li> |
| <li><p>[AIRFLOW-298] fix incubator diclaimer in docs</p></li> |
| <li><p>[AIRFLOW-284] HiveServer2Hook fix for cursor scope for get_results</p></li> |
| <li><p>[AIRFLOW-260] More graceful exit when issues can’t be closed</p></li> |
| <li><p>[AIRFLOW-260] Handle case when no version is found</p></li> |
| <li><p>[AIRFLOW-228] Handle empty version list in PR tool</p></li> |
| <li><p>[AIRFLOW-302] Improve default squash commit message</p></li> |
| <li><p>[AIRFLOW-187] Improve prompt styling</p></li> |
| <li><p>[AIRFLOW-187] Fix typo in argument name</p></li> |
| <li><p>[AIRFLOW-187] Move “Close XXX” message to end of squash commit</p></li> |
| <li><p>[AIRFLOW-247] Add EMR hook, operators and sensors. Add AWS base hook</p></li> |
| <li><p>[AIRFLOW-301] Fix broken unit test</p></li> |
| <li><p>[AIRFLOW-100] Add execution_date_fn to ExternalTaskSensor</p></li> |
| <li><p>[AIRFLOW-282] Remove PR Tool logic that depends on version formatting</p></li> |
| <li><p>[AIRFLOW-291] Add index for state in TI table</p></li> |
| <li><p>[AIRFLOW-269] Add some unit tests for PostgreSQL</p></li> |
| <li><p>[AIRFLOW-296] template_ext is being treated as a string rather than a tuple in qubole operator</p></li> |
| <li><p>[AIRFLOW-286] Improve FTPHook to implement context manager interface</p></li> |
| <li><p>[AIRFLOW-243] Create NamedHivePartitionSensor</p></li> |
| <li><p>[AIRFLOW-246] Improve dag_stats endpoint query</p></li> |
| <li><p>[AIRFLOW-189] Highlighting of Parent/Child nodes in Graphs</p></li> |
| <li><p>[ARFLOW-255] Check dagrun timeout when comparing active runs</p></li> |
| <li><p>[AIRFLOW-281] Add port to mssql_hook</p></li> |
| <li><p>[AIRFLOW-285] Use Airflow 2.0 style imports for all remaining hooks/operators</p></li> |
| <li><p>[AIRFLOW-40] Add LDAP group filtering feature.</p></li> |
| <li><p>[AIRFLOW-277] Multiple deletions does not work in Task Instances view if using SQLite backend</p></li> |
| <li><p>[AIRFLOW-200] Make hook/operator imports lazy, and print proper exceptions</p></li> |
| <li><p>[AIRFLOW-283] Make store_to_xcom_key a templated field in GoogleCloudStorageDownloadOperator</p></li> |
| <li><p>[AIRFLOW-278] Support utf-8 ecoding for SQL</p></li> |
| <li><p>[AIRFLOW-280] clean up tmp druid table no matter if an ingestion job succeeds or not</p></li> |
| <li><p>[AIRFLOW-274] Add XCom functionality to GoogleCloudStorageDownloadOperator</p></li> |
| <li><p>[AIRFLOW-273] Create an svg version of the airflow logo.</p></li> |
| <li><p>[AIRFLOW-275] Update contributing guidelines</p></li> |
| <li><p>[AIRFLOW-244] Modify hive operator to inject analysis data</p></li> |
| <li><p>[AIRFLOW-162] Allow variable to be accessible into templates</p></li> |
| <li><p>[AIRFLOW-248] Add Apache license header to all files</p></li> |
| <li><p>[AIRFLOW-263] Remove temp backtick file</p></li> |
| <li><p>[AIRFLOW-252] Raise Sqlite exceptions when deleting tasks instance in WebUI</p></li> |
| <li><p>[AIRFLOW-180] Fix timeout behavior for sensors</p></li> |
| <li><p>[AIRFLOW-262] Simplify commands in MANIFEST.in</p></li> |
| <li><p>[AIRFLOW-31] Add zope dependency</p></li> |
| <li><p>[AIRFLOW-6] Remove dependency on Highcharts</p></li> |
| <li><p>[AIRFLOW-234] make task that aren’t <cite>running</cite> self-terminate</p></li> |
| <li><p>[AIRFLOW-256] Fix test_scheduler_reschedule heartrate</p></li> |
| <li><p>Add Python 3 compatibility fix</p></li> |
| <li><p>[AIRFLOW-31] Use standard imports for hooks/operators</p></li> |
| <li><p>[AIRFLOW-173] Initial implementation of FileSensor</p></li> |
| <li><p>[AIRFLOW-224] Collect orphaned tasks and reschedule them</p></li> |
| <li><p>[AIRFLOW-239] Fix tests indentation</p></li> |
| <li><p>[AIRFLOW-225] Better units for task duration graph</p></li> |
| <li><p>[AIRFLOW-241] Add testing done section to PR template</p></li> |
| <li><p>[AIRFLOW-222] Show duration of task instances in ui</p></li> |
| <li><p>[AIRFLOW-231] Do not eval user input in PrestoHook</p></li> |
| <li><p>[AIRFLOW-216] Add Sqoop Hook and Operator</p></li> |
| <li><p>[AIRFLOW-171] Add upgrade notes on email and S3 to 1.7.1.2</p></li> |
| <li><p>[AIRFLOW-238] Make compatible with flask-admin 1.4.1</p></li> |
| <li><p>[AIRFLOW-230] [HiveServer2Hook] adding multi statements support</p></li> |
| <li><p>[AIRFLOW-142] setup_env.sh doesn’t download hive tarball if hdp is specified as distro</p></li> |
| <li><p>[AIRFLOW-223] Make parametrable the IP on which Flower binds to</p></li> |
| <li><p>[AIRFLOW-218] Added option to enable webserver gunicorn access/err logs</p></li> |
| <li><p>[AIRFLOW-213] Add “Closes #X” phrase to commit messages</p></li> |
| <li><p>[AIRFLOW-68] Align start_date with the schedule_interval</p></li> |
| <li><p>[AIRFLOW-9] Improving docs to meet Apache’s standards</p></li> |
| <li><p>[AIRFLOW-131] Make XCom.clear more selective</p></li> |
| <li><p>[AIRFLOW-214] Fix occasion of detached taskinstance</p></li> |
| <li><p>[AIRFLOW-206] Add commit to close PR</p></li> |
| <li><p>[AIRFLOW-206] Always load local log files if they exist</p></li> |
| <li><p>[AIRFLOW-211] Fix JIRA “resolve” vs “close” behavior</p></li> |
| <li><p>[AIRFLOW-64] Add note about relative DAGS_FOLDER</p></li> |
| <li><p>[AIRFLOW-114] Sort plugins dropdown</p></li> |
| <li><p>[AIRFLOW-209] Add scheduler tests and improve lineage handling</p></li> |
| <li><p>[AIRFLOW-207] Improve JIRA auth workflow</p></li> |
| <li><p>[AIRFLOW-187] Improve PR tool UX</p></li> |
| <li><p>[AIRFLOW-155] Documentation of Qubole Operator</p></li> |
| <li><p>Optimize and refactor process_dag</p></li> |
| <li><p>[AIRFLOW-185] Handle empty versions list</p></li> |
| <li><p>[AIRFLOW-201] Fix for HiveMetastoreHook + kerberos</p></li> |
| <li><p>[AIRFLOW-202]: Fixes stray print line</p></li> |
| <li><p>[AIRFLOW-196] Fix bug that exception is not handled in HttpSensor</p></li> |
| <li><p>[AIRFLOW-195] : Add toggle support to subdag clearing in the CLI</p></li> |
| <li><p>[AIRFLOW-23] Support for Google Cloud DataProc</p></li> |
| <li><p>[AIRFLOW-25] Configuration for Celery always required</p></li> |
| <li><p>[AIRFLOW-190] Add codecov and remove download count</p></li> |
| <li><p>[AIRFLOW-168] Correct evaluation of @once schedule</p></li> |
| <li><p>[AIRFLOW-183] Fetch log from remote when worker returns 4xx/5xx response</p></li> |
| <li><p>[AIRFLOW-181] Fix failing unpacking of hadoop by redownloading</p></li> |
| <li><p>[AIRFLOW-176] remove unused formatting key</p></li> |
| <li><p>[AIRFLOW-167]: Add dag_state option in cli</p></li> |
| <li><p>[AIRFLOW-178] Fix bug so that zip file is detected in DAG folder</p></li> |
| <li><p>[AIRFLOW-176] Improve PR Tool JIRA workflow</p></li> |
| <li><p>AIRFLOW-45: Support Hidden Airflow Variables</p></li> |
| <li><p>[AIRFLOW-175] Run git-reset before checkout in PR tool</p></li> |
| <li><p>[AIRFLOW-157] Make PR tool Py3-compat; add JIRA command</p></li> |
| <li><p>[AIRFLOW-170] Add missing @apply_defaults</p></li> |
| </ul> |
| </div> |
| <div class="section" id="airflow-1-7-1-2016-05-19"> |
| <h2>Airflow 1.7.1, 2016-05-19<a class="headerlink" href="#airflow-1-7-1-2016-05-19" title="Permalink to this headline">¶</a></h2> |
| <ul class="simple"> |
| <li><p>Fix : Don’t treat premature tasks as could_not_run tasks</p></li> |
| <li><p>AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/airflow#1485</p></li> |
| <li><p>Add logic to lock DB and avoid race condition</p></li> |
| <li><p>Handle queued tasks from multiple jobs/executors</p></li> |
| <li><p>AIRFLOW-52 Warn about overwriting tasks in a DAG</p></li> |
| <li><p>Fix corner case with joining processes/queues (#1473)</p></li> |
| <li><p>[AIRFLOW-52] Fix bottlenecks when working with many tasks</p></li> |
| <li><p>Add columns to toggle extra detail in the connection list view.</p></li> |
| <li><p>Log the number of errors when importing DAGs</p></li> |
| <li><p>Log dagbag metrics dupplicate messages in queue into Statsd (#1406)</p></li> |
| <li><p>Clean up issue template (#1419)</p></li> |
| <li><p>correct missed arg.foreground to arg.daemon in cli</p></li> |
| <li><p>Reinstate imports for github enterprise auth</p></li> |
| <li><p>Use os.execvp instead of subprocess.Popen for the webserver</p></li> |
| <li><p>Revert from using “–foreground” to “–daemon”</p></li> |
| <li><p>Implement a Cloudant hook</p></li> |
| <li><p>Add missing args to <cite>airflow clear</cite></p></li> |
| <li><p>Fixed a bug in the scheduler: num_runs used where runs intended</p></li> |
| <li><p>Add multiprocessing support to the scheduler</p></li> |
| <li><p>Partial fix to make sure next_run_date cannot be None</p></li> |
| <li><p>Support list/get/set variables in the CLI</p></li> |
| <li><p>Properly handle BigQuery booleans in BigQuery hook.</p></li> |
| <li><p>Added the ability to view XCom variables in webserver</p></li> |
| <li><p>Change DAG.tasks from a list to a dict</p></li> |
| <li><p>Add support for zipped dags</p></li> |
| <li><p>Stop creating hook on instantiating of S3 operator</p></li> |
| <li><p>User subquery in views to find running DAGs</p></li> |
| <li><p>Prevent DAGs from being reloaded on every scheduler iteration</p></li> |
| <li><p>Add a missing word to docs</p></li> |
| <li><p>Document the parameters of <cite>DbApiHook</cite></p></li> |
| <li><p>added oracle operator with existing oracle hook</p></li> |
| <li><p>Add PyOpenSSL to Google cloud gcp_api.</p></li> |
| <li><p>Remove executor error unit test</p></li> |
| <li><p>Add DAG inference, deferral, and context manager</p></li> |
| <li><p>Don’t return error when writing files to Google cloud storage.</p></li> |
| <li><p>Fix GCS logging for gcp_api.</p></li> |
| <li><p>Ensure attr is in scope for error message</p></li> |
| <li><p>Fixing misnamed PULL_REQUEST_TEMPLATE</p></li> |
| <li><p>Extract non_pooled_task_slot_count into a configuration param</p></li> |
| <li><p>Update plugins.rst for clarity on the example (#1309)</p></li> |
| <li><p>Fix s3 logging issue</p></li> |
| <li><p>Add twitter feed example dag</p></li> |
| <li><p>Github ISSUE_TEMPLATE & PR_TEMPLATE cleanup</p></li> |
| <li><p>Reduce logger verbosity</p></li> |
| <li><p>Adding a PR Template</p></li> |
| <li><p>Add Lucid to list of users</p></li> |
| <li><p>Fix usage of asciiart</p></li> |
| <li><p>Use session instead of outdated main_session for are_dependencies_met</p></li> |
| <li><p>Fix celery flower port allocation</p></li> |
| <li><p>Fix for missing edit actions due to flask-admin upgrade</p></li> |
| <li><p>Fix typo in comment in prioritize_queued method</p></li> |
| <li><p>Add HipchatOperator</p></li> |
| <li><p>Include all example dags in backfill unit test</p></li> |
| <li><p>Make sure skipped jobs are actually skipped</p></li> |
| <li><p>Fixing a broken example dag, example_skip_dag.py</p></li> |
| <li><p>Add consistent and thorough signal handling and logging</p></li> |
| <li><p>Allow Operators to specify SKIPPED status internally</p></li> |
| <li><p>Update docstring for executor trap unit test</p></li> |
| <li><p>Doc: explain the usage of Jinja templating for templated params</p></li> |
| <li><p>Don’t schedule runs before the DAG’s start_date</p></li> |
| <li><p>Fix infinite retries with pools, with test</p></li> |
| <li><p>Fix handling of deadlocked jobs</p></li> |
| <li><p>Show only Airflow’s deprecation warnings</p></li> |
| <li><p>Set DAG_FOLDER for unit tests</p></li> |
| <li><p>Missing comma in setup.py</p></li> |
| <li><p>Deprecate args and kwargs in BaseOperator</p></li> |
| <li><p>Raise deep scheduler exceptions to force a process restart.</p></li> |
| <li><p>Change inconsistent example DAG owners</p></li> |
| <li><p>Fix module path of send_email_smtp in configuration</p></li> |
| <li><p>added Gentner Lab to list of users</p></li> |
| <li><p>Increase timeout time for unit test</p></li> |
| <li><p>Fix reading strings from conf</p></li> |
| <li><p>CHORE - Remove Trailing Spaces</p></li> |
| <li><p>Fix SSHExecuteOperator crash when using a custom ssh port</p></li> |
| <li><p>Add note about airflow components to template</p></li> |
| <li><p>Rewrite BackfillJob logic for clarity</p></li> |
| <li><p>Add unit tests</p></li> |
| <li><p>Fix miscellaneous bugs and clean up code</p></li> |
| <li><p>Fix logic for determining DagRun states</p></li> |
| <li><p>Make SchedulerJob not run EVERY queued task</p></li> |
| <li><p>Improve BackfillJob handling of queued/deadlocked tasks</p></li> |
| <li><p>Introduce ignore_depends_on_past parameters</p></li> |
| <li><p>Use Popen with CeleryExecutor</p></li> |
| <li><p>Rename user table to users to avoid conflict with postgres</p></li> |
| <li><p>Beware of negative pool slots.</p></li> |
| <li><p>Add support for calling_format from boto to S3_Hook</p></li> |
| <li><p>Add pypi meta data and sync version number</p></li> |
| <li><p>Set dags_are_paused_at_creation’s default value to True</p></li> |
| <li><p>Resurface S3Log class eaten by rebase/push -f</p></li> |
| <li><p>Add missing session.commit() at end of initdb</p></li> |
| <li><p>Validate that subdag tasks have pool slots available, and test</p></li> |
| <li><p>Use urlparse for remote GCS logs, and add unit tests</p></li> |
| <li><p>Make webserver worker timeout configurable</p></li> |
| <li><p>Fixed scheduling for @once interval</p></li> |
| <li><p>Use psycopg2’s API for serializing postgres cell values</p></li> |
| <li><p>Make the provide_session decorator more robust</p></li> |
| <li><p>update link to Lyft’s website</p></li> |
| <li><p>use num_shards instead of partitions to be consistent with batch ingestion</p></li> |
| <li><p>Add documentation links to README</p></li> |
| <li><p>Update docs with separate configuration section</p></li> |
| <li><p>Fix airflow.utils deprecation warning code being Python 3 incompatible</p></li> |
| <li><p>Extract dbapi cell serialization into its own method</p></li> |
| <li><p>Set Postgres autocommit as supported only if server version is < 7.4</p></li> |
| <li><p>Use refactored utils module in unit test imports</p></li> |
| <li><p>Add changelog for 1.7.0</p></li> |
| <li><p>Use LocalExecutor on Travis if possible</p></li> |
| <li><p>remove unused logging,errno, MiniHiveCluster imports</p></li> |
| <li><p>remove extra import of logging lib</p></li> |
| <li><p>Fix required gcloud version</p></li> |
| <li><p>Refactoring utils into smaller submodules</p></li> |
| <li><p>Properly measure number of task retry attempts</p></li> |
| <li><p>Add function to get configuration as dict, plus unit tests</p></li> |
| <li><p>Merge branch ‘master’ into hivemeta_sasl</p></li> |
| <li><p>Add wiki link to README.md</p></li> |
| <li><p>[hotfix] make email.Utils > email.utils for py3</p></li> |
| <li><p>Add the missing “Date” header to the warning e-mails</p></li> |
| <li><p>Add the missing “Date” header to the warning e-mails</p></li> |
| <li><p>Check name of SubDag class instead of class itself</p></li> |
| <li><p>[hotfix] removing repo_token from .coveralls.yml</p></li> |
| <li><p>Set the service_name in coverals.yml</p></li> |
| <li><p>Fixes #1223</p></li> |
| <li><p>Update Airflow docs for remote logging</p></li> |
| <li><p>Add unit tests for trapping Executor errors</p></li> |
| <li><p>Make sure Executors properly trap errors</p></li> |
| <li><p>Fix HttpOpSensorTest to use fake resquest session</p></li> |
| <li><p>Linting</p></li> |
| <li><p>Add an example on pool usage in the documentation</p></li> |
| <li><p>Add two methods to bigquery hook’s base cursor: run_table_upsert, which adds a table or updates an existing table; and run_grant_dataset_view_access, which grants view access to a given dataset for a given table.</p></li> |
| <li><p>Tasks references upstream and downstream tasks using strings instead of references</p></li> |
| <li><p>Fix typos in models.py</p></li> |
| <li><p>Fix broken links in documentation</p></li> |
| <li><p>[hotfix] fixing the Scheduler CLI to make dag_id optional</p></li> |
| <li><p>Update link to Common Pitfalls wiki page in README</p></li> |
| <li><p>Allow disabling periodic committing when inserting rows with DbApiHook</p></li> |
| <li><p>added Glassdoor to “who uses airflow”</p></li> |
| <li><p>Fix typo preventing from launching webserver</p></li> |
| <li><p>Documentation badge</p></li> |
| <li><p>Fixing ISSUE_TEMPLATE name to include .md suffix</p></li> |
| <li><p>Adding an ISSUE_TEMPLATE to ensure that issues are adequately defined</p></li> |
| <li><p>Linting & debugging</p></li> |
| <li><p>Refactoring the CLI to be data-driven</p></li> |
| <li><p>Updating the Bug Reporting protocol in the Contributing.md file</p></li> |
| <li><p>Fixing the docs</p></li> |
| <li><p>clean up references to old session</p></li> |
| <li><p>remove session reference</p></li> |
| <li><p>resolve conflict</p></li> |
| <li><p>clear xcom data when task instance starts</p></li> |
| <li><p>replace main_session with @provide_session</p></li> |
| <li><p>Add extras to installation.rst</p></li> |
| <li><p>Changes to Contributing to reflect more closely the current state of development.</p></li> |
| <li><p>Modifying README to link to the wiki committer list</p></li> |
| <li><p>docs: fixes a spelling mistake in default config</p></li> |
| <li><p>Set killMode to ‘control-group’ for webservice.service</p></li> |
| <li><p>Set KillMode to ‘control-group’ for worker.service</p></li> |
| <li><p>Linting</p></li> |
| <li><p>Fix WebHdfsSensor</p></li> |
| <li><p>Adding more licenses to pass checks</p></li> |
| <li><p>fixing landscape’s config</p></li> |
| <li><p>[hotfix] typo that made it in master</p></li> |
| <li><p>[hotfix] fixing landscape requirement detection</p></li> |
| <li><p>Make testing on hive conditional</p></li> |
| <li><p>Merge remote-tracking branch ‘upstream/master’ into minicluster</p></li> |
| <li><p>Update README.md</p></li> |
| <li><p>Throwing in a few license to pass the build</p></li> |
| <li><p>Adding a reqs.txt for landscape.io</p></li> |
| <li><p>Pointing to a reqs file</p></li> |
| <li><p>Some linting</p></li> |
| <li><p>Adding a .landscape.yml file</p></li> |
| <li><p>badge for pypi version</p></li> |
| <li><p>Add license and ignore for sql and csv</p></li> |
| <li><p>Use correct connection id</p></li> |
| <li><p>Use correct table name</p></li> |
| <li><p>Provide data for ci tests</p></li> |
| <li><p>new badge for showing staleness of reqs</p></li> |
| <li><p>removing requirements.txt as it is uni-dimensional</p></li> |
| <li><p>Make it work on py3</p></li> |
| <li><p>Remove decode for logging</p></li> |
| <li><p>Also keep py2 compatible</p></li> |
| <li><p>More py3 fixes</p></li> |
| <li><p>Convert to bytes for py3 compat</p></li> |
| <li><p>Make sure to be py3 compatible</p></li> |
| <li><p>Use unicodecsv to make it py3 compatible</p></li> |
| <li><p>Replace tab with spaces Remove unused import</p></li> |
| <li><p>Merge remote-tracking branch ‘upstream/master’</p></li> |
| <li><p>Support decimal types in MySQL to GCS</p></li> |
| <li><p>Make sure to write binary as string can be unicode</p></li> |
| <li><p>Ignore metastore</p></li> |
| <li><p>More impyla fixes</p></li> |
| <li><p>Test HivemetaStore if python 2</p></li> |
| <li><p>Allow users to set hdfs_namenode_principal in HDFSHook config</p></li> |
| <li><p>Add tests for Hiveserver2 and fix some issues from impyla</p></li> |
| <li><p>Merge branch ‘impyla’ into minicluster</p></li> |
| <li><p>This patch allows for testing of hive operators and hooks. Sasl is used (NoSasl in connection string is not possible). Tests have been adjusted.</p></li> |
| <li><p>Treat SKIPPED and SUCCESS the same way when evaluating depends_on_past=True</p></li> |
| <li><p>fix bigquery hook</p></li> |
| <li><p>version cap for gcp_api</p></li> |
| <li><p>Fix typo when returning VerticaHook</p></li> |
| <li><p>Adding fernet key to use it as part of stdout commands</p></li> |
| <li><p>Adding support for ssl parameters. (picking up from jthomas123)</p></li> |
| <li><p>more detail in error message.</p></li> |
| <li><p>make sure paths don’t conflict bc of trailing /</p></li> |
| <li><p>change gcs_hook to self.hook</p></li> |
| <li><p>refactor remote log read/write and add GCS support</p></li> |
| <li><p>Only use multipart upload in S3Hook if file is large enough</p></li> |
| <li><p>Merge branch ‘airbnb/master’</p></li> |
| <li><p>Add GSSAPI SASL to HiveMetaStoreHook.</p></li> |
| <li><p>Add warning for deprecated setting</p></li> |
| <li><p>Use kerberos_service_name = ‘hive’ as standard instead of ‘impala’.</p></li> |
| <li><p>Use GSSAPI instead of KERBEROS and provide backwards compatibility</p></li> |
| <li><p>ISSUE-1123 Use impyla instead of pyhs2</p></li> |
| <li><p>set celery_executor to use queue name as exchange</p></li> |
| </ul> |
| </div> |
| </div> |
| |
| |
| </div> |
| |
| </div> |
| <footer> |
| |
| <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation"> |
| |
| <a href="faq.html" class="btn btn-neutral float-right" title="FAQ" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a> |
| |
| |
| <a href="lineage.html" class="btn btn-neutral float-left" title="Lineage" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a> |
| |
| </div> |
| |
| |
| <hr/> |
| |
| <div role="contentinfo"> |
| <p> |
| |
| </p> |
| </div> |
| Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. |
| |
| </footer> |
| |
| </div> |
| </div> |
| |
| </section> |
| |
| </div> |
| |
| |
| |
| <script type="text/javascript"> |
| jQuery(function () { |
| SphinxRtdTheme.Navigation.enable(true); |
| }); |
| </script> |
| |
| |
| |
| |
| |
| |
| </body> |
| </html> |
