| AIRFLOW 1.8.2rc1, 2017-06-05 |
| ------------------------- |
| |
| 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing Booleans |
| 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks |
| 93825d5 Re-enable caching for hadoop components |
| 33a9dcb Pin Hive and Hadoop to a specific version and create writable warehouse dir |
| 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask |
| c6a09c4 Updating CHANGELOG for 1.8.2rc1 |
| 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute |
| 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF requirements |
| e10af9a [AIRFLOW-XXX] Set version to 1.8.2rc1 |
| 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos |
| 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2 templates |
| 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[] |
| b4e9eb8 [AIRFLOW-1089] Add Spark application arguments |
| a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions |
| 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits |
| a2c18a5 [AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView |
| d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly |
| b4ee88a [AIRFLOW-1036] Randomize exponential backoff |
| 9fca409 [AIRFLOW-993] Update date inference logic |
| 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification time |
| c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change |
| acd0166 [AIRFLOW-1263] Dynamic height for charts |
| 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis |
| fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow' |
| 2e9eee3 [AIRFLOW-1282] Fix known event column sorting |
| 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun |
| bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator |
| 57d5bcd [AIRFLOW-1281] Sort variables by key field by default |
| 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name |
| 1232b6a [AIRFLOW-1243] DAGs table has no default entries to show |
| b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view |
| c406652 [AIRFLOW-1226] Remove empty column on the Jobs view |
| 51a83cc [AIRFLOW-1199] Fix create modal |
| cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key |
| 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date |
| f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before set to True If we're looking for the closest date before, we should take the latest date in the list of date before. |
| 93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection |
| bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately |
| 093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[] |
| 777f181 [AIRFLOW-1168] Add closing() to all connections and cursors |
| bc8e912 [AIRFLOW-XXX] Updating CHANGELOG, README, and UPDATING after 1.8.1 release |
| |
| AIRFLOW 1.8.1, 2017-05-09 |
| ------------------------- |
| |
| [AIRFLOW-1160] Update Spark parameters for Mesos |
| [AIRFLOW 1149] Allow for custom filters in Jinja2 templates |
| [AIRFLOW-1119] Fix unload query so headers are on first row[] |
| [AIRFLOW-1089] Add Spark application arguments |
| [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions |
| [AIRFLOW-1074] Don't count queued tasks for concurrency limits |
| [AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView |
| [AIRFLOW-1038] Specify celery serialization options explicitly |
| [AIRFLOW-1036] Randomize exponential backoff |
| [AIRFLOW-993] Update date inference logic |
| [AIRFLOW-1167] Support microseconds in FTPHook modification time |
| [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change |
| [AIRFLOW-1263] Dynamic height for charts |
| [AIRFLOW-1266] Increase width of gantt y axis |
| [AIRFLOW-1290] set docs author to 'Apache Airflow' |
| [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun |
| [AIRFLOW-1192] Some enhancements to qubole_operator |
| [AIRFLOW-1281] Sort variables by key field by default |
| [AIRFLOW-1244] Forbid creation of a pool with empty name |
| [AIRFLOW-1243] DAGs table has no default entries to show |
| [AIRFLOW-1227] Remove empty column on the Logs view |
| [AIRFLOW-1226] Remove empty column on the Jobs view |
| [AIRFLOW-1199] Fix create modal |
| [AIRFLOW-1200] Forbid creation of a variable with an empty key |
| [AIRFLOW-1186] Sort dag.get_task_instances by execution_date |
| [AIRFLOW-1145] Fix closest_date_partition function with before set to True If we're looking for the closest date before, we should take the latest date in the list of date before. |
| [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection |
| [AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately |
| [AIRFLOW-1150] Fix scripts execution in sparksql hook[] |
| [AIRFLOW-1168] Add closing() to all connections and cursors |
| >>>>>>> Stashed changes |
| |
| AIRFLOW 1.8.0, 2017-03-12 |
| ------------------------- |
| |
| [AIRFLOW-900] Double trigger should not kill original task instance |
| [AIRFLOW-900] Fixes bugs in LocalTaskJob for double run protection |
| [AIRFLOW-932] Do not mark tasks removed when backfilling |
| [AIRFLOW-961] run onkill when SIGTERMed |
| [AIRFLOW-910] Use parallel task execution for backfills |
| [AIRFLOW-967] Wrap strings in native for py2 ldap compatibility |
| [AIRFLOW-941] Use defined parameters for psycopg2 |
| [AIRFLOW-719] Prevent DAGs from ending prematurely |
| [AIRFLOW-938] Use test for True in task_stats queries |
| [AIRFLOW-937] Improve performance of task_stats |
| [AIRFLOW-933] use ast.literal_eval rather eval because ast.literal_eval does not execute input. |
| [AIRFLOW-925] Revert airflow.hooks change that cherry-pick picked |
| [AIRFLOW-919] Running tasks with no start date shouldn't break a DAGs UI |
| [AIRFLOW-802] Add spark-submit operator/hook |
| [AIRFLOW-897] Prevent dagruns from failing with unfinished tasks |
| [AIRFLOW-861] make pickle_info endpoint be login_required |
| [AIRFLOW-853] use utf8 encoding for stdout line decode |
| [AIRFLOW-856] Make sure execution date is set for local client |
| [AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log verbosity |
| [AIRFLOW-831] Restore import to fix broken tests |
| [AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN exclusively from settings |
| [AIRFLOW-694] Fix config behaviour for empty envvar |
| [AIRFLOW-365] Set dag.fileloc explicitly and use for Code view |
| [AIRFLOW-931] Do not set QUEUED in TaskInstances |
| [AIRFLOW-899] Tasks in SCHEDULED state should be white in the UI instead of black |
| [AIRFLOW-895] Address Apache release incompliancies |
| [AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a dagrun has no start date |
| [AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer |
| [AIRFLOW-863] Example DAGs should have recent start dates |
| [AIRFLOW-869] Refactor mark success functionality |
| [AIRFLOW-856] Make sure execution date is set for local client |
| [AIRFLOW-814] Fix Presto*CheckOperator.__init__ |
| [AIRFLOW-844] Fix cgroups directory creation |
| [AIRFLOW-816] Use static nvd3 and d3 |
| [AIRFLOW-821] Fix py3 compatibility |
| [AIRFLOW-817] Check for None value of execution_date in endpoint |
| [AIRFLOW-822] Close db before exception |
| [AIRFLOW-815] Add prev/next execution dates to template variables |
| [AIRFLOW-813] Fix unterminated unit tests in SchedulerJobTest |
| [AIRFLOW-813] Fix unterminated scheduler unit tests |
| [AIRFLOW-806] UI should properly ignore DAG doc when it is None |
| [AIRFLOW-812] Fix the scheduler termination bug. |
| [AIRFLOW-780] Fix dag import errors no longer working |
| [AIRFLOW-783] Fix py3 incompatibility in BaseTaskRunner |
| [AIRFLOW-810] Correct down_revision dag_id/state index creation |
| [AIRFLOW-807] Improve scheduler performance for large DAGs |
| [AIRFLOW-798] Check return_code before forcing termination |
| [AIRFLOW-139] Let psycopg2 handle autocommit for PostgresHook |
| [AIRFLOW-776] Add missing cgroups devel dependency |
| [AIRFLOW-777] Fix expression to check if a DagRun is in running state |
| [AIRFLOW-785] Don't import CgroupTaskRunner at global scope |
| [AIRFLOW-784] Pin funcsigs to 1.0.0 |
| [AIRFLOW-624] Fix setup.py to not import airflow.version as version |
| [AIRFLOW-779] Task should fail with specific message when deleted |
| [AIRFLOW-778] Fix completey broken MetastorePartitionSensor |
| [AIRFLOW-739] Set pickle_info log to debug |
| [AIRFLOW-771] Make S3 logs append instead of clobber |
| [AIRFLOW-773] Fix flaky datetime addition in api test |
| [AIRFLOW-219][AIRFLOW-398] Cgroups + impersonation |
| [AIRFLOW-683] Add jira hook, operator and sensor |
| [AIRFLOW-762] Add Google DataProc delete operator |
| [AIRFLOW-760] Update systemd config |
| [AIRFLOW-759] Use previous dag_run to verify depend_on_past |
| [AIRFLOW-757] Set child_process_log_directory default more sensible |
| [AIRFLOW-692] Open XCom page to super-admins only |
| [AIRFLOW-737] Fix HDFS Sensor directory. |
| [AIRFLOW-747] Fix retry_delay not honoured |
| [AIRFLOW-558] Add Support for dag.catchup=(True|False) Option |
| [AIRFLOW-489] Allow specifying execution date in trigger_dag API |
| [AIRFLOW-738] Commit deleted xcom items before insert |
| [AIRFLOW-729] Add Google Cloud Dataproc cluster creation operator |
| [AIRFLOW-728] Add Google BigQuery table sensor |
| [AIRFLOW-741] Log to debug instead of info for app.py |
| [AIRFLOW-731] Fix period bug for NamedHivePartitionSensor |
| [AIRFLOW-740] Pin jinja2 to < 2.9.0 |
| [AIRFLOW-663] Improve time units for task performance charts |
| [AIRFLOW-665] Fix email attachments |
| [AIRFLOW-734] Fix SMTP auth regression when not using user/pass |
| [AIRFLOW-702] Fix LDAP Regex Bug |
| [AIRFLOW-717] Add Cloud Storage updated sensor |
| [AIRFLOW-695] Retries do not execute because dagrun is in FAILED state |
| [AIRFLOW-673] Add operational metrics test for SchedulerJob |
| [AIRFLOW-727] try_number is not increased |
| [AIRFLOW-715] A more efficient HDFS Sensor: |
| [AIRFLOW-716] Allow AVRO BigQuery load-job without schema |
| [AIRFLOW-718] Allow the query URI for DataProc Pig |
| Log needs to be part of try/catch block |
| [AIRFLOW-721] Descendant process can disappear before termination |
| [AIRFLOW-403] Bash operator's kill method leaves underlying processes running |
| [AIRFLOW-657] Add AutoCommit Parameter for MSSQL |
| [AIRFLOW-641] Improve pull request instructions |
| [AIRFLOW-685] Add test for MySqlHook.bulk_load() |
| [AIRFLOW-686] Match auth backend config section |
| [AIRFLOW-691] Add SSH KeepAlive option to SSH_hook |
| [AIRFLOW-709] Use same engine for migrations and reflection |
| [AIRFLOW-700] Update to reference to web authentication documentation |
| [AIRFLOW-649] Support non-sched DAGs in LatestOnlyOp |
| [AIRFLOW-712] Fix AIRFLOW-667 to use proper HTTP error properties |
| [AIRFLOW-710] Add OneFineStay as official user |
| [AIRFLOW-703][AIRFLOW-1] Stop Xcom being cleared too early |
| [AIRFLOW-679] Stop concurrent task instances from running |
| [AIRFLOW-704][AIRFLOW-1] Fix invalid syntax in BQ hook |
| [AIRFLOW-667] Handle BigQuery 503 error |
| [AIRFLOW-680] Disable connection pool for commands |
| [AIRFLOW-678] Prevent scheduler from double triggering TIs |
| [AIRFLOW-677] Kill task if it fails to heartbeat |
| [AIRFLOW-674] Ability to add descriptions for DAGs |
| [AIRFLOW-682] Bump MAX_PERIODS to make mark_success work for large DAGs |
| Use jdk selector to set required jdk |
| [AIRFLOW-647] Restore dag.get_active_runs |
| [AIRFLOW-662] Change seasons to months in project description |
| [AIRFLOW-656] Add dag/task/date index to xcom table |
| [AIRFLOW-658] Improve schema_update_options in GCP |
| [AIRFLOW-41] Fix pool oversubscription |
| [AIRFLOW-489] Add API Framework |
| [AIRFLOW-653] Add some missing endpoint tests |
| [AIRFLOW-652] Remove obsolete endpoint |
| [AIRFLOW-345] Add contrib ECSOperator |
| [AIRFLOW-650] Adding Celect to user list |
| [AIRFLOW-510] Filter Paused Dags, show Last Run & Trigger Dag |
| [AIRFLOW-643] Improve date handling for sf_hook |
| [AIRFLOW-638] Add schema_update_options to GCP ops |
| [AIRFLOW-640] Install and enable nose-ignore-docstring |
| [AIRFLOW-639]AIRFLOW-639] Alphasort package names |
| [AIRFLOW-375] Fix pylint errors |
| [AIRFLOW-347] Show empty DAG runs in tree view |
| [AIRFLOW-628] Adding SalesforceHook to contrib/hooks |
| [AIRFLOW-514] hive hook loads data from pandas DataFrame into hive and infers types |
| [AIRFLOW-565] Fixes DockerOperator on Python3.x |
| [AIRFLOW-635] Encryption option for S3 hook |
| [AIRFLOW-137] Fix max_active_runs on clearing tasks |
| [AIRFLOW-343] Fix schema plumbing in HiveServer2Hook |
| [AIRFLOW-130] Fix ssh operator macosx |
| [AIRFLOW-633] Show TI attributes in TI view |
| [AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment |
| [AIRFLOW-533] Set autocommit via set_autocommit |
| [AIRFLOW-629] stop pinning lxml |
| [AIRFLOW-464] Add setdefault method to Variable |
| [AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment |
| [AIRFLOW-591] Add datadog hook & sensor |
| [AIRFLOW-561] Add RedshiftToS3Transfer operator |
| [AIRFLOW-570] Pass root to date form on gantt |
| [AIRFLOW-504] Store fractional seconds in MySQL tables |
| [AIRFLOW-623] LDAP attributes not always a list |
| [AIRFLOW-611] source_format in BigQueryBaseCursor |
| [AIRFLOW-619] Fix exception in Gannt chart |
| [AIRFLOW-618] Cast DateTimes to avoid sqllite errors |
| [AIRFLOW-422] Add JSON endpoint for task info |
| [AIRFLOW-616][AIRFLOW-617] Minor fixes to PR tool UX |
| [AIRFLOW-179] Fix DbApiHook with non-ASCII chars |
| [AIRFLOW-566] Add timeout while fetching logs |
| [AIRFLOW-615] Set graph glyphicon first |
| [AIRFLOW-609] Add application_name to PostgresHook |
| [AIRFLOW-604] Revert .first() to .one() |
| [AIRFLOW-370] Create AirflowConfigException in exceptions.py |
| [AIRFLOW-582] Fixes TI.get_dagrun filter (removes start_date) |
| [AIRFLOW-568] Fix double task_stats count if a DagRun is active |
| [AIRFLOW-585] Fix race condition in backfill execution loop |
| [AIRFLOW-580] Prevent landscape warning on .format |
| [AIRFLOW-597] Check if content is None, not false-equivalent |
| [AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m. |
| [AIRFLOW-453] Add XCom Admin Page |
| [AIRFLOW-588] Add Google Cloud Storage Object sensor[] |
| [AIRFLOW-592] example_xcom import Error |
| [AIRFLOW-587] Fix incorrect scope for Google Auth[] |
| [AIRFLOW-589] Add templatable job_name[] |
| [AIRFLOW-227] Show running config in config view |
| [AIRFLOW-319]AIRFLOW-319] xcom push response in HTTP Operator |
| [AIRFLOW-385] Add symlink to latest scheduler log directory |
| [AIRFLOW-583] Fix decode error in gcs_to_bq |
| [AIRFLOW-96] s3_conn_id using environment variable |
| [AIRFLOW-575] Clarify tutorial and FAQ about `schedule_interval` always inheriting from DAG object |
| [AIRFLOW-577] Output BigQuery job for improved debugging |
| [AIRFLOW-560] Get URI & SQLA engine from Connection |
| [AIRFLOW-518] Require DataProfilingMixin for Variables CRUD |
| [AIRFLOW-553] Fix load path for filters.js |
| [AIRFLOW-554] Add Jinja support to Spark-sql |
| [AIRFLOW-550] Make ssl config check empty string safe |
| [AIRFLOW-500] Use id for github allowed teams |
| [AIRFLOW-556] Add UI PR guidelines |
| [AIRFLOW-358][AIRFLOW-430] Add `connections` cli |
| [AIRFLOW-548] Load DAGs immediately & continually |
| [AIRFLOW-539] Updated BQ hook and BQ operator to support Standard SQL. |
| [AIRFLOW-378] Add string casting to params of spark-sql operator |
| [AIRFLOW-544] Add Pause/Resume toggle button |
| [AIRFLOW-333][AIRFLOW-258] Fix non-module plugin components |
| [AIRFLOW-542] Add tooltip to DAGs links icons |
| [AIRFLOW-530] Update docs to reflect connection environment var has to be in uppercase |
| [AIRFLOW-525] Update template_fields in Qubole Op |
| [AIRFLOW-480] Support binary file download from GCS |
| [AIRFLOW-198] Implement latest_only_operator |
| [AIRFLOW-91] Add SSL config option for the webserver |
| [AIRFLOW-191] Fix connection leak with PostgreSQL backend |
| [AIRFLOW-512] Fix 'bellow' typo in docs & comments |
| [AIRFLOW-509][AIRFLOW-1] Create operator to delete tables in BigQuery |
| [AIRFLOW-498] Remove hard-coded gcp project id |
| [AIRFLOW-505] Support unicode characters in authors' names |
| [AIRFLOW-494] Add per-operator success/failure metrics |
| [AIRFLOW-488] Fix test_simple fail |
| [AIRFLOW-468] Update Panda requirement to 0.17.1 |
| [AIRFLOW-159] Add cloud integration section + GCP documentation |
| [AIRFLOW-477][AIRFLOW-478] Restructure security section for clarity |
| [AIRFLOW-467] Allow defining of project_id in BigQueryHook |
| [AIRFLOW-483] Change print to logging statement |
| [AIRFLOW-475] make the segment granularity in Druid hook configurable |
| |
| |
| AIRFLOW 1.7.2 |
| ------------- |
| |
| [AIRFLOW-463] Link Airflow icon to landing page |
| [AIRFLOW-149] Task Dependency Engine + Why Isn't My Task Running View |
| [AIRFLOW-361] Add default failure handler for the Qubole Operator |
| [AIRFLOW-353] Fix dag run status update failure |
| [AIRFLOW-447] Store source URIs in Python 3 compatible list |
| [AIRFLOW-443] Make module names unique when importing |
| [AIRFLOW-444] Add Google authentication backend |
| [AIRFLOW-446][AIRFLOW-445] Adds missing dataproc submit options |
| [AIRFLOW-431] Add CLI for CRUD operations on pools |
| [AIRFLOW-329] Update Dag Overview Page with Better Status Columns |
| [AIRFLOW-360] Fix style warnings in models.py |
| [AIRFLOW-425] Add white fill for null state tasks in tree view. |
| [AIRFLOW-69] Use dag runs in backfill jobs |
| [AIRFLOW-415] Make dag_id not found error clearer |
| [AIRFLOW-416] Use ordinals in README's company list |
| [AIRFLOW-369] Allow setting default DAG orientation |
| [AIRFLOW-410] Add 2 Q/A to the FAQ in the docs |
| [AIRFLOW-407] Add different colors for some sensors |
| [AIRFLOW-414] Improve error message for missing FERNET_KEY |
| [AIRFLOW-406] Sphinx/rst fixes |
| [AIRFLOW-412] Fix lxml dependency |
| [AIRFLOW-413] Fix unset path bug when backfilling via pickle |
| [AIRFLOW-78] Airflow clear leaves dag_runs |
| [AIRFLOW-402] Remove NamedHivePartitionSensor static check, add docs |
| [AIRFLOW-394] Add an option to the Task Duration graph to show cumulative times |
| [AIRFLOW-404] Retry download if unpacking fails for hive |
| [AIRFLOW-276] Gunicorn rolling restart |
| [AIRFLOW-399] Remove dags/testdruid.py |
| [AIRFLOW-400] models.py/DAG.set_dag_runs_state() does not correctly set state |
| [AIRFLOW-395] Fix colon/equal signs typo for resources in default config |
| [AIRFLOW-397] Documentation: Fix typo "instatiating" to "instantiating" |
| [AIRFLOW-395] Remove trailing commas from resources in config |
| [AIRFLOW-388] Add a new chart for Task_Tries for each DAG |
| [AIRFLOW-322] Fix typo in FAQ section |
| [AIRFLOW-375] Pylint fixes |
| limit scope to user email only AIRFLOW-386 |
| [AIRFLOW-383] Cleanup example qubole operator dag |
| [AIRFLOW-160] Parse DAG files through child processes |
| [AIRFLOW-381] Manual UI Dag Run creation: require dag_id field |
| [AIRFLOW-373] Enhance CLI variables functionality |
| [AIRFLOW-379] Enhance Variables page functionality: import/export variables |
| [AIRFLOW-331] modify the LDAP authentication config lines in 'Security' sample codes |
| [AIRFLOW-356][AIRFLOW-355][AIRFLOW-354] Replace nobr, enable DAG only exists locally message, change edit DAG icon |
| [AIRFLOW-362] Import __future__ division |
| [AIRFLOW-359] Pin flask-login to 0.2.11 |
| [AIRFLOW-261] Add bcc and cc fields to EmailOperator |
| [AIRFLOW-348] Fix code style warnings |
| [AIRFLOW-349] Add metric for number of zombies killed |
| [AIRFLOW-340] Remove unused dependency on Babel |
| [AIRFLOW-339]: Ability to pass a flower conf file |
| [AIRFLOW-341][operators] Add resource requirement attributes to operators |
| [AIRFLOW-335] Fix simple style errors/warnings |
| [AIRFLOW-337] Add __repr__ to VariableAccessor and VariableJsonAccessor |
| [AIRFLOW-334] Fix using undefined variable |
| [AIRFLOW-315] Fix blank lines code style warnings |
| [AIRFLOW-306] Add Spark-sql Hook and Operator |
| [AIRFLOW-327] Add rename method to the FTPHook |
| [AIRFLOW-321] Fix a wrong code example about tests/dags |
| [AIRFLOW-316] Always check DB state for Backfill Job execution |
| [AIRFLOW-264] Adding workload management for Hive |
| [AIRFLOW-297] support exponential backoff option for retry delay |
| [AIRFLOW-31][AIRFLOW-200] Add note to updating.md |
| [AIRFLOW-307] There is no __neq__ python magic method. |
| [AIRFLOW-309] Add requirements of develop dependencies to docs |
| [AIRFLOW-307] Rename __neq__ to __ne__ python magic method. |
| [AIRFLOW-313] Fix code style for sqoop_hook.py |
| [AIRFLOW-311] Fix wrong path in CONTRIBUTING.md |
| [AIRFLOW-24] DataFlow Java Operator |
| [AIRFLOW-308] Add link to refresh DAG within DAG view header |
| [AIRFLOW-314] Fix BigQuery cursor run_table_upsert method |
| [AIRFLOW-298] fix incubator diclaimer in docs |
| [AIRFLOW-284] HiveServer2Hook fix for cursor scope for get_results |
| [AIRFLOW-260] More graceful exit when issues can't be closed |
| [AIRFLOW-260] Handle case when no version is found |
| [AIRFLOW-228] Handle empty version list in PR tool |
| [AIRFLOW-302] Improve default squash commit message |
| [AIRFLOW-187] Improve prompt styling |
| [AIRFLOW-187] Fix typo in argument name |
| [AIRFLOW-187] Move "Close XXX" message to end of squash commit |
| [AIRFLOW-247] Add EMR hook, operators and sensors. Add AWS base hook |
| [AIRFLOW-301] Fix broken unit test |
| [AIRFLOW-100] Add execution_date_fn to ExternalTaskSensor |
| [AIRFLOW-282] Remove PR Tool logic that depends on version formatting |
| [AIRFLOW-291] Add index for state in TI table |
| [AIRFLOW-269] Add some unit tests for PostgreSQL |
| [AIRFLOW-296] template_ext is being treated as a string rather than a tuple in qubole operator |
| [AIRFLOW-286] Improve FTPHook to implement context manager interface |
| [AIRFLOW-243] Create NamedHivePartitionSensor |
| [AIRFLOW-246] Improve dag_stats endpoint query |
| [AIRFLOW-189] Highlighting of Parent/Child nodes in Graphs |
| [ARFLOW-255] Check dagrun timeout when comparing active runs |
| [AIRFLOW-281] Add port to mssql_hook |
| [AIRFLOW-285] Use Airflow 2.0 style imports for all remaining hooks/operators |
| [AIRFLOW-40] Add LDAP group filtering feature. |
| [AIRFLOW-277] Multiple deletions does not work in Task Instances view if using SQLite backend |
| [AIRFLOW-200] Make hook/operator imports lazy, and print proper exceptions |
| [AIRFLOW-283] Make store_to_xcom_key a templated field in GoogleCloudStorageDownloadOperator |
| [AIRFLOW-278] Support utf-8 ecoding for SQL |
| [AIRFLOW-280] clean up tmp druid table no matter if an ingestion job succeeds or not |
| [AIRFLOW-274] Add XCom functionality to GoogleCloudStorageDownloadOperator |
| [AIRFLOW-273] Create an svg version of the airflow logo. |
| [AIRFLOW-275] Update contributing guidelines |
| [AIRFLOW-244] Modify hive operator to inject analysis data |
| [AIRFLOW-162] Allow variable to be accessible into templates |
| [AIRFLOW-248] Add Apache license header to all files |
| [AIRFLOW-263] Remove temp backtick file |
| [AIRFLOW-252] Raise Sqlite exceptions when deleting tasks instance in WebUI |
| [AIRFLOW-180] Fix timeout behavior for sensors |
| [AIRFLOW-262] Simplify commands in MANIFEST.in |
| [AIRFLOW-31] Add zope dependency |
| [AIRFLOW-6] Remove dependency on Highcharts |
| [AIRFLOW-234] make task that aren't `running` self-terminate |
| [AIRFLOW-256] Fix test_scheduler_reschedule heartrate |
| Add Python 3 compatibility fix |
| [AIRFLOW-31] Use standard imports for hooks/operators |
| [AIRFLOW-173] Initial implementation of FileSensor |
| [AIRFLOW-224] Collect orphaned tasks and reschedule them |
| [AIRFLOW-239] Fix tests indentation |
| [AIRFLOW-225] Better units for task duration graph |
| [AIRFLOW-241] Add testing done section to PR template |
| [AIRFLOW-222] Show duration of task instances in ui |
| [AIRFLOW-231] Do not eval user input in PrestoHook |
| [AIRFLOW-216] Add Sqoop Hook and Operator |
| [AIRFLOW-171] Add upgrade notes on email and S3 to 1.7.1.2 |
| [AIRFLOW-238] Make compatible with flask-admin 1.4.1 |
| [AIRFLOW-230] [HiveServer2Hook] adding multi statements support |
| [AIRFLOW-142] setup_env.sh doesn't download hive tarball if hdp is specified as distro |
| [AIRFLOW-223] Make parametrable the IP on which Flower binds to |
| [AIRFLOW-218] Added option to enable webserver gunicorn access/err logs |
| [AIRFLOW-213] Add "Closes #X" phrase to commit messages |
| [AIRFLOW-68] Align start_date with the schedule_interval |
| [AIRFLOW-9] Improving docs to meet Apache's standards |
| [AIRFLOW-131] Make XCom.clear more selective |
| [AIRFLOW-214] Fix occasion of detached taskinstance |
| [AIRFLOW-206] Add commit to close PR |
| [AIRFLOW-206] Always load local log files if they exist |
| [AIRFLOW-211] Fix JIRA "resolve" vs "close" behavior |
| [AIRFLOW-64] Add note about relative DAGS_FOLDER |
| [AIRFLOW-114] Sort plugins dropdown |
| [AIRFLOW-209] Add scheduler tests and improve lineage handling |
| [AIRFLOW-207] Improve JIRA auth workflow |
| [AIRFLOW-187] Improve PR tool UX |
| [AIRFLOW-155] Documentation of Qubole Operator |
| Optimize and refactor process_dag |
| [AIRFLOW-185] Handle empty versions list |
| [AIRFLOW-201] Fix for HiveMetastoreHook + kerberos |
| [AIRFLOW-202]: Fixes stray print line |
| [AIRFLOW-196] Fix bug that exception is not handled in HttpSensor |
| [AIRFLOW-195] : Add toggle support to subdag clearing in the CLI |
| [AIRFLOW-23] Support for Google Cloud DataProc |
| [AIRFLOW-25] Configuration for Celery always required |
| [AIRFLOW-190] Add codecov and remove download count |
| [AIRFLOW-168] Correct evaluation of @once schedule |
| [AIRFLOW-183] Fetch log from remote when worker returns 4xx/5xx response |
| [AIRFLOW-181] Fix failing unpacking of hadoop by redownloading |
| [AIRFLOW-176] remove unused formatting key |
| [AIRFLOW-167]: Add dag_state option in cli |
| [AIRFLOW-178] Fix bug so that zip file is detected in DAG folder |
| [AIRFLOW-176] Improve PR Tool JIRA workflow |
| AIRFLOW-45: Support Hidden Airflow Variables |
| [AIRFLOW-175] Run git-reset before checkout in PR tool |
| [AIRFLOW-157] Make PR tool Py3-compat; add JIRA command |
| [AIRFLOW-170] Add missing @apply_defaults |
| |
| |
| AIRFLOW 1.7.1, 2016-05-19 |
| ------------------------- |
| |
| - Fix : Don't treat premature tasks as could_not_run tasks |
| - AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/incubator-airflow#1485 |
| - Add logic to lock DB and avoid race condition |
| - Handle queued tasks from multiple jobs/executors |
| - AIRFLOW-52 Warn about overwriting tasks in a DAG |
| - Fix corner case with joining processes/queues (#1473) |
| - [AIRFLOW-52] Fix bottlenecks when working with many tasks |
| - Add columns to toggle extra detail in the connection list view. |
| - Log the number of errors when importing DAGs |
| - Log dagbag metrics dupplicate messages in queue into Statsd (#1406) |
| - Clean up issue template (#1419) |
| - correct missed arg.foreground to arg.daemon in cli |
| - Reinstate imports for github enterprise auth |
| - Use os.execvp instead of subprocess.Popen for the webserver |
| - Revert from using "--foreground" to "--daemon" |
| - Implement a Cloudant hook |
| - Add missing args to `airflow clear` |
| - Fixed a bug in the scheduler: num_runs used where runs intended |
| - Add multiprocessing support to the scheduler |
| - Partial fix to make sure next_run_date cannot be None |
| - Support list/get/set variables in the CLI |
| - Properly handle BigQuery booleans in BigQuery hook. |
| - Added the ability to view XCom variables in webserver |
| - Change DAG.tasks from a list to a dict |
| - Add support for zipped dags |
| - Stop creating hook on instantiating of S3 operator |
| - User subquery in views to find running DAGs |
| - Prevent DAGs from being reloaded on every scheduler iteration |
| - Add a missing word to docs |
| - Document the parameters of `DbApiHook` |
| - added oracle operator with existing oracle hook |
| - Add PyOpenSSL to Google cloud gcp_api. |
| - Remove executor error unit test |
| - Add DAG inference, deferral, and context manager |
| - Don't return error when writing files to Google cloud storage. |
| - Fix GCS logging for gcp_api. |
| - Ensure attr is in scope for error message |
| - Fixing misnamed PULL_REQUEST_TEMPLATE |
| - Extract non_pooled_task_slot_count into a configuration param |
| - Update plugins.rst for clarity on the example (#1309) |
| - Fix s3 logging issue |
| - Add twitter feed example dag |
| - Github ISSUE_TEMPLATE & PR_TEMPLATE cleanup |
| - Reduce logger verbosity |
| - Adding a PR Template |
| - Add Lucid to list of users |
| - Fix usage of asciiart |
| - Use session instead of outdated main_session for are_dependencies_met |
| - Fix celery flower port allocation |
| - Fix for missing edit actions due to flask-admin upgrade |
| - Fix typo in comment in prioritize_queued method |
| - Add HipchatOperator |
| - Include all example dags in backfill unit test |
| - Make sure skipped jobs are actually skipped |
| - Fixing a broken example dag, example_skip_dag.py |
| - Add consistent and thorough signal handling and logging |
| - Allow Operators to specify SKIPPED status internally |
| - Update docstring for executor trap unit test |
| - Doc: explain the usage of Jinja templating for templated params |
| - Don't schedule runs before the DAG's start_date |
| - Fix infinite retries with pools, with test |
| - Fix handling of deadlocked jobs |
| - Show only Airflow's deprecation warnings |
| - Set DAG_FOLDER for unit tests |
| - Missing comma in setup.py |
| - Deprecate *args and **kwargs in BaseOperator |
| - Raise deep scheduler exceptions to force a process restart. |
| - Change inconsistent example DAG owners |
| - Fix module path of send_email_smtp in configuration |
| - added Gentner Lab to list of users |
| - Increase timeout time for unit test |
| - Fix reading strings from conf |
| - CHORE - Remove Trailing Spaces |
| - Fix SSHExecuteOperator crash when using a custom ssh port |
| - Add note about airflow components to template |
| - Rewrite BackfillJob logic for clarity |
| - Add unit tests |
| - Fix miscellaneous bugs and clean up code |
| - Fix logic for determining DagRun states |
| - Make SchedulerJob not run EVERY queued task |
| - Improve BackfillJob handling of queued/deadlocked tasks |
| - Introduce ignore_depends_on_past parameters |
| - Use Popen with CeleryExecutor |
| - Rename user table to users to avoid conflict with postgres |
| - Beware of negative pool slots. |
| - Add support for calling_format from boto to S3_Hook |
| - Add pypi meta data and sync version number |
| - Set dags_are_paused_at_creation's default value to True |
| - Resurface S3Log class eaten by rebase/push -f |
| - Add missing session.commit() at end of initdb |
| - Validate that subdag tasks have pool slots available, and test |
| - Use urlparse for remote GCS logs, and add unit tests |
| - Make webserver worker timeout configurable |
| - Fixed scheduling for @once interval |
| - Use psycopg2's API for serializing postgres cell values |
| - Make the provide_session decorator more robust |
| - update link to Lyft's website |
| - use num_shards instead of partitions to be consistent with batch ingestion |
| - Add documentation links to README |
| - Update docs with separate configuration section |
| - Fix airflow.utils deprecation warning code being Python 3 incompatible |
| - Extract dbapi cell serialization into its own method |
| - Set Postgres autocommit as supported only if server version is < 7.4 |
| - Use refactored utils module in unit test imports |
| - Add changelog for 1.7.0 |
| - Use LocalExecutor on Travis if possible |
| - remove unused logging,errno, MiniHiveCluster imports |
| - remove extra import of logging lib |
| - Fix required gcloud version |
| - Refactoring utils into smaller submodules |
| - Properly measure number of task retry attempts |
| - Add function to get configuration as dict, plus unit tests |
| - Merge branch 'master' into hivemeta_sasl |
| - Add wiki link to README.md |
| - [hotfix] make email.Utils > email.utils for py3 |
| - Add the missing "Date" header to the warning e-mails |
| - Add the missing "Date" header to the warning e-mails |
| - Check name of SubDag class instead of class itself |
| - [hotfix] removing repo_token from .coveralls.yml |
| - Set the service_name in coverals.yml |
| - Fixes #1223 |
| - Update Airflow docs for remote logging |
| - Add unit tests for trapping Executor errors |
| - Make sure Executors properly trap errors |
| - Fix HttpOpSensorTest to use fake resquest session |
| - Linting |
| - Add an example on pool usage in the documentation |
| - Add two methods to bigquery hook's base cursor: run_table_upsert, which adds a table or updates an existing table; and run_grant_dataset_view_access, which grants view access to a given dataset for a given table. |
| - Tasks references upstream and downstream tasks using strings instead of references |
| - Fix typos in models.py |
| - Fix broken links in documentation |
| - [hotfix] fixing the Scheduler CLI to make dag_id optional |
| - Update link to Common Pitfalls wiki page in README |
| - Allow disabling periodic committing when inserting rows with DbApiHook |
| - added Glassdoor to "who uses airflow" |
| - Fix typo preventing from launching webserver |
| - Documentation badge |
| - Fixing ISSUE_TEMPLATE name to include .md suffix |
| - Adding an ISSUE_TEMPLATE to ensure that issues are adequately defined |
| - Linting & debugging |
| - Refactoring the CLI to be data-driven |
| - Updating the Bug Reporting protocol in the Contributing.md file |
| - Fixing the docs |
| - clean up references to old session |
| - remove session reference |
| - resolve conflict |
| - clear xcom data when task instance starts |
| - replace main_session with @provide_session |
| - Add extras to installation.rst |
| - Changes to Contributing to reflect more closely the current state of development. |
| - Modifying README to link to the wiki committer list |
| - docs: fixes a spelling mistake in default config |
| - Set killMode to 'control-group' for webservice.service |
| - Set KillMode to 'control-group' for worker.service |
| - Linting |
| - Fix WebHdfsSensor |
| - Adding more licenses to pass checks |
| - fixing landscape's config |
| - [hotfix] typo that made it in master |
| - [hotfix] fixing landscape requirement detection |
| - Make testing on hive conditional |
| - Merge remote-tracking branch 'upstream/master' into minicluster |
| - Update README.md |
| - Throwing in a few license to pass the build |
| - Adding a reqs.txt for landscape.io |
| - Pointing to a reqs file |
| - Some linting |
| - Adding a .landscape.yml file |
| - badge for pypi version |
| - Add license and ignore for sql and csv |
| - Use correct connection id |
| - Use correct table name |
| - Provide data for ci tests |
| - new badge for showing staleness of reqs |
| - removing requirements.txt as it is uni-dimensional |
| - Make it work on py3 |
| - Remove decode for logging |
| - Also keep py2 compatible |
| - More py3 fixes |
| - Convert to bytes for py3 compat |
| - Make sure to be py3 compatible |
| - Use unicodecsv to make it py3 compatible |
| - Replace tab with spaces Remove unused import |
| - Merge remote-tracking branch 'upstream/master' |
| - Support decimal types in MySQL to GCS |
| - Make sure to write binary as string can be unicode |
| - Ignore metastore |
| - More impyla fixes |
| - Test HivemetaStore if python 2 |
| - Allow users to set hdfs_namenode_principal in HDFSHook config |
| - Add tests for Hiveserver2 and fix some issues from impyla |
| - Merge branch 'impyla' into minicluster |
| - This patch allows for testing of hive operators and hooks. Sasl is used (NoSasl in connection string is not possible). Tests have been adjusted. |
| - Treat SKIPPED and SUCCESS the same way when evaluating depends_on_past=True |
| - fix bigquery hook |
| - version cap for gcp_api |
| - Fix typo when returning VerticaHook |
| - Adding fernet key to use it as part of stdout commands |
| - Adding support for ssl parameters. (picking up from jthomas123) |
| - more detail in error message. |
| - make sure paths don't conflict bc of trailing / |
| - change gcs_hook to self.hook |
| - refactor remote log read/write and add GCS support |
| - Only use multipart upload in S3Hook if file is large enough |
| - Merge branch 'airbnb/master' |
| - Add GSSAPI SASL to HiveMetaStoreHook. |
| - Add warning for deprecated setting |
| - Use kerberos_service_name = 'hive' as standard instead of 'impala'. |
| - Use GSSAPI instead of KERBEROS and provide backwards compatibility |
| - ISSUE-1123 Use impyla instead of pyhs2 |
| - set celery_executor to use queue name as exchange |
