| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> |
| <title>Apache Flink: Apache Flink Roadmap</title> |
| <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> |
| <link rel="icon" href="/favicon.ico" type="image/x-icon"> |
| |
| <!-- Bootstrap --> |
| <link rel="stylesheet" href="/css/bootstrap.min.css"> |
| <link rel="stylesheet" href="/css/flink.css"> |
| <link rel="stylesheet" href="/css/syntax.css"> |
| |
| <!-- Blog RSS feed --> |
| <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" /> |
| |
| <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> |
| <!-- We need to load Jquery in the header for custom google analytics event tracking--> |
| <script src="/js/jquery.min.js"></script> |
| |
| <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> |
| <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> |
| <!--[if lt IE 9]> |
| <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> |
| <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| <![endif]--> |
| <!-- Matomo --> |
| <script> |
| var _paq = window._paq = window._paq || []; |
| /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ |
| /* We explicitly disable cookie tracking to avoid privacy issues */ |
| _paq.push(['disableCookies']); |
| /* Measure a visit to flink.apache.org and nightlies.apache.org/flink as the same visit */ |
| _paq.push(["setDomains", ["*.flink.apache.org","*.nightlies.apache.org/flink"]]); |
| _paq.push(['trackPageView']); |
| _paq.push(['enableLinkTracking']); |
| (function() { |
| var u="//matomo.privacy.apache.org/"; |
| _paq.push(['setTrackerUrl', u+'matomo.php']); |
| _paq.push(['setSiteId', '1']); |
| var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; |
| g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); |
| })(); |
| </script> |
| <!-- End Matomo Code --> |
| </head> |
| <body> |
| |
| |
| <!-- Main content. --> |
| <div class="container"> |
| <div class="row"> |
| |
| |
| <div id="sidebar" class="col-sm-3"> |
| |
| |
| <!-- Top navbar. --> |
| <nav class="navbar navbar-default"> |
| <!-- The logo. --> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1"> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <div class="navbar-logo"> |
| <a href="/"> |
| <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px"> |
| </a> |
| </div> |
| </div><!-- /.navbar-header --> |
| |
| <!-- The navigation links. --> |
| <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> |
| <ul class="nav navbar-nav navbar-main"> |
| |
| <!-- First menu section explains visitors what Flink is --> |
| |
| <!-- What is Stream Processing? --> |
| <!-- |
| <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li> |
| --> |
| |
| <!-- What is Flink? --> |
| <li><a href="/flink-architecture.html">What is Apache Flink?</a></li> |
| |
| |
| |
| <!-- Stateful Functions? --> |
| |
| <li><a href="https://nightlies.apache.org/flink/flink-statefun-docs-stable/">What is Stateful Functions?</a></li> |
| |
| <!-- Flink ML? --> |
| |
| <li><a href="https://nightlies.apache.org/flink/flink-ml-docs-stable/">What is Flink ML?</a></li> |
| |
| <!-- Flink Kubernetes Operator? --> |
| |
| <li><a href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/">What is the Flink Kubernetes Operator?</a></li> |
| |
| <!-- Flink Table Store? --> |
| |
| <li><a href="https://nightlies.apache.org/flink/flink-table-store-docs-stable/">What is Flink Table Store?</a></li> |
| |
| <!-- Use cases --> |
| <li><a href="/usecases.html">Use Cases</a></li> |
| |
| <!-- Powered by --> |
| <li><a href="/poweredby.html">Powered By</a></li> |
| |
| |
| |
| <!-- Second menu section aims to support Flink users --> |
| |
| <!-- Downloads --> |
| <li><a href="/downloads.html">Downloads</a></li> |
| |
| <!-- Getting Started --> |
| <li class="dropdown"> |
| <a class="dropdown-toggle" data-toggle="dropdown" href="#">Getting Started<span class="caret"></span></a> |
| <ul class="dropdown-menu"> |
| <li><a href="https://nightlies.apache.org/flink/flink-docs-release-1.15//docs/try-flink/local_installation/" target="_blank">With Flink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-statefun-docs-release-3.2/getting-started/project-setup.html" target="_blank">With Flink Stateful Functions <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-ml-docs-release-2.1/try-flink-ml/quick-start.html" target="_blank">With Flink ML <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.1/try-flink-kubernetes-operator/quick-start.html" target="_blank">With Flink Kubernetes Operator <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-table-store-docs-release-0.1/try-table-store/quick-start.html" target="_blank">With Flink Table Store <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="/training.html">Training Course</a></li> |
| </ul> |
| </li> |
| |
| <!-- Documentation --> |
| <li class="dropdown"> |
| <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a> |
| <ul class="dropdown-menu"> |
| <li><a href="https://nightlies.apache.org/flink/flink-docs-release-1.15" target="_blank">Flink 1.15 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-docs-master" target="_blank">Flink Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-statefun-docs-release-3.2" target="_blank">Flink Stateful Functions 3.2 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-statefun-docs-master" target="_blank">Flink Stateful Functions Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-ml-docs-release-2.1" target="_blank">Flink ML 2.1 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-ml-docs-master" target="_blank">Flink ML Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.1" target="_blank">Flink Kubernetes Operator 1.1 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main" target="_blank">Flink Kubernetes Operator Main (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-table-store-docs-release-0.1" target="_blank">Flink Table Store 0.1 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://nightlies.apache.org/flink/flink-table-store-docs-master" target="_blank">Flink Table Store Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| </ul> |
| </li> |
| |
| <!-- getting help --> |
| <li><a href="/gettinghelp.html">Getting Help</a></li> |
| |
| <!-- Blog --> |
| <li><a href="/blog/"><b>Flink Blog</b></a></li> |
| |
| |
| <!-- Flink-packages --> |
| <li> |
| <a href="https://flink-packages.org" target="_blank">flink-packages.org <small><span class="glyphicon glyphicon-new-window"></span></small></a> |
| </li> |
| |
| |
| <!-- Third menu section aim to support community and contributors --> |
| |
| <!-- Community --> |
| <li><a href="/community.html">Community & Project Info</a></li> |
| |
| <!-- Roadmap --> |
| <li class="active"><a href="/roadmap.html">Roadmap</a></li> |
| |
| <!-- Contribute --> |
| <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li> |
| |
| |
| <!-- GitHub --> |
| <li> |
| <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a> |
| </li> |
| |
| |
| |
| <!-- Language Switcher --> |
| <li> |
| |
| |
| <a href="/zh/roadmap.html">中文版</a> |
| |
| |
| </li> |
| |
| </ul> |
| |
| <style> |
| .smalllinks:link { |
| display: inline-block !important; background: none; padding-top: 0px; padding-bottom: 0px; padding-right: 0px; min-width: 75px; |
| } |
| </style> |
| |
| <ul class="nav navbar-nav navbar-bottom"> |
| <hr /> |
| |
| <!-- Twitter --> |
| <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| |
| <!-- Visualizer --> |
| <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| |
| <li > |
| <a href="/security.html">Flink Security</a> |
| </li> |
| |
| <hr /> |
| |
| <li><a href="https://apache.org" target="_blank">Apache Software Foundation <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| |
| <li> |
| |
| <a class="smalllinks" href="https://www.apache.org/licenses/" target="_blank">License</a> <small><span class="glyphicon glyphicon-new-window"></span></small> |
| |
| <a class="smalllinks" href="https://www.apache.org/security/" target="_blank">Security</a> <small><span class="glyphicon glyphicon-new-window"></span></small> |
| |
| <a class="smalllinks" href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Donate</a> <small><span class="glyphicon glyphicon-new-window"></span></small> |
| |
| <a class="smalllinks" href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a> <small><span class="glyphicon glyphicon-new-window"></span></small> |
| </li> |
| |
| </ul> |
| </div><!-- /.navbar-collapse --> |
| </nav> |
| |
| </div> |
| <div class="col-sm-9"> |
| <div class="row-fluid"> |
| <div class="col-sm-12"> |
| <h1>Apache Flink Roadmap</h1> |
| |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| |
| <hr /> |
| |
| <div class="page-toc"> |
| <ul id="markdown-toc"> |
| <li><a href="#feature-radar" id="markdown-toc-feature-radar">Feature Radar</a> <ul> |
| <li><a href="#feature-stages" id="markdown-toc-feature-stages">Feature Stages</a></li> |
| </ul> |
| </li> |
| <li><a href="#unified-analytics-where-batch-and-streaming-come-together-sql-and-beyond" id="markdown-toc-unified-analytics-where-batch-and-streaming-come-together-sql-and-beyond">Unified Analytics: Where Batch and Streaming come Together; SQL and beyond.</a> <ul> |
| <li><a href="#a-unified-sql-platform" id="markdown-toc-a-unified-sql-platform">A unified SQL Platform</a></li> |
| <li><a href="#deep-batch--streaming-unification-for-the-datastream-api" id="markdown-toc-deep-batch--streaming-unification-for-the-datastream-api">Deep Batch / Streaming Unification for the DataStream API</a></li> |
| </ul> |
| </li> |
| <li><a href="#applications-vs-clusters-flink-as-a-library" id="markdown-toc-applications-vs-clusters-flink-as-a-library">Applications vs. Clusters; “Flink as a Library”</a></li> |
| <li><a href="#performance" id="markdown-toc-performance">Performance</a> <ul> |
| <li><a href="#faster-checkpoints-and-recovery" id="markdown-toc-faster-checkpoints-and-recovery">Faster Checkpoints and Recovery</a></li> |
| </ul> |
| </li> |
| <li><a href="#apache-flink-as-part-of-an-ever-evolving-data-ecosystem" id="markdown-toc-apache-flink-as-part-of-an-ever-evolving-data-ecosystem">Apache Flink as part of an ever evolving data ecosystem</a></li> |
| <li><a href="#documentation" id="markdown-toc-documentation">Documentation</a></li> |
| <li><a href="#stateful-functions" id="markdown-toc-stateful-functions">Stateful Functions</a></li> |
| <li><a href="#flink-kubernetes-operator" id="markdown-toc-flink-kubernetes-operator">Flink Kubernetes Operator</a></li> |
| <li><a href="#flink-table-store" id="markdown-toc-flink-table-store">Flink Table Store</a></li> |
| </ul> |
| |
| </div> |
| |
| <p><strong>Preamble:</strong> This roadmap means to provide user and contributors with a high-level summary of ongoing efforts, |
| grouped by the major threads to which the efforts belong. With so much that is happening in Flink, we |
| hope that this helps with understanding the direction of the project. |
| The roadmap contains both efforts in early stages as well as nearly completed |
| efforts, so that users may get a better impression of the overall status and direction of those developments.</p> |
| |
| <p>More details and various smaller changes can be found in the |
| <a href="https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals">FLIPs</a></p> |
| |
| <p>The roadmap is continuously updated. New features and efforts should be added to the roadmap once |
| there is consensus that they will happen and what they will roughly look like for the user.</p> |
| |
| <p><strong>Last Update:</strong> 2022-04-19</p> |
| |
| <hr /> |
| |
| <h1 id="feature-radar">Feature Radar</h1> |
| |
| <p>The feature radar is meant to give users guidance regarding feature maturity, as well as which features |
| are approaching end-of-life. For questions, please contact the developer mailing list: |
| <a href="mailto:dev@flink.apache.org">dev@flink.apache.org</a></p> |
| |
| <div class="row front-graphic"> |
| <img src="/img/flink_feature_radar_3.svg" width="700px" /> |
| </div> |
| |
| <h2 id="feature-stages">Feature Stages</h2> |
| |
| <ul> |
| <li><strong>MVP:</strong> Have a look, consider whether this can help you in the future.</li> |
| <li><strong>Beta:</strong> You can benefit from this, but you should carefully evaluate the feature.</li> |
| <li><strong>Ready and Evolving:</strong> Ready to use in production, but be aware you may need to make some adjustments to your application and setup in the future, when you upgrade Flink.</li> |
| <li><strong>Stable:</strong> Unrestricted use in production</li> |
| <li><strong>Reaching End-of-Life:</strong> Stable, still feel free to use, but think about alternatives. Not a good match for new long-lived projects.</li> |
| <li><strong>Deprecated:</strong> Start looking for alternatives now</li> |
| </ul> |
| |
| <hr /> |
| |
| <h1 id="unified-analytics-where-batch-and-streaming-come-together-sql-and-beyond">Unified Analytics: Where Batch and Streaming come Together; SQL and beyond.</h1> |
| |
| <p>Flink is a streaming data system in its core, that executes “batch as a special case of streaming”. |
| Efficient execution of batch jobs is powerful in its own right; but even more so, batch processing |
| capabilities (efficient processing of bounded streams) open the way for a seamless unification of |
| batch and streaming applications.</p> |
| |
| <p>Unified streaming/batch up-levels the streaming data paradigm: It gives users consistent semantics across |
| their real-time and lag-time applications. Furthermore, streaming applications often need to be complemented |
| by batch (bounded stream) processing, for example when reprocessing data after bugs or data quality issues, |
| or when bootstrapping new applications. A unified API and system make this much easier.</p> |
| |
| <h2 id="a-unified-sql-platform">A unified SQL Platform</h2> |
| |
| <p>The community has been building Flink to a powerful basis for a unified (batch and streaming) SQL analytics |
| platform, and is continuing to do so.</p> |
| |
| <p>SQL has very strong cross-batch-streaming semantics, allowing users to use the same queries for ad-hoc analytics |
| and as continuous queries. Flink already contains an efficient unified query engine, and a wide set of |
| integrations. With user feedback, those are continuously improved.</p> |
| |
| <p><strong>Going Beyond a SQL Stream/Batch Processing Engine</strong></p> |
| |
| <ul> |
| <li>To extend the capability of a pure stream processor and make Flink ready for future use cases, |
| <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-188%3A+Introduce+Built-in+Dynamic+Table+Storage">FLIP-188</a> |
| has been announced adding built in dynamic table storage.</li> |
| <li>The experience of updating Flink SQL based jobs has been rather cumbersome as it could have |
| lead to new job graphs making restoring from savepoints/checkpoints impossible. |
| <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191336489&src=contextnavpagetreemode">FLIP-190</a> |
| that already has been shipped as MVP is targeting this.</li> |
| </ul> |
| |
| <p><strong>Platform Infrastructure</strong></p> |
| |
| <ul> |
| <li>After <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-163%3A+SQL+Client+Improvements">FLIP-163</a> |
| the community is working again on a set of SQL Client usability improvements |
| (<a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-189%3A+SQL+Client+Usability+Improvements">FLIP-189</a>) |
| which is aiming at improving the user experience, when using the SQL client.</li> |
| </ul> |
| |
| <p><strong>Support for Common Languages, Formats, Catalogs</strong></p> |
| |
| <ul> |
| <li>With <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-216%3A++Introduce+pluggable+dialect+and++decouple+Hive+connector">FLIP-216</a> |
| there’s now the initiative to introduce pluggable dialects on the example of the Hive connector. |
| Including so many dependencies to make dialects work has lead to an overhead for contributors |
| and users.</li> |
| </ul> |
| |
| <p>Flink has a broad SQL coverage for batch (full TPC-DS support) and a state-of-the-art set of supported |
| operations in streaming. There is continuous effort to add more functions and cover more SQL operations.</p> |
| |
| <h2 id="deep-batch--streaming-unification-for-the-datastream-api">Deep Batch / Streaming Unification for the DataStream API</h2> |
| |
| <p>The <em>DataStream API</em> is Flink’s <em>physical</em> API, for use cases where users need very explicit control over data |
| types, streams, state, and time. This API is evolving to support efficient batch execution on bounded data.</p> |
| |
| <p>DataStream API executes the same dataflow shape in batch as in streaming, keeping the same operators. |
| That way users keep the same level of control over the dataflow, and our goal is to mix and switch between |
| batch/streaming execution in the future to make it a seamless experience.</p> |
| |
| <p><strong>Unified Sources and Sinks</strong></p> |
| |
| <ul> |
| <li> |
| <p>The first APIs and implementations of sources were specific to either streaming programs in the DataStream API |
| (<a href="https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/SourceFunction.java">SourceFunction</a>), |
| or to batch programs in the DataSet API (<a href="https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/InputFormat.java">InputFormat</a>).</p> |
| |
| <p>In this effort, we are creating sources that work across batch and streaming execution. The aim is to give |
| users a consistent experience across both modes, and to allow them to easily switch between streaming and batch |
| execution for their unbounded and bounded streaming applications. |
| The interface for this New Source API is done and available, and we are working on migrating more source connectors |
| to this new model, see <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface">FLIP-27</a>.</p> |
| </li> |
| <li> |
| <p>Similar to the sources, the original sink APIs are also specific to streaming |
| (<a href="https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java">SinkFunction</a>) |
| and batch (<a href="https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/OutputFormat.java">OutputFormat</a>) |
| APIs and execution.</p> |
| |
| <p>We have introduced a new API for sinks that consistently handles result writing and committing (<em>Transactions</em>) |
| across batch and streaming. The first iteration of the API exists, and we are porting sinks and refining the |
| API in the process. See <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-143%3A+Unified+Sink+API">FLIP-143</a>.</p> |
| </li> |
| </ul> |
| |
| <hr /> |
| |
| <h1 id="applications-vs-clusters-flink-as-a-library">Applications vs. Clusters; “Flink as a Library”</h1> |
| |
| <p>The goal of these efforts is to make it feel natural to deploy (long running streaming) Flink applications. |
| Instead of starting a cluster and submitting a job to that cluster, these efforts support deploying a streaming |
| job as a self contained application.</p> |
| |
| <p>For example as a simple Kubernetes deployment; deployed and scaled like a regular application without extra workflows.</p> |
| |
| <ul> |
| <li>There is currently a Kubernetes Operator being developed by the community. See |
| <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator">FLIP-212</a>.</li> |
| </ul> |
| |
| <hr /> |
| |
| <h1 id="performance">Performance</h1> |
| |
| <p>Continuous work to keep improving performance and recovery speed.</p> |
| |
| <h2 id="faster-checkpoints-and-recovery">Faster Checkpoints and Recovery</h2> |
| |
| <p>The community is continuously working on improving checkpointing and recovery speed. |
| Checkpoints and recovery are stable and have been a reliable workhorse for years. We are still |
| trying to make it faster, more predictable, and to remove some confusions and inflexibility in some areas.</p> |
| |
| <ul> |
| <li><a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-183%3A+Dynamic+buffer+size+adjustment">FLIP-183</a> |
| is targeting size of checkpoints by debloating the buffers. A first beta is already available.</li> |
| <li>With <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-151%3A+Incremental+snapshots+for+heap-based+state+backend">FLIP-151</a> |
| there is an ongoing effort to implement a heap based state backend.</li> |
| </ul> |
| |
| <hr /> |
| |
| <h1 id="apache-flink-as-part-of-an-ever-evolving-data-ecosystem">Apache Flink as part of an ever evolving data ecosystem</h1> |
| |
| <p>There is almost no use case in which Apache Flink is used on its own. It has established itself |
| as part of many data related reference architectures. In fact you’ll find the squirrel logo covering |
| several aspects. The community has added a lot of connectors and formats. With the already mentionend |
| <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface">FLIP-27</a> and |
| <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-143%3A+Unified+Sink+API">FLIP-143</a> |
| a new default for connectors has been established.</p> |
| |
| <ul> |
| <li>There are efforts to revise the formats API with |
| <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-219%3A+Revised+Format+API">FLIP-219</a></li> |
| <li>There is ongoing work on new connectors |
| (e.g. <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177045634">Pinot</a>)</li> |
| <li>Connectors will be hosted in an external repository going forward. See the |
| <a href="https://lists.apache.org/thread/8k1xonqt7hn0xldbky1cxfx3fzh6sj7h">ML thread</a></li> |
| </ul> |
| |
| <hr /> |
| |
| <h1 id="documentation">Documentation</h1> |
| |
| <p>There are various dedicated efforts to simplify the maintenance and structure (more intuitive navigation/reading) |
| of the documentation.</p> |
| |
| <ul> |
| <li>Docs Tech Stack: <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-157+Migrate+Flink+Documentation+from+Jekyll+to+Hugo">FLIP-157</a></li> |
| <li>General Docs Structure: <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-42%3A+Rework+Flink+Documentation">FLIP-42</a></li> |
| <li>SQL Docs: <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685">FLIP-60</a></li> |
| </ul> |
| |
| <hr /> |
| |
| <h1 id="stateful-functions">Stateful Functions</h1> |
| |
| <p>The Stateful Functions subproject has its own roadmap published under <a href="https://statefun.io/">statefun.io</a>.</p> |
| |
| <h1 id="flink-kubernetes-operator">Flink Kubernetes Operator</h1> |
| |
| <p>The Flink Kubernetes Operator subproject has its own roadmap under the <a href="https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/development/roadmap/">documentation</a>.</p> |
| |
| <h1 id="flink-table-store">Flink Table Store</h1> |
| |
| <p>The Flink Table Store subproject has its own roadmap under the <a href="https://nightlies.apache.org/flink/flink-table-store-docs-master/docs/development/roadmap/">documentation</a>.</p> |
| |
| |
| </div> |
| </div> |
| |
| </div> |
| </div> |
| |
| <hr /> |
| |
| <div class="row"> |
| <div class="footer text-center col-sm-12"> |
| <p>Copyright © 2014-2022 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p> |
| <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p> |
| <p><a href="https://privacy.apache.org/policies/privacy-policy-public.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p> |
| </div> |
| </div> |
| </div><!-- /.container --> |
| |
| <!-- Include all compiled plugins (below), or include individual files as needed --> |
| <script src="/js/jquery.matchHeight-min.js"></script> |
| <script src="/js/bootstrap.min.js"></script> |
| <script src="/js/codetabs.js"></script> |
| <script src="/js/stickysidebar.js"></script> |
| </body> |
| </html> |
