Rebuild website
diff --git a/content/blog/feed.xml b/content/blog/feed.xml
index 20b395c..2ae389a 100644
--- a/content/blog/feed.xml
+++ b/content/blog/feed.xml
@@ -7,6 +7,162 @@
<atom:link href="https://flink.apache.org/blog/feed.xml" rel="self" type="application/rss+xml" />
<item>
+<title>Apache Beam: How Beam Runs on Top of Flink</title>
+<description><p>Note: This blog post is based on the talk <a href="https://www.youtube.com/watch?v=hxHGLrshnCY">“Beam on Flink: How Does It Actually Work?”</a>.</p>
+
+<p><a href="https://flink.apache.org/">Apache Flink</a> and <a href="https://beam.apache.org/">Apache Beam</a> are open-source frameworks for parallel, distributed data processing at scale. Unlike Flink, Beam does not come with a full-blown execution engine of its own but plugs into other execution engines, such as Apache Flink, Apache Spark, or Google Cloud Dataflow. In this blog post we discuss the reasons to use Flink together with Beam for your batch and stream processing needs. We also take a closer look at how Beam works with Flink to provide an idea of the technical aspects of running Beam pipelines with Flink. We hope you find some useful information on how and why the two frameworks can be utilized in combination. For more information, you can refer to the corresponding <a href="https://beam.apache.org/documentation/runners/flink/">documentation</a> on the Beam website or contact the community through the <a href="https://beam.apache.org/community/contact-us/">Beam mailing list</a>.</p>
+
+<h1 id="what-is-apache-beam">What is Apache Beam</h1>
+
+<p><a href="https://beam.apache.org/">Apache Beam</a> is an open-source, unified model for defining batch and streaming data-parallel processing pipelines. It is unified in the sense that you use a single API, in contrast to using a separate API for batch and streaming like it is the case in Flink. Beam was originally developed by Google which released it in 2014 as the Cloud Dataflow SDK. In 2016, it was donated to <a href="https://www.apache.org/">the Apache Software Foundation</a> with the name of Beam. It has been developed by the open-source community ever since. With Apache Beam, developers can write data processing jobs, also known as pipelines, in multiple languages, e.g. Java, Python, Go, SQL. A pipeline is then executed by one of Beam’s Runners. A Runner is responsible for translating Beam pipelines such that they can run on an execution engine. Every supported execution engine has a Runner. The following Runners are available: Apache Flink, Apache Spark, Apache Samza, Hazelcast Jet, Google Cloud Dataflow, and others.</p>
+
+<p>The execution model, as well as the API of Apache Beam, are similar to Flink’s. Both frameworks are inspired by the <a href="https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf">MapReduce</a>, <a href="https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41378.pdf">MillWheel</a>, and <a href="https://research.google/pubs/pub43864/">Dataflow</a> papers. Like Flink, Beam is designed for parallel, distributed data processing. Both have similar transformations, support for windowing, event/processing time, watermarks, timers, triggers, and much more. However, Beam not being a full runtime focuses on providing the framework for building portable, multi-language batch and stream processing pipelines such that they can be run across several execution engines. The idea is that you write your pipeline once and feed it with either batch or streaming data. When you run it, you just pick one of the supported backends to execute. A large integration test suite in Beam called “ValidatesRunner” ensures that the results will be the same, regardless of which backend you choose for the execution.</p>
+
+<p>One of the most exciting developments in the Beam technology is the framework’s support for multiple programming languages including Java, Python, Go, Scala and SQL. Essentially, developers can write their applications in a programming language of their choice. Beam, with the help of the Runners, translates the program to one of the execution engines, as shown in the diagram below.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-beam-vision.png" width="600px" alt="The vision of Apache Beam" />
+</center>
+
+<h1 id="reasons-to-use-beam-with-flink">Reasons to use Beam with Flink</h1>
+
+<p>Why would you want to use Beam with Flink instead of directly using Flink? Ultimately, Beam and Flink complement each other and provide additional value to the user. The main reasons for using Beam with Flink are the following:</p>
+
+<ul>
+ <li>Beam provides a unified API for both batch and streaming scenarios.</li>
+ <li>Beam comes with native support for different programming languages, like Python or Go with all their libraries like Numpy, Pandas, Tensorflow, or TFX.</li>
+ <li>You get the power of Apache Flink like its exactly-once semantics, strong memory management and robustness.</li>
+ <li>Beam programs run on your existing Flink infrastructure or infrastructure for other supported Runners, like Spark or Google Cloud Dataflow.</li>
+ <li>You get additional features like side inputs and cross-language pipelines that are not supported natively in Flink but only supported when using Beam with Flink.</li>
+</ul>
+
+<h1 id="the-flink-runner-in-beam">The Flink Runner in Beam</h1>
+
+<p>The Flink Runner in Beam translates Beam pipelines into Flink jobs. The translation can be parameterized using Beam’s pipeline options which are parameters for settings like configuring the job name, parallelism, checkpointing, or metrics reporting.</p>
+
+<p>If you are familiar with a DataSet or a DataStream, you will have no problems understanding what a PCollection is. PCollection stands for parallel collection in Beam and is exactly what DataSet/DataStream would be in Flink. Due to Beam’s unified API we only have one type of results of transformation: PCollection.</p>
+
+<p>Beam pipelines are composed of transforms. Transforms are like operators in Flink and come in two flavors: primitive and composite transforms. The beauty of all this is that Beam only comes with a small set of primitive transforms which are:</p>
+
+<ul>
+ <li><code>Source</code> (for loading data)</li>
+ <li><code>ParDo</code> (think of a flat map operator on steroids)</li>
+ <li><code>GroupByKey</code> (think of keyBy() in Flink)</li>
+ <li><code>AssignWindows</code> (windows can be assigned at any point in time in Beam)</li>
+ <li><code>Flatten</code> (like a union() operation in Flink)</li>
+</ul>
+
+<p>Composite transforms are built by combining the above primitive transforms. For example, <code>Combine = GroupByKey + ParDo</code>.</p>
+
+<h1 id="flink-runner-internals">Flink Runner Internals</h1>
+
+<p>Although using the Flink Runner in Beam has no prerequisite to understanding its internals, we provide more details of how the Flink runner works in Beam to share knowledge of how the two frameworks can integrate and work together to provide state-of-the-art streaming data pipelines.</p>
+
+<p>The Flink Runner has two translation paths. Depending on whether we execute in batch or streaming mode, the Runner either translates into Flink’s DataSet or into Flink’s DataStream API. Since multi-language support has been added to Beam, another two translation paths have been added. To summarize the four modes:</p>
+
+<ol>
+ <li><strong>The Classic Flink Runner for batch jobs:</strong> Executes batch Java pipelines</li>
+ <li><strong>The Classic Flink Runner for streaming jobs:</strong> Executes streaming Java pipelines</li>
+ <li><strong>The Portable Flink Runner for batch jobs:</strong> Executes Java as well as Python, Go and other supported SDK pipelines for batch scenarios</li>
+ <li><strong>The Portable Flink Runner for streaming jobs:</strong> Executes Java as well as Python, Go and other supported SDK pipelines for streaming scenarios</li>
+</ol>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-runner-translation-paths.png" width="300px" alt="The 4 translation paths in the Beam's Flink Runner" />
+</center>
+
+<h2 id="the-classic-flink-runner-in-beam">The “Classic” Flink Runner in Beam</h2>
+
+<p>The classic Flink Runner was the initial version of the Runner, hence the “classic” name. Beam pipelines are represented as a graph in Java which is composed of the aforementioned composite and primitive transforms. Beam provides translators which traverse the graph in topological order. Topological order means that we start from all the sources first as we iterate through the graph. Presented with a transform from the graph, the Flink Runner generates the API calls as you would normally when writing a Flink job.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/classic-flink-runner-beam.png" width="600px" alt="The Classic Flink Runner in Beam" />
+</center>
+
+<p>While Beam and Flink share very similar concepts, there are enough differences between the two frameworks that make Beam pipelines impossible to be translated 1:1 into a Flink program. In the following sections, we will present the key differences:</p>
+
+<h3 id="serializers-vs-coders">Serializers vs Coders</h3>
+
+<p>When data is transferred over the wire in Flink, it has to be turned into bytes. This is done with the help of serializers. Flink has a type system to instantiate the correct coder for a given type, e.g. <code>StringTypeSerializer</code> for a String. Apache Beam also has its own type system which is similar to Flink’s but uses slightly different interfaces. Serializers are called Coders in Beam. In order to make a Beam Coder run in Flink, we have to make the two serializer types compatible. This is done by creating a special Flink type information that looks like the one in Flink but calls the appropriate Beam coder. That way, we can use Beam’s coders although we are executing the Beam job with Flink. Flink operators expect a TypeInformation, e.g. <code>StringTypeInformation</code>, for which we use a <code>CoderTypeInformation</code> in Beam. The type information returns the serializer for which we return a <code>CoderTypeSerializer</code>, which calls the underlying Beam Coder.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-serializers-coders.png" width="300px" alt="Serializers vs Coders" />
+</center>
+
+<h3 id="read">Read</h3>
+
+<p>The <code>Read</code> transform provides a way to read data into your pipeline in Beam. The Read transform is supported by two wrappers in Beam, the <code>SourceInputFormat</code> for batch processing and the <code>UnboundedSourceWrapper</code> for stream processing.</p>
+
+<h3 id="pardo">ParDo</h3>
+
+<p><code>ParDo</code> is the swiss army knife of Beam and can be compared to a <code>RichFlatMapFunction</code> in Flink with additional features such as <code>SideInputs</code>, <code>SideOutputs</code>, State and Timers. <code>ParDo</code> is essentially translated by the Flink runner using the <code>FlinkDoFnFunction</code> for batch processing or the <code>FlinkStatefulDoFnFunction</code>, while for streaming scenarios the translation is executed with the <code>DoFnOperator</code> that takes care of checkpointing and buffering of data during checkpoints, watermark emissions and maintenance of state and timers. This is all executed by Beam’s interface, called the <code>DoFnRunner</code>, that encapsulates Beam-specific execution logic, like retrieving state, executing state and timers, or reporting metrics.</p>
+
+<h3 id="side-inputs">Side Inputs</h3>
+
+<p>In addition to the main input, ParDo transforms can have a number of side inputs. A side input can be a static set of data that you want to have available at all parallel instances. However, it is more flexible than that. You can have keyed and even windowed side input which updates based on the window size. This is a very powerful concept which does not exist in Flink but is added on top of Flink using Beam.</p>
+
+<h3 id="assignwindows">AssignWindows</h3>
+
+<p>In Flink, windows are assigned by the <code>WindowOperator</code> when you use the <code>window()</code> in the API. In Beam, windows can be assigned at any point in time. Any element is implicitly part of a window. If no window is assigned explicitly, the element is part of the <code>GlobalWindow</code>. Window information is stored for each element in a wrapper called <code>WindowedValue</code>. The window information is only used once we issue a <code>GroupByKey</code>.</p>
+
+<h3 id="groupbykey">GroupByKey</h3>
+
+<p>Most of the time it is useful to partition the data by a key. In Flink, this is done via the <code>keyBy()</code> API call. In Beam the <code>GroupByKey</code> transform can only be applied if the input is of the form <code>KV&lt;Key, Value&gt;</code>. Unlike Flink where the key can even be nested inside the data, Beam enforces the key to always be explicit. The <code>GroupByKey</code> transform then groups the data by key and by window which is similar to what <code>keyBy(..).window(..)</code> would give us in Flink. Beam has its own set of libraries to do that because Beam has its own set of window functions and triggers. Essentially, GroupByKey is very similar to what the WindowOperator does in Flink.</p>
+
+<h3 id="flatten">Flatten</h3>
+
+<p>The Flatten operator takes multiple DataSet/DataStreams, called P[arallel]Collections in Beam, and combines them into one collection. This is equivalent to Flink’s <code>union()</code> operation.</p>
+
+<h2 id="the-portable-flink-runner-in-beam">The “Portable” Flink Runner in Beam</h2>
+
+<p>The portable Flink Runner in Beam is the evolution of the classic Runner. Classic Runners are tied to the JVM ecosystem, but the Beam community wanted to move past this and also execute Python, Go and other languages. This adds another dimension to Beam in terms of portability because, like previously mentioned, Beam already had portability across execution engines. It was necessary to change the translation logic of the Runner to be able to support language portability.</p>
+
+<p>There are two important building blocks for portable Runners:</p>
+
+<ol>
+ <li>A common pipeline format across all the languages: The Runner API</li>
+ <li>A common interface during execution for the communication between the Runner and the code written in any language: The Fn API</li>
+</ol>
+
+<p>The Runner API provides a universal representation of the pipeline as Protobuf which contains the transforms, types, and user code. Protobuf was chosen as the format because every language has libraries available for it. Similarly, for the execution part, Beam introduced the Fn API interface to handle the communication between the Runner/execution engine and the user code that may be written in a different language and executes in a different process. Fn API is pronounced “fun API”, you may guess why.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability.png" width="600px" alt="Language Portability in Apache Beam" />
+</center>
+
+<h2 id="how-are-beam-programs-translated-in-language-portability">How Are Beam Programs Translated In Language Portability?</h2>
+
+<p>Users write their Beam pipelines in one language, but they may get executed in an environment based on a completely different language. How does that work? To explain that, let’s follow the lifecycle of a pipeline. Let’s suppose we use the Python SDK to write the pipeline. Before submitting the pipeline via the Job API to Beam’s JobServer, Beam would convert it to the Runner API, the language-agnostic format we described before. The JobServer is also a Beam component that handles the staging of the required dependencies during execution. The JobServer will then kick-off the translation which is similar to the classic Runner. However, an important change is the so-called <code>ExecutableStage</code> transform. It is essentially a ParDo transform that we already know but designed for holding language-dependent code. Beam tries to combine as many of these transforms into one “executable stage”. The result again is a Flink program which is then sent to the Flink cluster and executed there. The major difference compared to the classic Runner is that during execution we will start <em>environments</em> to execute the aforementioned <em>ExecutableStages</em>. The following environments are available:</p>
+
+<ul>
+ <li>Docker-based (the default)</li>
+ <li>Process-based (a simple process is started)</li>
+ <li>Externally-provided (K8s or other schedulers)</li>
+ <li>Embedded (intended for testing and only works with Java)</li>
+</ul>
+
+<p>Environments hold the <em>SDK Harness</em> which is the code that handles the execution and the communication with the Runner over the Fn API. For example, when Flink executes Python code, it sends the data to the Python environment containing the Python SDK Harness. Sending data to an external process involves a minor overhead which we have measured to be 5-10% slower than the classic Java pipelines. However, Beam uses a fusion of transforms to execute as many transforms as possible in the same environment which share the same input or output. That’s why in real-world scenarios the overhead could be much lower.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability-architecture.png" width="600px" alt="Language Portability Architecture in beam" />
+</center>
+
+<p>Environments can be present for many languages. This opens up an entirely new type of pipelines: cross-language pipelines. In cross-language pipelines we can combine transforms of two or more languages, e.g. a machine learning pipeline with the feature generation written in Java and the learning written in Python. All this can be run on top of Flink.</p>
+
+<h2 id="conclusion">Conclusion</h2>
+
+<p>Using Apache Beam with Apache Flink combines (a.) the power of Flink with (b.) the flexibility of Beam. All it takes to run Beam is a Flink cluster, which you may already have. Apache Beam’s fully-fledged Python API is probably the most compelling argument for using Beam with Flink, but the unified API which allows to “write-once” and “execute-anywhere” is also very appealing to Beam users. On top of this, features like side inputs and a rich connector ecosystem are also reasons why people like Beam.</p>
+
+<p>With the introduction of schemas, a new format for handling type information, Beam is heading in a similar direction as Flink with its type system which is essential for the Table API or SQL. Speaking of, the next Flink release will include a Python version of the Table API which is based on the language portability of Beam. Looking ahead, the Beam community plans to extend the support for interactive programs like notebooks. TFX, which is built with Beam, is a very powerful way to solve many problems around training and validating machine learning models.</p>
+
+<p>For many years, Beam and Flink have inspired and learned from each other. With the Python support being based on Beam in Flink, they only seem to come closer to each other. That’s all the better for the community, and also users have more options and functionality to choose from.</p>
+</description>
+<pubDate>Sat, 22 Feb 2020 13:00:00 +0100</pubDate>
+<link>https://flink.apache.org/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html</link>
+<guid isPermaLink="true">/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html</guid>
+</item>
+
+<item>
<title>No Java Required: Configuring Sources and Sinks in SQL</title>
<description><h1 id="introduction">Introduction</h1>
@@ -16366,88 +16522,5 @@
<guid isPermaLink="true">/news/2014/09/26/release-0.6.1.html</guid>
</item>
-<item>
-<title>Apache Flink 0.6 available</title>
-<description><p>We are happy to announce the availability of Flink 0.6. This is the
-first release of the system inside the Apache Incubator and under the
-name Flink. Releases up to 0.5 were under the name Stratosphere, the
-academic and open source project that Flink originates from.</p>
-
-<h2 id="what-is-flink">What is Flink?</h2>
-
-<p>Apache Flink is a general-purpose data processing engine for
-clusters. It runs on YARN clusters on top of data stored in Hadoop, as
-well as stand-alone. Flink currently has programming APIs in Java and
-Scala. Jobs are executed via Flink’s own runtime engine. Flink
-features:</p>
-
-<p><strong>Robust in-memory and out-of-core processing:</strong> once read, data stays
- in memory as much as possible, and is gracefully de-staged to disk in
- the presence of memory pressure from limited memory or other
- applications. The runtime is designed to perform very well both in
- setups with abundant memory and in setups where memory is scarce.</p>
-
-<p><strong>POJO-based APIs:</strong> when programming, you do not have to pack your
- data into key-value pairs or some other framework-specific data
- model. Rather, you can use arbitrary Java and Scala types to model
- your data.</p>
-
-<p><strong>Efficient iterative processing:</strong> Flink contains explicit “iterate” operators
- that enable very efficient loops over data sets, e.g., for machine
- learning and graph applications.</p>
-
-<p><strong>A modular system stack:</strong> Flink is not a direct implementation of its
- APIs but a layered system. All programming APIs are translated to an
- intermediate program representation that is compiled and optimized
- via a cost-based optimizer. Lower-level layers of Flink also expose
- programming APIs for extending the system.</p>
-
-<p><strong>Data pipelining/streaming:</strong> Flink’s runtime is designed as a
- pipelined data processing engine rather than a batch processing
- engine. Operators do not wait for their predecessors to finish in
- order to start processing data. This results to very efficient
- handling of large data sets.</p>
-
-<h2 id="release-06">Release 0.6</h2>
-
-<p>Flink 0.6 builds on the latest Stratosphere 0.5 release. It includes
-many bug fixes and improvements that make the system more stable and
-robust, as well as breaking API changes.</p>
-
-<p>The full release notes are available <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&amp;version=12327101">here</a>.</p>
-
-<p>Download the release <a href="http://flink.incubator.apache.org/downloads.html">here</a>.</p>
-
-<h2 id="contributors">Contributors</h2>
-
-<ul>
- <li>Wilson Cao</li>
- <li>Ufuk Celebi</li>
- <li>Stephan Ewen</li>
- <li>Jonathan Hasenburg</li>
- <li>Markus Holzemer</li>
- <li>Fabian Hueske</li>
- <li>Sebastian Kunert</li>
- <li>Vikhyat Korrapati</li>
- <li>Aljoscha Krettek</li>
- <li>Sebastian Kruse</li>
- <li>Raymond Liu</li>
- <li>Robert Metzger</li>
- <li>Mingliang Qi</li>
- <li>Till Rohrmann</li>
- <li>Henry Saputra</li>
- <li>Chesnay Schepler</li>
- <li>Kostas Tzoumas</li>
- <li>Robert Waury</li>
- <li>Timo Walther</li>
- <li>Daniel Warneke</li>
- <li>Tobias Wiens</li>
-</ul>
-</description>
-<pubDate>Tue, 26 Aug 2014 12:00:00 +0200</pubDate>
-<link>https://flink.apache.org/news/2014/08/26/release-0.6.html</link>
-<guid isPermaLink="true">/news/2014/08/26/release-0.6.html</guid>
-</item>
-
</channel>
</rss>
diff --git a/content/blog/index.html b/content/blog/index.html
index e79c091..a8bc137 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -187,6 +187,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></h2>
+
+ <p>22 Feb 2020
+ Maximilian Michels (<a href="https://twitter.com/stadtlegende">@stadtlegende</a>) & Markos Sfikas (<a href="https://twitter.com/MarkSfik">@MarkSfik</a>)</p>
+
+ <p>This blog post discusses the reasons to use Flink together with Beam for your stream processing needs and takes a closer look at how Flink works with Beam under the hood.</p>
+
+ <p><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></h2>
<p>20 Feb 2020
@@ -309,21 +322,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1 Released</a></h2>
-
- <p>18 Oct 2019
- Jark Wu (<a href="https://twitter.com/JarkWu">@JarkWu</a>)</p>
-
- <p><p>The Apache Flink community released the first bugfix version of the Apache Flink 1.9 series.</p>
-
-</p>
-
- <p><a href="/news/2019/10/18/release-1.9.1.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -334,7 +332,7 @@
</li>
<li>
- <span class="page_number ">Page: 1 of 10</span>
+ <span class="page_number ">Page: 1 of 11</span>
</li>
<li>
@@ -356,6 +354,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page10/index.html b/content/blog/page10/index.html
index cec91a6..05c760d 100644
--- a/content/blog/page10/index.html
+++ b/content/blog/page10/index.html
@@ -187,6 +187,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html">Peeking into Apache Flink's Engine Room</a></h2>
+
+ <p>13 Mar 2015 by Fabian Hüske (<a href="https://twitter.com/">@fhueske</a>)
+ </p>
+
+ <p>Joins are prevalent operations in many data processing applications. Most data processing systems feature APIs that make joining data sets very easy. However, the internal algorithms for join processing are much more involved – especially if large data sets need to be efficiently handled. In this blog post, we cut through Apache Flink’s layered architecture and take a look at its internals with a focus on how it handles joins.</p>
+
+ <p><a href="/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2015/03/02/february-2015-in-flink.html">February 2015 in the Flink community</a></h2>
<p>02 Mar 2015
@@ -328,24 +341,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2014/08/26/release-0.6.html">Apache Flink 0.6 available</a></h2>
-
- <p>26 Aug 2014
- </p>
-
- <p><p>We are happy to announce the availability of Flink 0.6. This is the
-first release of the system inside the Apache Incubator and under the
-name Flink. Releases up to 0.5 were under the name Stratosphere, the
-academic and open source project that Flink originates from.</p>
-
-</p>
-
- <p><a href="/news/2014/08/26/release-0.6.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -356,11 +351,11 @@
</li>
<li>
- <span class="page_number ">Page: 10 of 10</span>
+ <span class="page_number ">Page: 10 of 11</span>
</li>
<li>
- <span>Next</span>
+ <a href="/blog/page11" class="next">Next</a>
</li>
</ul>
@@ -378,6 +373,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page11/index.html b/content/blog/page11/index.html
new file mode 100644
index 0000000..fd62458
--- /dev/null
+++ b/content/blog/page11/index.html
@@ -0,0 +1,1308 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Blog</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache Flink?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/faq.html">FAQ</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Getting Started -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.10/getting-started/index.html" target="_blank">Getting Started <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.10" target="_blank">1.10 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li class="active"><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+ <!-- Flink-packages -->
+ <li>
+ <a href="https://flink-packages.org" target="_blank">flink-packages.org <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <!-- link to the Chinese home page when current is blog page -->
+ <a href="/zh">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <hr />
+
+ <li><a href="https://apache.org" target="_blank">Apache Software Foundation <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <li>
+ <style>
+ .smalllinks:link {
+ display: inline-block !important; background: none; padding-top: 0px; padding-bottom: 0px; padding-right: 0px; min-width: 75px;
+ }
+ </style>
+
+ <a class="smalllinks" href="https://www.apache.org/licenses/" target="_blank">License</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/security/" target="_blank">Security</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Donate</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+ </li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <h1>Blog</h1>
+<hr />
+
+<div class="row">
+ <div class="col-sm-8">
+ <!-- Blog posts -->
+
+ <article>
+ <h2 class="blog-title"><a href="/news/2014/08/26/release-0.6.html">Apache Flink 0.6 available</a></h2>
+
+ <p>26 Aug 2014
+ </p>
+
+ <p><p>We are happy to announce the availability of Flink 0.6. This is the
+first release of the system inside the Apache Incubator and under the
+name Flink. Releases up to 0.5 were under the name Stratosphere, the
+academic and open source project that Flink originates from.</p>
+
+</p>
+
+ <p><a href="/news/2014/08/26/release-0.6.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+
+ <!-- Pagination links -->
+
+ <ul class="pager">
+ <li>
+
+ <a href="/blog/page10" class="previous">Previous</a>
+
+ </li>
+ <li>
+ <span class="page_number ">Page: 11 of 11</span>
+ </li>
+ <li>
+
+ <span>Next</span>
+
+ </li>
+ </ul>
+
+ </div>
+
+ <div class="col-sm-4" markdown="1">
+ <!-- Blog posts by YEAR -->
+
+
+
+
+
+ <h2>2020</h2>
+
+ <ul id="markdown-toc">
+
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2020/02/11/release-1.10.0.html">Apache Flink 1.10.0 Release Announcement</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">A Guide for Unit Testing in Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2020/01/30/release-1.9.2.html">Apache Flink 1.9.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink.html">State Unlocked: Interacting with State in Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2020/01/15/demo-fraud-detection.html">Advanced Flink Application Patterns Vol.1: Case Study of a Fraud Detection System</a></li>
+
+
+
+ </ul>
+ <hr>
+ <h2>2019</h2>
+ <ul id="markdown-toc">
+
+
+
+
+
+
+
+ <li><a href="/news/2019/12/11/release-1.8.3.html">Apache Flink 1.8.3 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/12/09/flink-kubernetes-kudo.html">Running Apache Flink on Kubernetes with KUDO</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/feature/2019/09/13/state-processor-api.html">The State Processor API: How to Read, write and modify the state of Flink applications</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/09/11/release-1.8.2.html">Apache Flink 1.8.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/09/10/community-update.html">Flink Community Update - September'19</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/08/22/release-1.9.0.html">Apache Flink 1.9.0 Release Announcement</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/2019/07/23/flink-network-stack-2.html">Flink Network Stack Vol. 2: Monitoring, Metrics, and that Backpressure Thing</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/07/02/release-1.8.1.html">Apache Flink 1.8.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/2019/06/26/broadcast-state.html">A Practical Guide to Broadcast State in Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/2019/06/05/flink-network-stack.html">A Deep-Dive into Flink's Network Stack</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/2019/05/19/state-ttl.html">State TTL in Flink 1.8.0: How to Automatically Cleanup Application State in Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/2019/05/14/temporal-tables.html">Flux capacitor, huh? Temporal Tables and Joins in Streaming SQL</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/2019/05/03/pulsar-flink.html">When Flink & Pulsar Come Together</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/04/17/sod.html">Apache Flink's Application to Season of Docs</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/04/09/release-1.8.0.html">Apache Flink 1.8.0 Release Announcement</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/02/25/monitoring-best-practices.html">Monitoring Apache Flink Applications 101</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/02/25/release-1.6.4.html">Apache Flink 1.6.4 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/02/15/release-1.7.2.html">Apache Flink 1.7.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2019/02/13/unified-batch-streaming-blink.html">Batch as a Special Case of Streaming and Alibaba's contribution of Blink</a></li>
+
+
+
+ </ul>
+ <hr>
+ <h2>2018</h2>
+ <ul id="markdown-toc">
+
+
+
+
+
+
+
+ <li><a href="/news/2018/12/26/release-1.5.6.html">Apache Flink 1.5.6 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/12/22/release-1.6.3.html">Apache Flink 1.6.3 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/12/21/release-1.7.1.html">Apache Flink 1.7.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/11/30/release-1.7.0.html">Apache Flink 1.7.0 Release Announcement</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/10/29/release-1.6.2.html">Apache Flink 1.6.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/10/29/release-1.5.5.html">Apache Flink 1.5.5 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/09/20/release-1.6.1.html">Apache Flink 1.6.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/09/20/release-1.5.4.html">Apache Flink 1.5.4 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/08/21/release-1.5.3.html">Apache Flink 1.5.3 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/08/09/release-1.6.0.html">Apache Flink 1.6.0 Release Announcement</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/07/31/release-1.5.2.html">Apache Flink 1.5.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/07/12/release-1.5.1.html">Apache Flink 1.5.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/05/25/release-1.5.0.html">Apache Flink 1.5.0 Release Announcement</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/03/15/release-1.3.3.html">Apache Flink 1.3.3 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/03/08/release-1.4.2.html">Apache Flink 1.4.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/features/2018/03/01/end-to-end-exactly-once-apache-flink.html">An Overview of End-to-End Exactly-Once Processing in Apache Flink (with Apache Kafka, too!)</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2018/02/15/release-1.4.1.html">Apache Flink 1.4.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/features/2018/01/30/incremental-checkpointing.html">Managing Large State in Apache Flink: An Intro to Incremental Checkpointing</a></li>
+
+
+
+ </ul>
+ <hr>
+ <h2>2017</h2>
+ <ul id="markdown-toc">
+
+
+
+
+
+
+
+ <li><a href="/news/2017/12/21/2017-year-in-review.html">Apache Flink in 2017: Year in Review</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/12/12/release-1.4.0.html">Apache Flink 1.4.0 Release Announcement</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/11/22/release-1.4-and-1.5-timeline.html">Looking Ahead to Apache Flink 1.4.0 and 1.5.0</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/08/05/release-1.3.2.html">Apache Flink 1.3.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/features/2017/07/04/flink-rescalable-state.html">A Deep Dive into Rescalable State in Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/06/23/release-1.3.1.html">Apache Flink 1.3.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/06/01/release-1.3.0.html">Apache Flink 1.3.0 Release Announcement</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/05/16/official-docker-image.html">Introducing Docker Images for Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/04/26/release-1.2.1.html">Apache Flink 1.2.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/04/04/dynamic-tables.html">Continuous Queries on Dynamic Tables</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/03/29/table-sql-api-update.html">From Streams to Tables and Back Again: An Update on Flink's Table & SQL API</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/03/23/release-1.1.5.html">Apache Flink 1.1.5 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2017/02/06/release-1.2.0.html">Announcing Apache Flink 1.2.0</a></li>
+
+
+
+ </ul>
+ <hr>
+ <h2>2016</h2>
+ <ul id="markdown-toc">
+
+
+
+
+
+
+
+ <li><a href="/news/2016/12/21/release-1.1.4.html">Apache Flink 1.1.4 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/12/19/2016-year-in-review.html">Apache Flink in 2016: Year in Review</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/10/12/release-1.1.3.html">Apache Flink 1.1.3 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/09/05/release-1.1.2.html">Apache Flink 1.1.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/08/24/ff16-keynotes-panels.html">Flink Forward 2016: Announcing Schedule, Keynotes, and Panel Discussion</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/08/11/release-1.1.1.html">Flink 1.1.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/08/08/release-1.1.0.html">Announcing Apache Flink 1.1.0</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/05/24/stream-sql.html">Stream Processing for Everyone with SQL and Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/05/11/release-1.0.3.html">Flink 1.0.3 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/04/22/release-1.0.2.html">Flink 1.0.2 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/04/14/flink-forward-announce.html">Flink Forward 2016 Call for Submissions Is Now Open</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/04/06/cep-monitoring.html">Introducing Complex Event Processing (CEP) with Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/04/06/release-1.0.1.html">Flink 1.0.1 Released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/03/08/release-1.0.0.html">Announcing Apache Flink 1.0.0</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2016/02/11/release-0.10.2.html">Flink 0.10.2 Released</a></li>
+
+
+
+ </ul>
+ <hr>
+ <h2>2015</h2>
+ <ul id="markdown-toc">
+
+
+
+
+
+
+
+ <li><a href="/news/2015/12/18/a-year-in-review.html">Flink 2015: A year in review, and a lookout to 2016</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/12/11/storm-compatibility.html">Storm Compatibility in Apache Flink: How to run existing Storm topologies on Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/12/04/Introducing-windows.html">Introducing Stream Windows in Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/11/27/release-0.10.1.html">Flink 0.10.1 released</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/11/16/release-0.10.0.html">Announcing Apache Flink 0.10.0</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/09/16/off-heap-memory.html">Off-heap Memory in Apache Flink and the curious JIT compiler</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/09/03/flink-forward.html">Announcing Flink Forward 2015</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/09/01/release-0.9.1.html">Apache Flink 0.9.1 available</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/08/24/introducing-flink-gelly.html">Introducing Gelly: Graph Processing with Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/06/24/announcing-apache-flink-0.9.0-release.html">Announcing Apache Flink 0.9.0</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/05/14/Community-update-April.html">April 2015 in the Flink community</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/05/11/Juggling-with-Bits-and-Bytes.html">Juggling with Bits and Bytes</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/04/13/release-0.9.0-milestone1.html">Announcing Flink 0.9.0-milestone1 preview release</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/04/07/march-in-flink.html">March 2015 in the Flink community</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html">Peeking into Apache Flink's Engine Room</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/03/02/february-2015-in-flink.html">February 2015 in the Flink community</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/02/09/streaming-example.html">Introducing Flink Streaming</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/02/04/january-in-flink.html">January 2015 in the Flink community</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/01/21/release-0.8.html">Apache Flink 0.8.0 available</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2015/01/06/december-in-flink.html">December 2014 in the Flink community</a></li>
+
+
+
+ </ul>
+ <hr>
+ <h2>2014</h2>
+ <ul id="markdown-toc">
+
+
+
+
+
+
+
+ <li><a href="/news/2014/11/18/hadoop-compatibility.html">Hadoop Compatibility in Flink</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2014/11/04/release-0.7.0.html">Apache Flink 0.7.0 available</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2014/10/03/upcoming_events.html">Upcoming Events</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2014/09/26/release-0.6.1.html">Apache Flink 0.6.1 available</a></li>
+
+
+
+
+
+
+
+
+
+ <li><a href="/news/2014/08/26/release-0.6.html">Apache Flink 0.6 available</a></li>
+
+
+ </ul>
+
+
+ </div>
+</div>
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html
index 13dd7f5..5d63f15 100644
--- a/content/blog/page2/index.html
+++ b/content/blog/page2/index.html
@@ -187,6 +187,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1 Released</a></h2>
+
+ <p>18 Oct 2019
+ Jark Wu (<a href="https://twitter.com/JarkWu">@JarkWu</a>)</p>
+
+ <p><p>The Apache Flink community released the first bugfix version of the Apache Flink 1.9 series.</p>
+
+</p>
+
+ <p><a href="/news/2019/10/18/release-1.9.1.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/feature/2019/09/13/state-processor-api.html">The State Processor API: How to Read, write and modify the state of Flink applications</a></h2>
<p>13 Sep 2019
@@ -310,19 +325,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/2019/05/14/temporal-tables.html">Flux capacitor, huh? Temporal Tables and Joins in Streaming SQL</a></h2>
-
- <p>14 May 2019
- Marta Paes (<a href="https://twitter.com/morsapaes">@morsapaes</a>)</p>
-
- <p>Apache Flink natively supports temporal table joins since the 1.7 release for straightforward temporal data handling. In this blog post, we provide an overview of how this new concept can be leveraged for effective point-in-time analysis in streaming scenarios.</p>
-
- <p><a href="/2019/05/14/temporal-tables.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -333,7 +335,7 @@
</li>
<li>
- <span class="page_number ">Page: 2 of 10</span>
+ <span class="page_number ">Page: 2 of 11</span>
</li>
<li>
@@ -355,6 +357,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html
index 0b1cf3c..2bdf8a7 100644
--- a/content/blog/page3/index.html
+++ b/content/blog/page3/index.html
@@ -187,6 +187,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/2019/05/14/temporal-tables.html">Flux capacitor, huh? Temporal Tables and Joins in Streaming SQL</a></h2>
+
+ <p>14 May 2019
+ Marta Paes (<a href="https://twitter.com/morsapaes">@morsapaes</a>)</p>
+
+ <p>Apache Flink natively supports temporal table joins since the 1.7 release for straightforward temporal data handling. In this blog post, we provide an overview of how this new concept can be leveraged for effective point-in-time analysis in streaming scenarios.</p>
+
+ <p><a href="/2019/05/14/temporal-tables.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/2019/05/03/pulsar-flink.html">When Flink & Pulsar Come Together</a></h2>
<p>03 May 2019
@@ -315,21 +328,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2018/12/26/release-1.5.6.html">Apache Flink 1.5.6 Released</a></h2>
-
- <p>26 Dec 2018
- </p>
-
- <p><p>The Apache Flink community released the sixth and last bugfix version of the Apache Flink 1.5 series.</p>
-
-</p>
-
- <p><a href="/news/2018/12/26/release-1.5.6.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -340,7 +338,7 @@
</li>
<li>
- <span class="page_number ">Page: 3 of 10</span>
+ <span class="page_number ">Page: 3 of 11</span>
</li>
<li>
@@ -362,6 +360,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html
index 79f0272..0f37517 100644
--- a/content/blog/page4/index.html
+++ b/content/blog/page4/index.html
@@ -187,6 +187,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2018/12/26/release-1.5.6.html">Apache Flink 1.5.6 Released</a></h2>
+
+ <p>26 Dec 2018
+ </p>
+
+ <p><p>The Apache Flink community released the sixth and last bugfix version of the Apache Flink 1.5 series.</p>
+
+</p>
+
+ <p><a href="/news/2018/12/26/release-1.5.6.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2018/12/22/release-1.6.3.html">Apache Flink 1.6.3 Released</a></h2>
<p>22 Dec 2018
@@ -323,21 +338,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2018/07/31/release-1.5.2.html">Apache Flink 1.5.2 Released</a></h2>
-
- <p>31 Jul 2018
- </p>
-
- <p><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.5 series.</p>
-
-</p>
-
- <p><a href="/news/2018/07/31/release-1.5.2.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -348,7 +348,7 @@
</li>
<li>
- <span class="page_number ">Page: 4 of 10</span>
+ <span class="page_number ">Page: 4 of 11</span>
</li>
<li>
@@ -370,6 +370,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page5/index.html b/content/blog/page5/index.html
index 5cc9053..4abfa39 100644
--- a/content/blog/page5/index.html
+++ b/content/blog/page5/index.html
@@ -187,6 +187,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2018/07/31/release-1.5.2.html">Apache Flink 1.5.2 Released</a></h2>
+
+ <p>31 Jul 2018
+ </p>
+
+ <p><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.5 series.</p>
+
+</p>
+
+ <p><a href="/news/2018/07/31/release-1.5.2.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2018/07/12/release-1.5.1.html">Apache Flink 1.5.1 Released</a></h2>
<p>12 Jul 2018
@@ -317,24 +332,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2017/11/22/release-1.4-and-1.5-timeline.html">Looking Ahead to Apache Flink 1.4.0 and 1.5.0</a></h2>
-
- <p>22 Nov 2017
- Stephan Ewen (<a href="https://twitter.com/StephanEwen">@StephanEwen</a>), Aljoscha Krettek (<a href="https://twitter.com/aljoscha">@aljoscha</a>), & Mike Winters (<a href="https://twitter.com/wints">@wints</a>)</p>
-
- <p><p>The Apache Flink 1.4.0 release is on track to happen in the next couple of weeks, and for all of the
-readers out there who haven’t been following the release discussion on <a href="http://flink.apache.org/community.html#mailing-lists">Flink’s developer mailing
-list</a>, we’d like to provide some details on
-what’s coming in Flink 1.4.0 as well as a preview of what the Flink community will save for 1.5.0.</p>
-
-</p>
-
- <p><a href="/news/2017/11/22/release-1.4-and-1.5-timeline.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -345,7 +342,7 @@
</li>
<li>
- <span class="page_number ">Page: 5 of 10</span>
+ <span class="page_number ">Page: 5 of 11</span>
</li>
<li>
@@ -367,6 +364,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page6/index.html b/content/blog/page6/index.html
index b6bf1ea..5a8c216 100644
--- a/content/blog/page6/index.html
+++ b/content/blog/page6/index.html
@@ -187,6 +187,24 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2017/11/22/release-1.4-and-1.5-timeline.html">Looking Ahead to Apache Flink 1.4.0 and 1.5.0</a></h2>
+
+ <p>22 Nov 2017
+ Stephan Ewen (<a href="https://twitter.com/StephanEwen">@StephanEwen</a>), Aljoscha Krettek (<a href="https://twitter.com/aljoscha">@aljoscha</a>), & Mike Winters (<a href="https://twitter.com/wints">@wints</a>)</p>
+
+ <p><p>The Apache Flink 1.4.0 release is on track to happen in the next couple of weeks, and for all of the
+readers out there who haven’t been following the release discussion on <a href="http://flink.apache.org/community.html#mailing-lists">Flink’s developer mailing
+list</a>, we’d like to provide some details on
+what’s coming in Flink 1.4.0 as well as a preview of what the Flink community will save for 1.5.0.</p>
+
+</p>
+
+ <p><a href="/news/2017/11/22/release-1.4-and-1.5-timeline.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2017/08/05/release-1.3.2.html">Apache Flink 1.3.2 Released</a></h2>
<p>05 Aug 2017
@@ -316,19 +334,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2017/02/06/release-1.2.0.html">Announcing Apache Flink 1.2.0</a></h2>
-
- <p>06 Feb 2017 by Robert Metzger
- </p>
-
- <p><p>The Apache Flink community is excited to announce the 1.2.0 release.</p></p>
-
- <p><a href="/news/2017/02/06/release-1.2.0.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -339,7 +344,7 @@
</li>
<li>
- <span class="page_number ">Page: 6 of 10</span>
+ <span class="page_number ">Page: 6 of 11</span>
</li>
<li>
@@ -361,6 +366,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page7/index.html b/content/blog/page7/index.html
index b803a0c..b03aef7 100644
--- a/content/blog/page7/index.html
+++ b/content/blog/page7/index.html
@@ -187,6 +187,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2017/02/06/release-1.2.0.html">Announcing Apache Flink 1.2.0</a></h2>
+
+ <p>06 Feb 2017 by Robert Metzger
+ </p>
+
+ <p><p>The Apache Flink community is excited to announce the 1.2.0 release.</p></p>
+
+ <p><a href="/news/2017/02/06/release-1.2.0.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2016/12/21/release-1.1.4.html">Apache Flink 1.1.4 Released</a></h2>
<p>21 Dec 2016
@@ -318,21 +331,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2016/04/22/release-1.0.2.html">Flink 1.0.2 Released</a></h2>
-
- <p>22 Apr 2016
- </p>
-
- <p><p>Today, the Flink community released Flink version <strong>1.0.2</strong>, the second bugfix release of the 1.0 series.</p>
-
-</p>
-
- <p><a href="/news/2016/04/22/release-1.0.2.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -343,7 +341,7 @@
</li>
<li>
- <span class="page_number ">Page: 7 of 10</span>
+ <span class="page_number ">Page: 7 of 11</span>
</li>
<li>
@@ -365,6 +363,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page8/index.html b/content/blog/page8/index.html
index 78cf897..e1e19e8 100644
--- a/content/blog/page8/index.html
+++ b/content/blog/page8/index.html
@@ -187,6 +187,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2016/04/22/release-1.0.2.html">Flink 1.0.2 Released</a></h2>
+
+ <p>22 Apr 2016
+ </p>
+
+ <p><p>Today, the Flink community released Flink version <strong>1.0.2</strong>, the second bugfix release of the 1.0 series.</p>
+
+</p>
+
+ <p><a href="/news/2016/04/22/release-1.0.2.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2016/04/14/flink-forward-announce.html">Flink Forward 2016 Call for Submissions Is Now Open</a></h2>
<p>14 Apr 2016 by Aljoscha Krettek (<a href="https://twitter.com/">@aljoscha</a>)
@@ -314,21 +329,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2015/11/16/release-0.10.0.html">Announcing Apache Flink 0.10.0</a></h2>
-
- <p>16 Nov 2015
- </p>
-
- <p><p>The Apache Flink community is pleased to announce the availability of the 0.10.0 release. The community put significant effort into improving and extending Apache Flink since the last release, focusing on data stream processing and operational features. About 80 contributors provided bug fixes, improvements, and new features such that in total more than 400 JIRA issues could be resolved.</p>
-
-</p>
-
- <p><a href="/news/2015/11/16/release-0.10.0.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -339,7 +339,7 @@
</li>
<li>
- <span class="page_number ">Page: 8 of 10</span>
+ <span class="page_number ">Page: 8 of 11</span>
</li>
<li>
@@ -361,6 +361,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/blog/page9/index.html b/content/blog/page9/index.html
index df5157a..f2bbaff 100644
--- a/content/blog/page9/index.html
+++ b/content/blog/page9/index.html
@@ -187,6 +187,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2015/11/16/release-0.10.0.html">Announcing Apache Flink 0.10.0</a></h2>
+
+ <p>16 Nov 2015
+ </p>
+
+ <p><p>The Apache Flink community is pleased to announce the availability of the 0.10.0 release. The community put significant effort into improving and extending Apache Flink since the last release, focusing on data stream processing and operational features. About 80 contributors provided bug fixes, improvements, and new features such that in total more than 400 JIRA issues could be resolved.</p>
+
+</p>
+
+ <p><a href="/news/2015/11/16/release-0.10.0.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2015/09/16/off-heap-memory.html">Off-heap Memory in Apache Flink and the curious JIT compiler</a></h2>
<p>16 Sep 2015 by Stephan Ewen (<a href="https://twitter.com/">@stephanewen</a>)
@@ -329,19 +344,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html">Peeking into Apache Flink's Engine Room</a></h2>
-
- <p>13 Mar 2015 by Fabian Hüske (<a href="https://twitter.com/">@fhueske</a>)
- </p>
-
- <p>Joins are prevalent operations in many data processing applications. Most data processing systems feature APIs that make joining data sets very easy. However, the internal algorithms for join processing are much more involved – especially if large data sets need to be efficiently handled. In this blog post, we cut through Apache Flink’s layered architecture and take a look at its internals with a focus on how it handles joins.</p>
-
- <p><a href="/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -352,7 +354,7 @@
</li>
<li>
- <span class="page_number ">Page: 9 of 10</span>
+ <span class="page_number ">Page: 9 of 11</span>
</li>
<li>
@@ -374,6 +376,16 @@
<ul id="markdown-toc">
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
diff --git a/content/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html b/content/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html
new file mode 100644
index 0000000..84f5f21
--- /dev/null
+++ b/content/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html
@@ -0,0 +1,388 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Apache Beam: How Beam Runs on Top of Flink</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache Flink?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/faq.html">FAQ</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Getting Started -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.10/getting-started/index.html" target="_blank">Getting Started <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.10" target="_blank">1.10 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+ <!-- Flink-packages -->
+ <li>
+ <a href="https://flink-packages.org" target="_blank">flink-packages.org <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <a href="/zh/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <hr />
+
+ <li><a href="https://apache.org" target="_blank">Apache Software Foundation <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <li>
+ <style>
+ .smalllinks:link {
+ display: inline-block !important; background: none; padding-top: 0px; padding-bottom: 0px; padding-right: 0px; min-width: 75px;
+ }
+ </style>
+
+ <a class="smalllinks" href="https://www.apache.org/licenses/" target="_blank">License</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/security/" target="_blank">Security</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Donate</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+ </li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <div class="row">
+ <h1>Apache Beam: How Beam Runs on Top of Flink</h1>
+
+ <article>
+ <p>22 Feb 2020 Maximilian Michels (<a href="https://twitter.com/stadtlegende">@stadtlegende</a>) & Markos Sfikas (<a href="https://twitter.com/MarkSfik">@MarkSfik</a>)</p>
+
+<p>Note: This blog post is based on the talk <a href="https://www.youtube.com/watch?v=hxHGLrshnCY">“Beam on Flink: How Does It Actually Work?”</a>.</p>
+
+<p><a href="https://flink.apache.org/">Apache Flink</a> and <a href="https://beam.apache.org/">Apache Beam</a> are open-source frameworks for parallel, distributed data processing at scale. Unlike Flink, Beam does not come with a full-blown execution engine of its own but plugs into other execution engines, such as Apache Flink, Apache Spark, or Google Cloud Dataflow. In this blog post we discuss the reasons to use Flink together with Beam for your batch and stream processing needs. We also take a closer look at how Beam works with Flink to provide an idea of the technical aspects of running Beam pipelines with Flink. We hope you find some useful information on how and why the two frameworks can be utilized in combination. For more information, you can refer to the corresponding <a href="https://beam.apache.org/documentation/runners/flink/">documentation</a> on the Beam website or contact the community through the <a href="https://beam.apache.org/community/contact-us/">Beam mailing list</a>.</p>
+
+<h1 id="what-is-apache-beam">What is Apache Beam</h1>
+
+<p><a href="https://beam.apache.org/">Apache Beam</a> is an open-source, unified model for defining batch and streaming data-parallel processing pipelines. It is unified in the sense that you use a single API, in contrast to using a separate API for batch and streaming like it is the case in Flink. Beam was originally developed by Google which released it in 2014 as the Cloud Dataflow SDK. In 2016, it was donated to <a href="https://www.apache.org/">the Apache Software Foundation</a> with the name of Beam. It has been developed by the open-source community ever since. With Apache Beam, developers can write data processing jobs, also known as pipelines, in multiple languages, e.g. Java, Python, Go, SQL. A pipeline is then executed by one of Beam’s Runners. A Runner is responsible for translating Beam pipelines such that they can run on an execution engine. Every supported execution engine has a Runner. The following Runners are available: Apache Flink, Apache Spark, Apache Samza, Hazelcast Jet, Google Cloud Dataflow, and others.</p>
+
+<p>The execution model, as well as the API of Apache Beam, are similar to Flink’s. Both frameworks are inspired by the <a href="https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf">MapReduce</a>, <a href="https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41378.pdf">MillWheel</a>, and <a href="https://research.google/pubs/pub43864/">Dataflow</a> papers. Like Flink, Beam is designed for parallel, distributed data processing. Both have similar transformations, support for windowing, event/processing time, watermarks, timers, triggers, and much more. However, Beam not being a full runtime focuses on providing the framework for building portable, multi-language batch and stream processing pipelines such that they can be run across several execution engines. The idea is that you write your pipeline once and feed it with either batch or streaming data. When you run it, you just pick one of the supported backends to execute. A large integration test suite in Beam called “ValidatesRunner” ensures that the results will be the same, regardless of which backend you choose for the execution.</p>
+
+<p>One of the most exciting developments in the Beam technology is the framework’s support for multiple programming languages including Java, Python, Go, Scala and SQL. Essentially, developers can write their applications in a programming language of their choice. Beam, with the help of the Runners, translates the program to one of the execution engines, as shown in the diagram below.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-beam-vision.png" width="600px" alt="The vision of Apache Beam" />
+</center>
+
+<h1 id="reasons-to-use-beam-with-flink">Reasons to use Beam with Flink</h1>
+
+<p>Why would you want to use Beam with Flink instead of directly using Flink? Ultimately, Beam and Flink complement each other and provide additional value to the user. The main reasons for using Beam with Flink are the following:</p>
+
+<ul>
+ <li>Beam provides a unified API for both batch and streaming scenarios.</li>
+ <li>Beam comes with native support for different programming languages, like Python or Go with all their libraries like Numpy, Pandas, Tensorflow, or TFX.</li>
+ <li>You get the power of Apache Flink like its exactly-once semantics, strong memory management and robustness.</li>
+ <li>Beam programs run on your existing Flink infrastructure or infrastructure for other supported Runners, like Spark or Google Cloud Dataflow.</li>
+ <li>You get additional features like side inputs and cross-language pipelines that are not supported natively in Flink but only supported when using Beam with Flink.</li>
+</ul>
+
+<h1 id="the-flink-runner-in-beam">The Flink Runner in Beam</h1>
+
+<p>The Flink Runner in Beam translates Beam pipelines into Flink jobs. The translation can be parameterized using Beam’s pipeline options which are parameters for settings like configuring the job name, parallelism, checkpointing, or metrics reporting.</p>
+
+<p>If you are familiar with a DataSet or a DataStream, you will have no problems understanding what a PCollection is. PCollection stands for parallel collection in Beam and is exactly what DataSet/DataStream would be in Flink. Due to Beam’s unified API we only have one type of results of transformation: PCollection.</p>
+
+<p>Beam pipelines are composed of transforms. Transforms are like operators in Flink and come in two flavors: primitive and composite transforms. The beauty of all this is that Beam only comes with a small set of primitive transforms which are:</p>
+
+<ul>
+ <li><code>Source</code> (for loading data)</li>
+ <li><code>ParDo</code> (think of a flat map operator on steroids)</li>
+ <li><code>GroupByKey</code> (think of keyBy() in Flink)</li>
+ <li><code>AssignWindows</code> (windows can be assigned at any point in time in Beam)</li>
+ <li><code>Flatten</code> (like a union() operation in Flink)</li>
+</ul>
+
+<p>Composite transforms are built by combining the above primitive transforms. For example, <code>Combine = GroupByKey + ParDo</code>.</p>
+
+<h1 id="flink-runner-internals">Flink Runner Internals</h1>
+
+<p>Although using the Flink Runner in Beam has no prerequisite to understanding its internals, we provide more details of how the Flink runner works in Beam to share knowledge of how the two frameworks can integrate and work together to provide state-of-the-art streaming data pipelines.</p>
+
+<p>The Flink Runner has two translation paths. Depending on whether we execute in batch or streaming mode, the Runner either translates into Flink’s DataSet or into Flink’s DataStream API. Since multi-language support has been added to Beam, another two translation paths have been added. To summarize the four modes:</p>
+
+<ol>
+ <li><strong>The Classic Flink Runner for batch jobs:</strong> Executes batch Java pipelines</li>
+ <li><strong>The Classic Flink Runner for streaming jobs:</strong> Executes streaming Java pipelines</li>
+ <li><strong>The Portable Flink Runner for batch jobs:</strong> Executes Java as well as Python, Go and other supported SDK pipelines for batch scenarios</li>
+ <li><strong>The Portable Flink Runner for streaming jobs:</strong> Executes Java as well as Python, Go and other supported SDK pipelines for streaming scenarios</li>
+</ol>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-runner-translation-paths.png" width="300px" alt="The 4 translation paths in the Beam's Flink Runner" />
+</center>
+
+<h2 id="the-classic-flink-runner-in-beam">The “Classic” Flink Runner in Beam</h2>
+
+<p>The classic Flink Runner was the initial version of the Runner, hence the “classic” name. Beam pipelines are represented as a graph in Java which is composed of the aforementioned composite and primitive transforms. Beam provides translators which traverse the graph in topological order. Topological order means that we start from all the sources first as we iterate through the graph. Presented with a transform from the graph, the Flink Runner generates the API calls as you would normally when writing a Flink job.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/classic-flink-runner-beam.png" width="600px" alt="The Classic Flink Runner in Beam" />
+</center>
+
+<p>While Beam and Flink share very similar concepts, there are enough differences between the two frameworks that make Beam pipelines impossible to be translated 1:1 into a Flink program. In the following sections, we will present the key differences:</p>
+
+<h3 id="serializers-vs-coders">Serializers vs Coders</h3>
+
+<p>When data is transferred over the wire in Flink, it has to be turned into bytes. This is done with the help of serializers. Flink has a type system to instantiate the correct coder for a given type, e.g. <code>StringTypeSerializer</code> for a String. Apache Beam also has its own type system which is similar to Flink’s but uses slightly different interfaces. Serializers are called Coders in Beam. In order to make a Beam Coder run in Flink, we have to make the two serializer types compatible. This is done by creating a special Flink type information that looks like the one in Flink but calls the appropriate Beam coder. That way, we can use Beam’s coders although we are executing the Beam job with Flink. Flink operators expect a TypeInformation, e.g. <code>StringTypeInformation</code>, for which we use a <code>CoderTypeInformation</code> in Beam. The type information returns the serializer for which we return a <code>CoderTypeSerializer</code>, which calls the underlying Beam Coder.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-serializers-coders.png" width="300px" alt="Serializers vs Coders" />
+</center>
+
+<h3 id="read">Read</h3>
+
+<p>The <code>Read</code> transform provides a way to read data into your pipeline in Beam. The Read transform is supported by two wrappers in Beam, the <code>SourceInputFormat</code> for batch processing and the <code>UnboundedSourceWrapper</code> for stream processing.</p>
+
+<h3 id="pardo">ParDo</h3>
+
+<p><code>ParDo</code> is the swiss army knife of Beam and can be compared to a <code>RichFlatMapFunction</code> in Flink with additional features such as <code>SideInputs</code>, <code>SideOutputs</code>, State and Timers. <code>ParDo</code> is essentially translated by the Flink runner using the <code>FlinkDoFnFunction</code> for batch processing or the <code>FlinkStatefulDoFnFunction</code>, while for streaming scenarios the translation is executed with the <code>DoFnOperator</code> that takes care of checkpointing and buffering of data during checkpoints, watermark emissions and maintenance of state and timers. This is all executed by Beam’s interface, called the <code>DoFnRunner</code>, that encapsulates Beam-specific execution logic, like retrieving state, executing state and timers, or reporting metrics.</p>
+
+<h3 id="side-inputs">Side Inputs</h3>
+
+<p>In addition to the main input, ParDo transforms can have a number of side inputs. A side input can be a static set of data that you want to have available at all parallel instances. However, it is more flexible than that. You can have keyed and even windowed side input which updates based on the window size. This is a very powerful concept which does not exist in Flink but is added on top of Flink using Beam.</p>
+
+<h3 id="assignwindows">AssignWindows</h3>
+
+<p>In Flink, windows are assigned by the <code>WindowOperator</code> when you use the <code>window()</code> in the API. In Beam, windows can be assigned at any point in time. Any element is implicitly part of a window. If no window is assigned explicitly, the element is part of the <code>GlobalWindow</code>. Window information is stored for each element in a wrapper called <code>WindowedValue</code>. The window information is only used once we issue a <code>GroupByKey</code>.</p>
+
+<h3 id="groupbykey">GroupByKey</h3>
+
+<p>Most of the time it is useful to partition the data by a key. In Flink, this is done via the <code>keyBy()</code> API call. In Beam the <code>GroupByKey</code> transform can only be applied if the input is of the form <code>KV<Key, Value></code>. Unlike Flink where the key can even be nested inside the data, Beam enforces the key to always be explicit. The <code>GroupByKey</code> transform then groups the data by key and by window which is similar to what <code>keyBy(..).window(..)</code> would give us in Flink. Beam has its own set of libraries to do that because Beam has its own set of window functions and triggers. Essentially, GroupByKey is very similar to what the WindowOperator does in Flink.</p>
+
+<h3 id="flatten">Flatten</h3>
+
+<p>The Flatten operator takes multiple DataSet/DataStreams, called P[arallel]Collections in Beam, and combines them into one collection. This is equivalent to Flink’s <code>union()</code> operation.</p>
+
+<h2 id="the-portable-flink-runner-in-beam">The “Portable” Flink Runner in Beam</h2>
+
+<p>The portable Flink Runner in Beam is the evolution of the classic Runner. Classic Runners are tied to the JVM ecosystem, but the Beam community wanted to move past this and also execute Python, Go and other languages. This adds another dimension to Beam in terms of portability because, like previously mentioned, Beam already had portability across execution engines. It was necessary to change the translation logic of the Runner to be able to support language portability.</p>
+
+<p>There are two important building blocks for portable Runners:</p>
+
+<ol>
+ <li>A common pipeline format across all the languages: The Runner API</li>
+ <li>A common interface during execution for the communication between the Runner and the code written in any language: The Fn API</li>
+</ol>
+
+<p>The Runner API provides a universal representation of the pipeline as Protobuf which contains the transforms, types, and user code. Protobuf was chosen as the format because every language has libraries available for it. Similarly, for the execution part, Beam introduced the Fn API interface to handle the communication between the Runner/execution engine and the user code that may be written in a different language and executes in a different process. Fn API is pronounced “fun API”, you may guess why.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability.png" width="600px" alt="Language Portability in Apache Beam" />
+</center>
+
+<h2 id="how-are-beam-programs-translated-in-language-portability">How Are Beam Programs Translated In Language Portability?</h2>
+
+<p>Users write their Beam pipelines in one language, but they may get executed in an environment based on a completely different language. How does that work? To explain that, let’s follow the lifecycle of a pipeline. Let’s suppose we use the Python SDK to write the pipeline. Before submitting the pipeline via the Job API to Beam’s JobServer, Beam would convert it to the Runner API, the language-agnostic format we described before. The JobServer is also a Beam component that handles the staging of the required dependencies during execution. The JobServer will then kick-off the translation which is similar to the classic Runner. However, an important change is the so-called <code>ExecutableStage</code> transform. It is essentially a ParDo transform that we already know but designed for holding language-dependent code. Beam tries to combine as many of these transforms into one “executable stage”. The result again is a Flink program which is then sent to the Flink cluster and executed there. The major difference compared to the classic Runner is that during execution we will start <em>environments</em> to execute the aforementioned <em>ExecutableStages</em>. The following environments are available:</p>
+
+<ul>
+ <li>Docker-based (the default)</li>
+ <li>Process-based (a simple process is started)</li>
+ <li>Externally-provided (K8s or other schedulers)</li>
+ <li>Embedded (intended for testing and only works with Java)</li>
+</ul>
+
+<p>Environments hold the <em>SDK Harness</em> which is the code that handles the execution and the communication with the Runner over the Fn API. For example, when Flink executes Python code, it sends the data to the Python environment containing the Python SDK Harness. Sending data to an external process involves a minor overhead which we have measured to be 5-10% slower than the classic Java pipelines. However, Beam uses a fusion of transforms to execute as many transforms as possible in the same environment which share the same input or output. That’s why in real-world scenarios the overhead could be much lower.</p>
+
+<center>
+<img src="/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability-architecture.png" width="600px" alt="Language Portability Architecture in beam" />
+</center>
+
+<p>Environments can be present for many languages. This opens up an entirely new type of pipelines: cross-language pipelines. In cross-language pipelines we can combine transforms of two or more languages, e.g. a machine learning pipeline with the feature generation written in Java and the learning written in Python. All this can be run on top of Flink.</p>
+
+<h2 id="conclusion">Conclusion</h2>
+
+<p>Using Apache Beam with Apache Flink combines (a.) the power of Flink with (b.) the flexibility of Beam. All it takes to run Beam is a Flink cluster, which you may already have. Apache Beam’s fully-fledged Python API is probably the most compelling argument for using Beam with Flink, but the unified API which allows to “write-once” and “execute-anywhere” is also very appealing to Beam users. On top of this, features like side inputs and a rich connector ecosystem are also reasons why people like Beam.</p>
+
+<p>With the introduction of schemas, a new format for handling type information, Beam is heading in a similar direction as Flink with its type system which is essential for the Table API or SQL. Speaking of, the next Flink release will include a Python version of the Table API which is based on the language portability of Beam. Looking ahead, the Beam community plans to extend the support for interactive programs like notebooks. TFX, which is built with Beam, is a very powerful way to solve many problems around training and validating machine learning models.</p>
+
+<p>For many years, Beam and Flink have inspired and learned from each other. With the Python support being based on Beam in Flink, they only seem to come closer to each other. That’s all the better for the community, and also users have more options and functionality to choose from.</p>
+
+ </article>
+ </div>
+
+ <div class="row">
+ <div id="disqus_thread"></div>
+ <script type="text/javascript">
+ /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */
+ var disqus_shortname = 'stratosphere-eu'; // required: replace example with your forum shortname
+
+ /* * * DON'T EDIT BELOW THIS LINE * * */
+ (function() {
+ var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
+ dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
+ (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
+ })();
+ </script>
+ </div>
+ </div>
+</div>
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/img/blog/2020-02-22-beam-on-flink/classic-flink-runner-beam.png b/content/img/blog/2020-02-22-beam-on-flink/classic-flink-runner-beam.png
new file mode 100644
index 0000000..39c47a5
--- /dev/null
+++ b/content/img/blog/2020-02-22-beam-on-flink/classic-flink-runner-beam.png
Binary files differ
diff --git a/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-beam-vision.png b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-beam-vision.png
new file mode 100644
index 0000000..51bc741
--- /dev/null
+++ b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-beam-vision.png
Binary files differ
diff --git a/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability-architecture.png b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability-architecture.png
new file mode 100644
index 0000000..a987d4b
--- /dev/null
+++ b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability-architecture.png
Binary files differ
diff --git a/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability.png b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability.png
new file mode 100644
index 0000000..628d607
--- /dev/null
+++ b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-language-portability.png
Binary files differ
diff --git a/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-runner-translation-paths.png b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-runner-translation-paths.png
new file mode 100644
index 0000000..ca80ada
--- /dev/null
+++ b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-runner-translation-paths.png
Binary files differ
diff --git a/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-serializers-coders.png b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-serializers-coders.png
new file mode 100644
index 0000000..014bbef
--- /dev/null
+++ b/content/img/blog/2020-02-22-beam-on-flink/flink-runner-beam-serializers-coders.png
Binary files differ
diff --git a/content/index.html b/content/index.html
index bf6e655..0122b06 100644
--- a/content/index.html
+++ b/content/index.html
@@ -559,6 +559,9 @@
<dl>
+ <dt> <a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></dt>
+ <dd>This blog post discusses the reasons to use Flink together with Beam for your stream processing needs and takes a closer look at how Flink works with Beam under the hood.</dd>
+
<dt> <a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></dt>
<dd>This post discusses the efforts of the Flink community as they relate to end to end applications with SQL in Apache Flink.</dd>
@@ -574,9 +577,6 @@
<dd><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.9 series.</p>
</dd>
-
- <dt> <a href="/news/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink.html">State Unlocked: Interacting with State in Apache Flink</a></dt>
- <dd>This post discusses the efforts of the Flink community as they relate to state management in Apache Flink. We showcase some practical examples of how the different features and APIs can be utilized and cover some future ideas for new and improved ways of managing state in Apache Flink.</dd>
</dl>
diff --git a/content/zh/index.html b/content/zh/index.html
index 6971060..3a81092 100644
--- a/content/zh/index.html
+++ b/content/zh/index.html
@@ -556,6 +556,9 @@
<dl>
+ <dt> <a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></dt>
+ <dd>This blog post discusses the reasons to use Flink together with Beam for your stream processing needs and takes a closer look at how Flink works with Beam under the hood.</dd>
+
<dt> <a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></dt>
<dd>This post discusses the efforts of the Flink community as they relate to end to end applications with SQL in Apache Flink.</dd>
@@ -571,9 +574,6 @@
<dd><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.9 series.</p>
</dd>
-
- <dt> <a href="/news/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink.html">State Unlocked: Interacting with State in Apache Flink</a></dt>
- <dd>This post discusses the efforts of the Flink community as they relate to state management in Apache Flink. We showcase some practical examples of how the different features and APIs can be utilized and cover some future ideas for new and improved ways of managing state in Apache Flink.</dd>
</dl>
