layout: post title: “Flink Community Update - September'19” date: 2019-09-10T12:00:00.000Z categories: news authors:
This has been an exciting, fast-paced year for the Apache Flink community. But with over 10k messages across the mailing lists, 3k Jira tickets and 2k pull requests, it is not easy to keep up with the latest state of the project. Plus everything happening around it. With that in mind, we want to bring back regular community updates to the Flink blog.
The first post in the series takes you on an little detour across the year, to freshen up and make sure you're all up to date.
{% toc %}
Two major versions were released this year: Flink 1.8 and Flink 1.9; paving the way for the goal of making Flink the first framework to seamlessly support stream and batch processing with a single, unified runtime. The contribution of Blink to Apache Flink was key in accelerating the path to this vision and reduced the waiting time for long-pending user requests — such as Hive integration, (better) Python support, the rework of Flink's Machine Learning library and...fine-grained failure recovery (FLIP-1).
The 1.9 release was the result of the biggest community effort the project has experienced so far, with the number of contributors soaring to 190 (see The Bigger Picture). For a quick overview of the upcoming work for Flink 1.10 (and beyond), have a look at the updated roadmap!
As the number of Chinese-speaking Flink users rapidly grows, the community is working on translating resources and creating dedicated spaces for discussion to invite and include these users in the wider Flink community. Part of the ongoing work is described in FLIP-35 and has resulted in:
In case you‘re interested in knowing more about this work in progress, Robert Metzger and Fabian Hueske will be diving into "Inviting Apache Flink’s Chinese User Community" at the upcoming ApacheCon Europe 2019 (see Upcoming Flink Community Events).
Besides the translation effort, the community has also been working quite hard on a Flink docs overhaul. The main goals are to:
Given that there has been some confusion in the past regarding unclear definition of core Flink concepts, one of the first completed efforts was to introduce a Glossary in the docs. To get up to speed with the roadmap for the remainder efforts, you can refer to FLIP-42 and the corresponding umbrella Jira ticket.
The guidelines to contribute to Apache Flink have been reworked on the website, in an effort to lower the entry barrier for new contributors and reduce the overall friction in the contribution process. In addition, the Flink community discussed and adopted bylaws to help the community collaborate and coordinate more smoothly.
For code contributors, a Code Style and Quality Guide that captures the expected standards for contributions was also added to the “Contributing” section of the Flink website.
It's important to stress that contributions are not restricted to code. Non-code contributions such as mailing list support, documentation work or organization of community events are equally as important to the development of the project and highly encouraged.
The Apache Flink community has welcomed 5 new Committers and 4 PMC (Project Management Committee) Members in 2019, so far:
Jincheng Sun, Kete (Kurt) Young, Kostas Kloudas, Thomas Weise
Andrey Zagrebin, Hequn, Jiangjie (Becket) Qin, Rong Rong, Zhijiang Wang
Congratulations and thank you for your hardworking commitment to Flink!
Flink continues to push the boundaries of (stream) data processing, and the community is proud to see an ever-increasingly diverse set of contributors, users and technologies join the ecosystem.
In the timeframe of three releases, the project jumped from 112 to 190 contributors, also doubling down on the number of requested changes and improvements. To top it off, the Flink GitHub repository recently reached the milestone of 10k stars, all the way up from the incubation days in 2014.
The activity across the user@ and dev@1 mailing lists shows a healthy heartbeat, and the gradual ramp up of user-zh@ suggests that this was a well-received community effort. Looking at the numbers for the same period in 2018, the dev@ mailing list has seen the biggest surge in activity, with an average growth of 2.5x in the number of messages and distinct users — a great reflection of the hyperactive pace of development of the Flink codebase.
In support of these observations, the report for the financial year of 2019 from the Apache Software Foundation (ASF) features Flink as one of the most thriving open source projects, with mentions for:
Hats off to our fellows at Apache Beam for an astounding year, too! For more detailed insights, check the full report.
As the conference and meetup season ramps up again, here are some events to keep an eye out for talks about Flink and opportunities to mingle with the wider stream processing community.
If you‘d like to keep a closer eye on what’s happening in the community, subscribe to the community mailing list to get fine-grained weekly updates, upcoming event announcements and more. Also, please reach out if you’re interested in organizing or being part of Flink events in your area!