title: “Delegation Token Framework: Obtain, Distribute and Use Temporary Credentials Automatically” date: “2023-01-20T08:00:00.000Z” authors:
The Apache Flink Community is pleased to announce that the upcoming minor version of Flink (1.17) includes the Delegation Token Framework proposed in FLIP-272. This enables Flink to authenticate to external services at a central location (JobManager) and distribute authentication tokens to the TaskManagers.
Authentication in distributed systems is not an easy task. Previously all worker nodes (TaskManagers) reading from or writing to an external system needed to authenticate on their own. In such a case several things can go wrong, including but not limited to:
The aim of Delegation Token Framework is to solve the above challenges. The framework is authentication protocol agnostic and pluggable. The primary design concept is that authentication happens only at a single location (JobManager), the obtained temporary credentials propagated automatically to all the task managers where they can be used. The token re-obtain process is also handled in the JobManager.
New authentication providers can be added with small amount of code which is going to be loaded by Flink automatically. At the moment the following external systems are supported:
Planned, but not yet implemented/contributed:
The design and implementation approach has already been proven in Apache Spark. Gabor is a Spark committer, he championed this feature in the Spark community. The most notable improvement we achieved compared to the current state in Spark is that the framework in Flink is already authentication protocol agnostic (and not bound to Kerberos).
For more details please refer to the following documentation:
Major tickets where the framework has been added:
Adding a new authentication protocol is relatively straight-forward:
FlinkTestJavaDelegationTokenProvider.obtainDelegationTokens
to obtain a custom token from any external serviceFlinkTestJavaDelegationTokenReceiver.onNewTokensObtained
to receive the previously obtained tokens on all task managersThe existing providers are tested with the Flink Kubernetes Operator but one can use any other supported deployment model, because the framework is not bound to any of them. We choose the Kubernetes Operator so that we could provide a completely containerized and easily reproducible test environment.
An example tutorial can be found here on external system authentication.
The Delegation Token Framework is feature complete on the master branch and is becoming generally available on the release of Flink 1.17. The framework obtains authentication tokens at a central location and propagates them to all workers on a re-occurring basis.
Any connector to an external system which supports authentication can be a potential user of this framework. To support authentication in your connector we encourage you to implement your own DelegationTokenProvider/DelegationTokenReceiver
pair.