tree 76c77eb28ed836ef8759f99a5c674ced9278995d
parent 5a099ebae130079e71c9b26797b41fe6e9252898
author mynameborat <bharath.kumarasubramanian@gmail.com> 1607454488 -0800
committer GitHub <noreply@github.com> 1607454488 -0800
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJfz88YCRBK7hj4Ov3rIwAAdHIIAJHV+JbUSEJKQmpKHqT6akXb
 GbD1FEO5BdiyvaGlTYk6SldvFmuTjEy4PTL995nh2NJSCONNf8/jNWPkCHU35N0/
 0WGXcP2p/2btnvoFIg1kUhLd1LStV1LWQgzvdkSjlftPGSth7Hbj2lX0FAb+T0ff
 lMP5al9rKCaYHOTN+qq+z7SVR3/yHMnBUotxUoK447H6N7WpoufObcSu+SE9yAf3
 zB1Kh1V3ZJnu7MJgwaFcvjjlKwLZtjX8ap2NuGd/qBRb/4uIYohphPEeG33InQJg
 r7lsKVPSmqoL6pdFQXN/TmqgKlHycB+clBsRfJAV81USfkPv/CEPy9/lbNmkMwA=
 =bvVZ
 -----END PGP SIGNATURE-----
 

SAMZA-2609: Data model changes to track lifecycle information of job coordinator (#1449)

Description:
With AM HA, the new AM will regenerate job model, do job planning and rewrite configurations and may have to spin up containers. The new rewritten configurations, job model can change from the existing metadata on which the containers are running. In order to handle this change of metadata, we need a way to identify lifecycle of the AM and store some information on the previous snapshot of the metadata.

Changes:
Epoch identifier of the AM attempt so that we can determine if the new attempt is part of the same lifecycle of AM or a new deployment lifecycle
Metadata on the job model, configuration so, that we can use this information to decide the course of action if current metadata changed in comparison with the previous snapshot.
Metrics capturing the changes in metadata
Coordinator stream message updates for new metadata
The code path as is not exercised actively and will be leveraged by the PRs to follow.

API Changes:
Added new Coordinator stream message type
Introduced data model for job coordinator metadata internal to samza