KeyedProcessFunction
and Timers (Long Ride Alerts)(Discussion of Lab: KeyedProcessFunction
and Timers (Long Ride Alerts))
These cases are worth noting:
These leaks could be addressed by either using state TTL, or another timer, to eventually clear any lingering state.
Regardless of how clever we are with what state we keep, and how long we choose to keep it, we should eventually clear it -- because otherwise our state will grow in an unbounded fashion. And having lost that information, we will run the risk of late events causing incorrect or duplicated results.
This tradeoff between keeping state indefinitely versus occasionally getting things wrong when events are late is a challenge that is inherent to stateful stream processing.
For each of these, add tests to check for the desired behavior.