tree 8a36821836e6d47a518bf0305b190042622e1adb
parent 9b0936a25e9fef138874ae059f3d671f83feee81
author Justin Berstler <bjustin@us.ibm.com> 1495743211 -0400
committer Carlos Santana <csantana23@gmail.com> 1495743211 -0400

Implement provider redundancy (#160)

This change allows multiple instances of the feed provider to run simultaneously for the purpose of failover redundancy, and even a little load balancing.

This is accomplished by allowing the feed action to write triggers directly to the DB, without having to converse directly with the feed provider instances. The provider instances pick up trigger changes by using a CouchDB changes feed to detect new, modified, and deleted triggers.

To allow the feed action to talk directly to the DB, without exposing the DB credentials to all users, a new web action is introduced. This action can be invoked by any user, but cannot be inspected by anyone other than the action owner (typically a system admin account). The non-web feed action is still added to a shared package, allowing all users to create triggers from it, but this action only invokes the new web action over a REST call.

Even though multiple instances of the provider are running and, in fact, handling the same set of triggers, the use of kafka consumer groups ensures that each trigger is only fired once per produced message by way of the fact that Kafka guarantees that only one consumer in the group will get each produced message.