Date: 2020-02-27
Accepted (lazy consensus) & implemented
Messages are denormalized in Cassandra in order to:
Here is the table organisation:
messageIdTable
Holds mailbox and flags for each message, lookup by mailbox ID + UIDimapUidTable
Holds mailbox and flags for each message, lookup by message IDFailures during the denormalization process will lead to inconsistencies between the two tables.
This can lead to the following user experience:
BOB receives a message The denormalization process fails BOB can read the message via JMAP BOB cannot read the message via IMAP BOB marks a message as SEEN The denormalization process fails The message is SEEN in JMAP The message is UNSEEN in IMAP
messageIdTable
then in imapUidTable
.imapUidTable
then in messageIdTable
.imapUidTable
then in messageIdTable
.imapUidTable
, then first delete the message in imapUidTable
then in messageIdTable
.messageIdTable
then in imapUidTable
.imapUidTable
then in messageIdTable
.imapUidTable
then in messageIdTable
.imapUidTable
then in messageIdTable
.Adopt imapUidTable
as a source of truth. Because messageId
allows tracking changes to messages accross mailboxes upon copy and moves. Furthermore, that is the table on which conditional flags updates are performed.
All writes will be performed to imapUidTable
then performed on messageIdTable
if successful.
We thus need to modify CassandraMessageMapper ‘add’ + ‘copy’ to first write to the source of truth (imapUidTable
)
We can adopt a retry policy of the messageIdTable
projection update as a mitigation strategy.
Using imapUidTable
table as a source of truth, we can rebuild the messageIdTable
projection:
imapUidTable
entries, we can rewrite entries in messageIdTable
messageIdTable
we can remove entries not referenced in imapUidTable
We will expose a webAdmin task for doing this.
User actions concurrent to the inconsistency fixing task could result in concurrency issues. New inconsistencies could be created. However table of truth would not be impacted hence rerunning the inconsistency fixing task will eventually fix all issues.
This task could be run safely online and can be scheduled on a recurring basis outside of peak traffic by an admin to ensure Cassandra message consistency.
The delay strategy to decrease concurrency issue occurrence is described here.