Date: 2020-04-13
Accepted (lazy consensus) & implemented
Some mailbox implementations of James store already parsed attachments for faster retrieval.
This attachment storage capabilities are required for two features:
Only Memory and Cassandra backends can be relied upon as a JMAP backend.
Other protocols relies on dynamic EML parsing to expose message subparts (IMAP)
Here are the POJOs related to these attachments:
The following classes work with the aforementioned POJOs:
This organisation causes attachment content to be loaded every time a message is fully read (which happens for instance when you open a message using JMAP) despite the fact that it is not needed, as attachments are downloadable through a separate JMAP endpoint, their content is not attached to the JMAP message JSON.
Also, the content being loaded “at once”, we allocate memory space to store the whole attachment, which is sub-optimal. We want to keep the consumed memory low per-message because a given server should be able to handle a high number of messages at a given time.
To be noted that JPA and maildir mailbox implementations do not support attachment storage. To retrieve attachments of a message, these implementations parse the messages to extract their attachments.
Cassandra mailbox prior schema version 4 stored attachment and its metadata in the same table, but from version 5 relies on the blobStore to store the attachment content.
Enforce cassandra schema version to be 5 from James release 3.5.0. This allows to drop attachment management prior version 5.
We will re-organize the attachment POJOs:
Supplier<InputStream>
)MessageParser
.Some adjustments are needed on class working with attachment:
InputStream
. This is done through a separate AttachmentLoader
interface.MessageManager
. This enables to determine if attachment should be parsed or not on an implementation aware fashion, saving attachment parsing upon writes for JPA and Maildir.Maildir and JPA no longer support attachment content loading. Only the JMAP protocol requires attachment content loading, which is not supported on top of these technologies.
Mailbox search attachment content criteria will be supported only on implementation supporting attachment storage.
Users running Cassandra schema version prior version 5 will have to go through James release 3.5.0 to upgrade to a version after version 5 before proceeding with their update.
We noticed performance enhancement when using IMAP FETCH and JMAP GetMessages. Running a gatling test suite exercising JMAP getMessages on a dataset containing attachments leads to the following observations:
We also expect improvements in James memory allocation.