Date: 2022-09-13
Accepted (lazy consensus).
Implemented.
Classic java programming involves doing some blocking calls. When performing some input-output, the thread performing the tasks at hand will hang until a response is received. This means that a server performing IOs, which is the case of James, would end up with a high number of threads: for an IMAP server, one IMAP request in flight equals one thread.
The widely documented issue with such an approach is that, in Java threads correspond to system threads, and are expensive. Typically, each thread takes up to ~1MB of memory. To cap the count of threads at hand, the Thread Pool model is used to cap the count of threads. Process at most N IMAP requests with N threads, and queue requests that cannot yet be handled.
The blocking thread pool model was the concurrency paradigm used by James IMAP implementation in 3.7.x.
Reactive programming instead does not perform blocking operation, thus one thread can handle several IO tasks. This can be seen as callbacks executed once the request is received. Reactive programming model, amongst others, leads to efficient resource usage. It helps keeping the count of applicative threads low, helps reduce context switches.
James have been, among the past few years, transitioning to a reactive programming model using the Reactor library. Migrated components involves:
It is to be noted that our IMAP network stack uses Netty 4.x library that allows handling requests asynchronously.
Migrate IMAP to a reactive model.
Implement a way to limit IMAP concurrency to protect James server from bursts.
Significant rewrite of the IMAP stack.
Going reactive on the IMAP stacks yields the following improvements:
Diagnostic tools will be harder to use on top of the IMAP stack:
Associated risk:
From Java 19 onward, similar concerns are addressed by project Loom. This project delivers “Virtual Threads” scheduled by the Java Virtual Machine on top of carrier system threads. This approach allow preserving the imperative programing style. Tooling also works better.
Yet, this promissing feature:
Some other components have not yet been migrated to a reactive style. This includes:
With lower throughput, the benefits of migrating such components to a reactive style are lower thus might not match the associated rewrite costs and operational risk. As such, the effort had not yet been undertaken, but might be in the future.
It is to be mentioned that APPEND command is buffered to a file, which, when happening is done on a dedicated thread. The associated code is complex as Netty 4 ByteToMessageDecoder is meant to be synchronous. Upcoming Netty 5 would enable us to simplify associated code.