This document covers every changes an Ops needs to be aware of when running James.
The following procedures are to take as it, and the Apache Software Foundation, nor its contributors, can not be responsible for any damages generated by following the below procedures.
Before performing these operations, you should ensure to have the skills to conduct the operations, and you should read other software documentation. Do not follow this guide blindly!
No changes yet.
Note: this section is in progress. It will be updated during all the development process until the release.
Changes to apply between 3.7.x and 3.8.0 will be reported here.
Change list:
Date: 19/04/2023
JIRA: https://issues.apache.org/jira/browse/JAMES-3777
Concerned products: Distributed James, Cassandra James Server
Cassandra Event store table needs to be modified in order to support snapshots:
ALTER TABLE james_keyspace.eventStore ADD snapshot int static;
We recommend disabling increments during rolling updates (as older nodes in the cluster won't support them). This can be done with the following system property:
-Djames.jmap.filters.eventsource.increments.enabled=false
Date: 10/02/2023
JIRA: https://issues.apache.org/jira/browse/JAMES-3882
Concerned products: Distributed James, Cassandra James Server
When the deleted message vault is enabled in Cassandra products, now the deleted messages are directly copied into the vault, asynchronously upon deletions. Configuring DeletedMessageVaultHook is thus no longer needed.
Date 05/01/2023
JIRA: https://issues.apache.org/jira/browse/JAMES-3756
Concerned product: Distributed James, Cassandra James Server
Add delegated_users column to user tables in order to store delegated users that user has access to.
In order to add this delegated_users column you need to run the following CQL commands:
ALTER TABLE james_keyspace.user ADD delegated_users set<text>;
Date 05/12/2022
JIRA: https://issues.apache.org/jira/browse/JAMES-3754
Concerned product: Distributed James, Cassandra James
Add saveDate to James mailbox index to enable saveDate searching as part of IMAP SAVEDATE extension.
We already have this field as part of newly created mappings, but we need to explicitly add this field to existing indices by doing:
curl -X PUT \
http://ip:port/mailbox_v1/_mapping \
-H 'Content-Type: application/json' \
-d "{
\"properties\": {
\"saveDate\": {
\"type\": \"date\",
\"format\": \"uuuu-MM-dd'T'HH:mm:ssX||uuuu-MM-dd'T'HH:mm:ssXXX||uuuu-MM-dd'T'HH:mm:ssXXXXX\"
}
}
}"
Date 01/12/2022
JIRA: https://issues.apache.org/jira/browse/JAMES-3754
Concerned product: Distributed James, Cassandra James Server
Add save_date column to messageIdTable and imapUidTable tables in order to store saveDate data as part of IMAP SAVEDATE extension.
In order to add this messageIdTable column you need to run the following CQL commands:
ALTER TABLE james_keyspace.messageIdTable ADD save_date timestamp; ALTER TABLE james_keyspace.imapUidTable ADD save_date timestamp;
Date: 06/10/2022
PBKDF2WithHmacSHA1 was reported being a weak cryptographic set up.
As such we decided to rely on PBKDF2WithHmacSHA512 by default.
Users having data encrypted with PBKDF2WithHmacSHA1 can configure it as a private key algorithm to access their data, in blobstore.properties:
encryption.aes.private.key.algorithm=PBKDF2WithHmacSHA1
We do not have a migration strategy for PBKDF2WithHmacSHA1 users to migrate to PBKDF2WithHmacSHA512.
Date: 06/10/2022
JIRA: JAMES-3833
When establishing an SMTPS or StartTLS connection during remote mail delivery, James will now check the remote server's host/domain name against its certificate (RFC 2595). If they do not match, the connection fails as a temporary delivery error.
This prevents attackers from spoofing legitimate servers and intercepting mails. However, it may prevent James from connecting to servers that have strange certificates, no DNS entries, are reachable by IP address only, and similar edge cases.
Users requiring such connectivity may disable this check within mailetcontainer.xml at the RemoteDelivery mailet configuration:
<verifyServerIdentity>false</verifyServerIdentity>
Date: 06/10/2022
Concerned products: Distributed James, Cassandra James
JIRA: https://issues.apache.org/jira/browse/JAMES-3831
We added a sortOrder field to the identity object. Modification to the underlying Cassandra table is needed:
cqlsh:apache_james> ALTER TABLE custom_identity ADD sort_order int ;
Date: 13/09/2022
Concerned products: Distributed James, Cassandra James
In order to limit Cassandra table count, amongst others, the deleted message vault is now disabled by default.
Users relying on it needs to enable it within deletedMessageVault.properties with the following property:
enabled=true
Date: 23/08/2022
Concerned products: Distributed James, Cassandra James
JIRA: https://issues.apache.org/jira/browse/JAMES-3809
This setting was duplicated and do not account for the fetch type. Removed to avoid confusion.
Use batchsize.properties configuration file instead.
Date: 01/08/2022
Concerned products: Distributed James, Cassandra James
ElasticSearch v7 being EOL and upper versions being moved to a non-OSI compliant license (java clients included), it's been decided to migrate to OpenSearch that is a fork of ElasticSearch 7.10 and using Apache v2 license.
You should migrate your ElasticSearch servers from version 7 to OpenSearch 2.1 before upgrading James.
Users can either perform a migration of their data from version 7 to OpenSearch (documentation here) or might start from an empty Opensearch v2.1 cluster and reindex data.
Keep in mind as well that the group listeners related to ElasticSearch changed names. You would need to manually delete the bindings from the event queues in RabbitMQ before restarting James.
Configuration file was relocated to opensearch.properties and properties are now prefixed by opensearch. rather than elasticsearch..
Furthermore, implementation name that could be specified explicitly in search.properties has changed from elasticsearch to opensearch.
Date: 07/06/2022
Concerned product: Distributed James, Cassandra James Server
Cassandra driver had been updated to its version 4, empowering better reactive chains, more frequent releases, more configuration options amongst other.
Users need to be aware that Cassandra driver 4 no longer allow in depth programmatic configuration, and instead relies on its own configuration mechanisms.
Especially, SSL, consistency levels, timeouts, slow logs, metrics are now only configurable through the mean of Cassandra driver native configuration. We dropped related options from the cassandra.properties file.
You can find here an example of what conf/cassandra-driver-conf looks like, with sane defaults.
Formal driver documentation: https://docs.datastax.com/en/developer/java-driver/4.13/manual/core/configuration/
Date 26/05/2022
JIRA: https://issues.apache.org/jira/browse/JAMES-3756
Concerned product: Distributed James, Cassandra James Server
Add authorized_users column to user table in order to store delegated users.
In order to add this authorized_users column you need to run the following CQL command:
ALTER TABLE james_keyspace.user ADD authorized_users set<text>;
No specific operation to conduct from a 3.7.3 version.
No specific operation to conduct from a 3.7.2 version.
No specific operation to conduct from a 3.7.1 version.
No specific operation to conduct from a 3.7.0 version.
Changes to apply between 3.6.2 and 3.7.0 are reported here.
Change list:
Date: 03/02/2022
JIRA: https://issues.apache.org/jira/browse/JAMES-3712
These mailets uses the mail being processed to send a new email to a third party.
Prior to this change, the prefix property was prepended to the processed email and not to the one sent to the third party. Ie: the bounce recipient will not have the prefix.
We changed that so that the prefix property is prepended to the new email and not to the processed one. Ie: the bounce recipient will now have the prefix.
Users willing to retain the previous behaviour can do so by removing the prefix property from their mailet and add a AddSubjectPrefix after their mailet. Please note that most user can (and likely should) omit those modifications that are only mentioned here for the sake of completeness.
Here is an example of how to retain the strict behaviour applied prior this change for Bounce:
<!-- what was --> <mailet matcher="All" class="Bounce> <prefix>[OUPS] </prefix> </mailet> <!-- can now be written --> <mailet matcher="All" class="Bounce/> <mailet matcher="All" class="AddSubjectPrefix> <subjectPrefix>[OUPS] </subjectPrefix> </mailet>
Date: 11/01/2022
JIRA: https://issues.apache.org/jira/browse/JAMES-3690
As a security safe guard, we now enforce the host configuration parameter defined in webadmin.properties when binding the listening interface for the webadmin server.
By default, it listens on the loopback interface.
The previous behaviour can be restored by using the following configuration:
host=0.0.0.0
Be aware that webadmin do not enforce authentication by default, and needs JWT protection to be safely exposed.
Date: 07/12/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3680
We reworked SMTP/LMTP configuration to improve security and better explicit the configuration effects.
We added a new setting within SMTP/LMTP configuration to avoid advertising authentication on unencrypted channels. For security reasons, it is enabled by default, which is a breaking change. To disable it:
<smtpserver enabled="true"> <jmxName>smtpserver-global</jmxName> <bind>0.0.0.0:25</bind> <!--- ... --> <auth> <announce>never</announce> <requireSSL>false</requireSSL> </auth> <!--- ... --> </smtpserver>
Also, requireAuth setting, which was misleading, had been renamed to auth.announce. This change did not affect backward compatibility and requireAuth is still read as a fallback.
<requireAuth>false</requireAuth> can now be specified with <auth><announce>never</announce></auth><requireAuth>announce</requireAuth> can now be specified with <auth><announce>always</announce></auth><requireAuth>true</requireAuth> can now be specified with <auth><announce>forUnauthorizedAddresses</announce></auth>Finally, an implicit wildcard value 0.0.0.0/0.0.0.0 was specified as authorizedAddresses (for James 2.3 backward compatibility) when <requireAuth>false</requireAuth> or (equivalent) <auth><announce>never</announce></auth>. This behaviour could result in a user believing she disabled authentication, but would ultimately result in an open relay, which is unsecure, and why we no longer apply this implicit value. Users relying on James as an open relay will need to specify authorizedAddresses explicitly:
<smtpserver enabled="true"> <jmxName>smtpserver-global</jmxName> <bind>0.0.0.0:25</bind> <!--- ... --> <auth> <announce>never</announce> </auth> <authorizedAddresses>0.0.0.0/0.0.0.0</authorizedAddresses> <!--- ... --> </smtpserver>
Date: 29/11/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3675
The Vacation Mailet is now included in the standard mailet set, so you do not need to specify a package name anymore:
class="org.apache.james.jmap.mailet.VacationMailet" => class="VacationMailet"
Please update your mailetcontainer.xml accordingly.
Date: 24/11/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3674
With this update, the password algorithm may include an option to salt the password hash with the user name, for increased security against rainbow table cracking. To use this feature, update your usersrepository.xml :
<algorithm>SHA-512/salted</algorithm>
Whenever users change their password, James will update the respective database entry with a salted hash.
Note that plain hashing mode is the default. If you were using legacy mode and want to keep it for some reason, you must specify this option explicitly:
<algorithm>SHA-512/legacy</algorithm>
Date 19/09/2021
Mailing list vote: https://www.mail-archive.com/server-dev@james.apache.org/msg71060.html
Concerned product: Spring configured with MailDir
The maildir implementation, deprecated in 3.6.1 release, following a non-interoperable James naming scheme, suffering from numerous bugs and unmaintained for several years had been removed for the 3.7.0 release.
We encourage maildir users to switch to the JPA implementation (relying on an embedded database by default). Tools like imapsync or pop2imap can be used for this migration. This link explains how to use IMAP Sync.
Date 27/08/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3637
We changed the default algorithm to hash user password with JPA with SHA-512 instead of insecure SHA-1 / MD5.
As such, upgrades might result in inability for users to login, unless you explicitly configure the unsecure algorithm you were using.
Changing the user password will upgrade them to the newly introduced hashing standards.
Date 06/08/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3630
Concerned product: Distributed James and Cassandra James server
We restructured maximum global quota in order to store them in a single Cassandra row. This enables to retrieve this data on a single primary key read, which is important for performance when receiving many emails over SMTP.
Please recreate the maximum quota after the upgrade.
defaultMaxQuota table can be dropped.
Date 06/08/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3629
Concerned product: Distributed James
In order to avoid uses of non frozen Cassandra collection by the MailQueue component, we introduced a new table, enqueuedMailsv4, not suffering from these pitfalls.
Pre existing table, enqueuedMailsV3 is no longer read and can be dropped.
To avoid data loss (only noticeable when browsing the mailqueue email, no email in flight in the mailqueue will be lost, they will just be non-browseable) do your rolling upgrade with an empty mailqueue.
To achieve an empty mailqueue, start a James server with SMTP and JMAP ports not exposed: they will consume existing mails without creating new ones.
Date 13/08/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3631
Concerned products: Cassandra James, only for installation performed in 2018 or before (3.4.0).
To audit if you need to perform this upgrade run on a Cassandra server:
nodetool tablestats KEYSPACE mailrepositorycontent
A non-zero value in the key estimates means you need to perform a migration.
Migrating content of mailrepositorycontent table needs to be done prior a rolling upgrade. After the rolling upgrade the table can be removed.
In order to move content of the mailrepositorycontent table to mailrepositorycontentv2 table for ``:
mailetcontainer.xml define an orphan processor with just a ToRepository mailet for the processor you wishes to migrate. Here: <processor state="migrate-address-error" enableJmx="true">
<mailet match="All" class="ToRepository">
<repositoryPath>cassandra://var/mail/address-error/</repositoryPath>
</mailet>
</processor>
(Restart James for the configuration changes to be applied...)
Then trigger a reprocessing from the mailRepository to the processor you created. This can be done via webamin: https://james.apache.org/server/manage-webadmin.html#Administrating_mail_repositories . Here:
curl -XPATCH http://ip:port/mailRepositories/var%2Fmail%2Faddress-error%2F/mails?action=reprocess&processor=migrate-address-error
Date 09/08/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3544
Concerned product: Distributed James, Cassandra James server
In order to simplify the query flow, and reduce the amount on primary key reads when working with attachments when using JMAP, we did add the messageId property to the attachmentV2 table.
You need to perform the corresponding schema changes prior an attempt for a rolling upgrade:
cqlsh:apache_james> ALTER TABLE attachmentV2 ADD message_id timeuuid ;
Newly stored attachment will then store their corresponding messageId alongside attachmentV2 and will no longer need to read messageIdTable.
We furthermore propose a migration task for adding message_id field to attachmentV2 and thus ensure you leverage the benefits of this change:
See how to upgrade to the latest schema version (V12).
Date 22/07/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3516
Concerned product: Distributed James
Add threadId to James document mapping to enable thread query search.
We already have this field as part of newly created mappings, but we need to explicitly add this field to existing indices by doing:
curl -X PUT \
http://ip:ESport/mailbox_v1/_mapping \
-H 'Content-Type: application/json' \
-d '{
"properties": {
"threadId": {
"type": "keyword"
}
}
}'
Date: 02/05/2021 JIRA: https://issues.apache.org/jira/browse/JAMES-3576 Concerned products: Cassandra Guice servers (including Distributed server)
The table structure had been reworked in order to fasten message reads (metadata and headers) workloads for both IMAP and JMAP protocols.
Information previously included solely in messagev3 table (and thus needed to be read) have been added to both messageIdTable (serving IMAP reads) and imapUidTable (serving JMAP reads), allowing skipping reading messageV3 table when retrieving messages METADATA and HEADERS, effectively decreasing Cassandra query workload.
Before deploying the new version of James one need to previously adapt the table structure:
cqlsh:apache_james> ALTER TABLE messageIdTable ADD internalDate timestamp ; cqlsh:apache_james> ALTER TABLE messageidtable ADD bodyStartOctet int ; cqlsh:apache_james> ALTER TABLE messageidtable ADD fullContentOctets bigint ; cqlsh:apache_james> ALTER TABLE messageidtable ADD headerContent text ; cqlsh:apache_james> ALTER TABLE imapUidTable ADD internalDate timestamp ; cqlsh:apache_james> ALTER TABLE imapUidTable ADD bodyStartOctet int ; cqlsh:apache_james> ALTER TABLE imapUidTable ADD fullContentOctets bigint ; cqlsh:apache_james> ALTER TABLE imapUidTable ADD headerContent text ;
Note that if for some message information is missing from messageIdTable or imapUidTable then messagev3 table will effectively be read, defaulting to previous behaviour.
We furthermore propose a migration task for adding missing information to imapUidTable and messageIdTable and thus ensure you leverage the benefits of this change:
See how to upgrade to the latest schema version (V11).
Change list:
Date 23/06/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3516
Concerned product: Distributed James
Add threadId column to messageIdTable and imapUidTable in order to get a message's threadId.
In order to add this threadId column we advise you to run the following commands:
ALTER TABLE james_keyspace.messageIdTable ADD threadId timeuuid; ALTER TABLE james_keyspace.imapUidTable ADD threadId timeuuid;
Date 14/06/2021
JIRA: https://issues.apache.org/jira/projects/JAMES/issues/JAMES-3599
Impacted products: Distributed James server
We now group listeners execution whenever possible. This minimizes:
Note that retries are still performed on a per-group basis.
One need, after a rolling upgrade to unbind group queues from the primary exchange. Group queues can be identified by their mailboxEvent-workQueue- prefix, and the primary exchange is named mailboxEvent-exchange. These operations can easily be performed via the rabbitMQ management web interface.
Date 09/06/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3594
The previous LDAP implementation was based on JNDI and was causing operational concerns by opening a connection for each user authentication. These limitations were inherent to JNDI thus to mitigate those we decided to migrate to a newer LDAP library: UnboundID.
As part of this migration the following change took place:
useConnectionPool : Removed. UnboundId implementation relies on a pool by default.poolSize : Added. Allow controlling the count of connection in the pool.maxRetries, retryStartInterval, retryMaxInterval, retryIntervalScale. Those values will be ignored.The “group restriction” feature should furthermore be considered experimental, its usage is discouraged.
Date 05/05/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3578
Concerned products: Distributed James, Cassandra-guice James server
In an effort to simplify the code base, we dropped support for Cassandra schema version prior version 8.
Installation running older schema version than version 5 needs to rely on releases 3.6.x to upgrade to schema version 8, before upgrading to an eventual newer version.
No specific operation to conduct from a 3.6.0 version.
No specific operation to conduct from a 3.6.0 version.
Changes to apply between 3.5.0 and 3.6.0 are reported here.
Change list:
Date: 05/03/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3512
Concerned products: Cassandra Guice servers (without LDAP)
Previous computation was not flushing the base64 encoding stream resulting in the 2 last characters of the base64 string being omitted, causing inter-operability issues when synchronizing password hash with external systems.
New digest algorithm closes the base64 encoding stream resulting in a breaking change forcing user to reset their password.
If you wish to avoid this breaking change you can add the hashingMode configuration option in usersrepository.xml configuration file, with the value legacy, which will ensure previous behaviour.
Example usersrepository.xml:
<usersrepository>
<algorithm>SHA-512-legacy</algorithm>
<hashingMode>legacy</hashingMode>
<!-- ... -->
</usersrepository>
Date: 11/03/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-3492
Concerned products: Cassandra Guice server (with and without rabbitMQ)
James is no longer compatible with Elasticsearch 6.x. The new environment uses Elasticsearch 7.10.2 and users are recommended to upgrade to this version as well.
This link describes a possible update procedure:
Updating from Elasticsearch < 6.8 needs an extra step of shutting down the cluster and restarting it after each node is upgraded
Date: 07/12/2020
JIRA: https://issues.apache.org/jira/browse/JAMES-2578
Concerned products: Spring server, JPA Guice servers, Cassandra Guice server (without rabbitMQ)
Java serialization is unsafe and leads to countless vulnerability issues. We removed the remaining usages as part of the JMS and ActiveMQ mail queues.
In order to do so, we introduced a breaking change in the message format.
Upgrades should be done with an empty mail queue. To do so:
Date: 15/02/2021
JIRA: https://issues.apache.org/jira/browse/JAMES-2514
Concerned products: Cassandra Guice server (with and without rabbitMQ)
James is no longer tested against Cassandra 3.11.3 but instead against Cassandra 3.11.10. Users are recommended to upgrade to this version as well.
This link describes a possible update procedure:
nodetool flush)Date 05/12/2020
JIRA: https://issues.apache.org/jira/browse/JAMES-3435
Concerned product: Distributed James
Handles Mailbox ACL transactionality with event-sourcing. We got read of SERIAL consistency upon reads thus unlocking a major performance enhancement.
In order to benefit from this work, you need to upgrade to the latest schema version.
A James restart is advised after this migration in order to skip schema version reads.
Date 20/10/2020
JIRA: https://issues.apache.org/jira/browse/JAMES-3430
Concerned product: Distributed James
Adopt a more compact representation for message properties. This improves performance of the Cassandra mailbox.
In order to benefit from this work, you need to upgrade to the latest schema version.
Date 13/10/2020
JIRA: https://issues.apache.org/jira/browse/JAMES-3409
Concerned product: Distributed James
Add UID_VALIDITY to mailboxPath table in order not to mandate mailbox table reads. This improves performance of the Cassandra mailbox.
In order to benefit from this work, you need to upgrade to the latest schema version.
A James restart is advised after this migration in order to skip schema version reads.
Date 14/09/2020
JIRA: https://issues.apache.org/jira/browse/JAMES-3028
Concerned product: Distributed James
OpenStack's Swift APIs support has been dropped, please use Cassandra or AWS S3 instead. Please note that modern Swift versions support S3 APIs out of the box, thus OpenStack Swift can still be used through the S3 API.
Date 08/07/2020
JIRA: https://issues.apache.org/jira/browse/JAMES-3300
Concerned product: all products relying on LDAP users repository
useConnectionPool is now false by default. If you really want it, you have to explicitely put it to true in usersrepository.xml. It is false by default because it can create too much connections on the LDAP server. If you have few users it can eventually still make sense.
Date 03/06/2020
JIRA: https://issues.apache.org/jira/browse/JAMES-2760
Concerned product: Distributed James
mailqueue.size.metricsEnabled is now false by default. If you previously used it, please be aware that it can have some important performance penalty, and set it explicitly to true if you still need it.
Date 06/04/2020
JIRA: https://issues.apache.org/jira/browse/JAMES-2997
Concerned product: Distributed James, Cassandra-guice James server
In an effort to simplify the code base, we dropped support for Cassandra schema version prior version 5.
Installation running older schema version than version 5 needs to rely on release 3.5.0 to upgrade to schema version 7, before upgrading to an eventual newer version.
Changes to apply between 3.4.0 and 3.5.0 are reported here.
Change list:
Date 20/03/2020
SHA-1 7682112258dc2e0b7322bfcf9cb44c1d287af422
JIRA: https://issues.apache.org/jira/browse/JAMES-3122
Concerned product: Spring
As Log4J 1.x is not compatible with Java 9+ runtime, we adopted Log4J2 as a logging solution for the Spring product.
As a consequence, the log configuration file for Spring product needs to be updated. See an example here.
Also, the deprecated LogEnabled API will be removed. We recommend extension developers to obtain their Logger instance using the SLF4J LoggerFactory class instead.
Date 18/03/2020
SHA-1 bd6074cb689517061ecef9ddca3ab7a88d133cda
JIRA: https://issues.apache.org/jira/browse/JAMES-3121
Impacted product: Guice distributed James
Our usage of Cassandra for the time series have been improved by fine tuning the compaction strategy and the read repair option.
In order to unlock these improvements we advise you running the following commands on existing instalations
ALTER TABLE james_keyspace.deletedMailsV2 WITH compaction = { 'class' : 'TimeWindowCompactionStrategy' };
ALTER TABLE james_keyspace.enqueuedMailsV3 WITH compaction = {'compaction_window_size': '1',
'compaction_window_unit': 'HOURS',
'class': 'org.apache.cassandra.db.compaction.TimeWindowCompaction' };
ALTER TABLE james_keyspace.applicableFlag WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.attachmentMessageId WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.attachmentV2 WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.firstUnseen WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.mailboxCounters WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.mailboxRecents WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.messageCounter WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.messageDeleted WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.modseq WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.imapUidTable WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.messageIdTable WITH compaction = { 'class' : 'SizeTieredCompactionStrategy' };
ALTER TABLE james_keyspace.eventStore WITH compaction = { 'class' : 'LeveledCompactionStrategy' };
ALTER TABLE james_keyspace.attachmentOwners WITH bloom_filter_fp_chance = 0.01;
ALTER TABLE james_keyspace.deletedMailsV2 WITH bloom_filter_fp_chance = 0.01;
ALTER TABLE james_keyspace.firstUnseen WITH bloom_filter_fp_chance = 0.01;
ALTER TABLE james_keyspace.mailboxCounters WITH bloom_filter_fp_chance = 0.01;
ALTER TABLE james_keyspace.messageCounter WITH bloom_filter_fp_chance = 0.01;
ALTER TABLE james_keyspace.deletedMailsV2 WITH read_repair_chance = 0.0;
ALTER TABLE james_keyspace.enqueuedMailsV3 WITH read_repair_chance = 0.0;
Date 19/03/2020
SHA-1 58b1d879ab
JIRA: https://issues.apache.org/jira/browse/JAMES-3119
ProtocolSession have been reworked in order to increase type strengh and reduce errors.
Now setAttachment and getAttachment are expecting an AttachmentKey as key and return an Optional instead of a null-able value.
Moreover setAttachment do not allow null values and removeAttachment should be use now to remove elements.
Date 10/03/2020
SHA-1 81139cd94211358ca6a31b75b56c11b20e894ee4
JIRA: https://issues.apache.org/jira/browse/JAMES-3112
Concerned products: Guice products
We added strong typing for Domain aliases (exposed by this endpoint) to distinguish them from domain mappings (exposed by this endpoint).
Read this page to understand the difference between Domain Alias and Domain Mapping.
As a consequence, existing values returned by the domain alias endpoint (before this fix this is domain mapping) will be considered as domain mappings and might need to be deleted and re-created.
Date 26/02/2020
SHA-1 930fd38717b67f753b96f8b0595e80af7903298a
JIRA: https://issues.apache.org/jira/browse/JAMES-3074
Concerned products: JPA mailbox backend, Cassandra mailbox backend
Non-Maildir backends could generate ‘0’ uid validity with a low probability, which is invalid regarding RFC-3501. We changed the generation mechanism to use valid UidValidity for newly created mailboxes. Regarding persisted mailboxes, we regenerate invalid UidValidity upon reads.
While this sanitizing is transparent to the end user and the admin, it might lead to rare IMAP client full mailbox resynchronisation. (one chance out of two billions).
Date 26/02/2020
SHA-1 837299290848cdc11a90c2e73c4af549a0f1106f
JIRA: https://issues.apache.org/jira/browse/JAMES-3074
Concerned products: Spring with maildir backend
Maildir generated too big values to conform to RFC-3501. We changed the generation mechanism to use valid UidValidity for newly created mailboxes. Regarding persisted mailboxes, we regenerate invalid UidValidity upon reads.
While this sanitizing is transparent to the end user and the admin, it might lead to IMAP client full resynchronisation.
Date 04/02/2020
SHA-1 4ebddfc107f7f47c29ff1d25dd045f314ff28672
JIRA: https://issues.apache.org/jira/browse/JAMES-2950
Even if this set of characters should be allowed for the local part of a Username regarding some context, as defined by the RFC3696#section-3, we decided that for reducing code complexity and more safety to not allow them anymore when creating new Usernames.
However, the read of Usernames already existing with some of those characters is still allowed, to not introduce any breaking change.
Date 12/02/2020
SHA-1 2422d3ef92a6c74e9da233039613d38fd833e632
Concerned products: Guice server, experimental LinShare blob export feature.
JIRA: https://issues.apache.org/jira/browse/JAMES-3040
Blob Export Configuration changed:
File configuration need to be adjusted: blob.properties
-You need to add new mandatory properties when blob.export.implementation property is set to linshare:
blob.export.linshare.technical.account.uuid blob.export.linshare.technical.account.password
-The legacy property blob.export.linshare.token will not used anymore, you can remove it.
Date 6/01/2020
SHA-1 b44bc7d93901773ad442a341cb1ba2d94848c449
Concerned products: Guice distributed James server
Union blobStore, allowing to store older blobs within Cassandra while storing new blobs into object storage, had been removed.
Hybrid blobStore had been replacing it, allowing to store blobs either in a low cost blobStore or in a high performance blobStore, allowing thus some performance improvement for small, often read blobs while big or unfrequently read blobs keeps being stored cheaply.
Users relying on the Union blobStore will need to adopt Hybrid blobStore. Please adjust “blob.properties” accordingly:
implementation=hybrid
Date 16/12/2019
SHA-1 d68a64d96ba8312ecbd34761e82c9c2c32348290
Concerned products: Guice products
In an effort to have a resource oriented webadmin REST API we decided to relocate the endpoint for user mailbox reIndexing.
curl -XPOST /mailboxes?task=reIndex&user=bob@apache.org
Had been moved to:
curl -XPOST /users/bob@apache.org/mailboxes?task=reIndex
Date 10/12/2019
SHA-1 cdbc0ee65f
Concerned products: Guice products
Health check return codes had been changed:
Depending on specific deployment usage of health-checks, management scripts might need to be updated.
Date 26/11/2019
SHA-1 0bf4e8384e
Concerned products: Guice distributed James (rabbitMQ)
The distributed James project (relying on Guice, Cassandra, ElasticSearch, RabbitMQ and optionally Swift) benefits from a new distributed task mananger.
In order to enforce task sequential processing at the cluster level, we rely on a single active consumer, which is a feature introduced in RabbitMQ 3.8.
Users of distributed James product thus need to upgrade their RabbitMQ server to be at least version 3.8.1.
Ensure no task is running or scheduled.
taskManagerWorkQueue queue in RabbitMQDate 21/11/2019
SHA-1 9e976d3f49
JIRA: https://issues.apache.org/jira/browse/JAMES-2949
Many users recently complained about mails non received when sending to upper cased local recipients. We decided to simplify the handling of case for local recipients and users by always storing them lower cased. Now all the users repositories are storing user in lower case to ensure that. If you previously used to store users in a case sensitive way (which is very unlikely as it is broking delivery), you could need to update your user database to lower case all your users.
Date 15/11/2019
SHA-1 bcf4d36500
In order to allow the usage of cassandra credentials limited to a keyspace, the default behaviour of James is now to NOT create the keyspace during start-up.
The automatic creation of the cassandra keyspace by James could be enabled by setting cassandra.keyspace.create=true in the cassandra.properties configuration file.
Date 08/11/2019
SHA-1 0f8ee6ce2a
This specific user store was deprecated for years. It relied on Java serialization to store directly core.User object.
If you are still using it, you should instead use an other user repository.
If you need to export data from it, please contact us, it could be possible to write a little extraction tool.
Date 10/10/2019
SHA-1 0d72783ff4
JIRAS:
Concerned product: Guice product relying on ElasticSearch
We significantly improved our usage of ElasticSearch. Underlying changes include:
The downside of these changes is that a reindex is needed, implying a downtime on search:
Date: 25/09/2019
SHA-1: f721747edf8deb50406a5a44f6476507a03e2543
JIRA: https://issues.apache.org/jira/browse/JAMES-2703
Concerned products: Spring, default bundled mailets
This includes:
Changes to apply between 3.3.x and 3.4.x will be reported here.
Change list:
Date: 31/07/2019
SHA-1: f19642aef4a67cd6675f66278792ba4aa85d6d6e
JIRA: https://issues.apache.org/jira/browse/JAMES-2850
Concerned products: (experimental) RabbitMQ MailQueue
RabbitMQ MailQueue projection in Cassandra relies on a map allowing a single specific header per recipient header. This limitation causes rejection of emails with multiple per-recipient headers (which happens when forwarding to a remote server a mail checked against SpamAssassin).
In order to fix this issue, the structure of the underlying table was updated. A new table is created upon the first startup following the upgrade, and the old one is ignored.
Impact: the mails enqueued before the update can not be browsed nor removed from the queue after the update.
Recommendation: Conduct this update with an empty mail queue.
To do so:
Date: XX/06/2019
SHA-1: XXXXX
JIRA: https://issues.apache.org/jira/browse/JAMES-2794
Concerned products: (experimental) RabbitMQ MailQueue
RabbitMQ mail queue combines RabbitMQ with projections in Cassandra to offer advanced management capabilities expected from a mail queue (browse, delete, size, clear). In these projections, the mails are identified by there name. Thus enqueuing a mail that had already been processed will lead the given email to be considered already deleted and it will be discarded and lost.
This is an issue, as several other components build features around submitting a mail several time with the name.
For instance:
We thus changed the table structure of RabbitMQ mail queue projections to be built around an EnqueueId. This additional level of indirection allows several enqueues with the same name.
Upgrade to the newest James server needs to be performed with an empty MailQueue.
To do so:
Date: 27/05/2019
SHA-1: bbdf88e56d7a22fe92e1360ef563004f3bc0dd98
JIRA: https://issues.apache.org/jira/browse/JAMES-2766
Concerned products: (experimental) Cassandra-guice products.
In version 3.3.0 indexing for the Cassandra product was handled using ElasticSearch 2.2 released on the 31 march 2016. Some major upgrades had been included in recent ElasticSearch version.
Note that ElasticSearch APIs had been undergoing some major changes, making a smooth migration hard to provide. We proposed 2 migration strategies. A simple one leading to major search inconsistencies in the process, and another one mitigating these inconsistencies (but getting rid of them).
ElasticSearch 6 driver is relying on the high-level REST client and no more on the internal transport protocol.
Thus, you need to update your configuration files accordingly:
In elasticsearch.properties modify the elasticsearch.port properties to reference the HTTP port of your ElasticSearch nodes (9200 by default instead of the previous default value of 9300).
Procedure:
Keep in mind that full reIndexing needs to process all users email and thus can be slow.
Obviously this approach trades search consistency against ease of migration.
If search consistency during the migration is important for you, consider the next approach
Procedure:
Keep in mind that full reIndexing needs to process all users email and thus can be slow.
Changes to apply between 3.2.0 and 3.3.0 had been reported here.
Change list:
Date: 30/11/2018
SHA-1: 7e32da51a29bee1c732b2b13708bb4b986140119
JIRA: https://issues.apache.org/jira/browse/MAILBOX-292
MailboxId are now persisted in a james-mailboxId file. This file is created on the fly, so no action is required for users relying on the MailDir mailbox.
Date: 30/11/2018
SHA-1: d9bcebc7dd546bd5f11f3d9b496491e7c9042fe2
JIRA: https://issues.apache.org/jira/browse/MAILBOX-354
Only user written components performing MailboxListener registration will be affected.
The MailboxPath is mutable and thus can be changed upon mailbox rename. This leads to significantly complex code with possible inconsistency windows.
Using the mailboxId, which is immutable, solves these issues.
Date: 05/12/2018
SHA-1: 985b9a4a75bfa75c331cba6cbf835c043185dbdb
JIRA: https://issues.apache.org/jira/browse/JAMES-2555
We made this API introduced in James 3.2.0 a bit more REST friendly. If you developed tools using this API, you will need to update them.
For more details please refer to the latest WebAdmin documentation.
Date: 19/12/2018
SHA-1: e25967664538be18ec29f47e73e661bdf29da41f
JIRA: https://issues.apache.org/jira/projects/MAILBOX/issues/MAILBOX-356
Required: Yes
Concerned products: all JPA related products
Rename KEY column in JAMES_MAILBOX_ANNOTATION table. The syntax is:
ALTER TABLE JAMES_MAILBOX_ANNOTATION CHANGE KEY ANNOTATION_KEY varchar(200);
ALTER TABLE JAMES_MAILBOX_ANNOTATION CHANGE COLUMN KEY ANNOTATION_KEY varchar(200);
or the syntax corresponding to your database.
In order to allow safe serialization and strong typing org.apache.mailet.Mail have changed.
These methods have been deprecated and replaced:
getSender() in favor of getMaybeSender()getAttribute(String) in favor of getAttribute(AttributeName)setAttribute(String, Serializable) in favor of setAttribute(Attribute)removeAttribute(String) in favor of removeAttribute(AttributeName)getAttributeNames() in favor of attributeNames() and attributesMap()Some plain-string AttributeName have also been replaced:
SMTP_AUTH_USER_ATTRIBUTE_NAME in favor of SMTP_AUTH_USERMAILET_ERROR_ATTRIBUTE_NAME in favor of MAILET_ERRORSENT_BY_MAILET in favor of SENT_BY_MAILET_ATTRIBUTE's name, it is recommended to directly set the Attribute.Changes to apply between 3.1.0 and 3.2.0 had been reported here.
Changelist:
Date: 31/10/2018
SHA-1: 485406252d82c2d23a4078c76b26d6fc8973bbd7
JIRA: https://issues.apache.org/jira/browse/JAMES-2557
Required: Yes
Concerned products: User developed extensions - mailet/matcher
As part of the SMTP protocol, a mail can be sent without sender. This was represented implicitly in James by a potentially null MailAddress (null or MailAddress.nullSender()). This means that mailet/matcher implementers needs to be aware, and handle these cases. This implicit handling makes nullSender hard to work with, and prooved to be error prone as part of the 3.2.0 development process.
Hence we propose an alternative API returning a MaybeSender object, requiring the caller to explicitly handle missing sender.
Mail::getSender had then been deprecated. We strongly encourage our users to rely on Mail::getMaybeSender.
Note: thanks to java-8 default API methods, this is not a breaking change.
Date: 30/08/2018
SHA-1: 9ba6a1dd270f99735c7f9d3d4b2adb5076583c10
JIRA: https://issues.apache.org/jira/browse/JAMES-2529
Required: Yes
Concerned products: Cassandra Guice products
This mailet allow users filtering rules to be applied for incoming emails.
Add this line before the LocalDelivery mailet of your transport processor:
<mailet match="RecipientIsLocal" class="org.apache.james.jmap.mailet.filter.JMAPFiltering"/>
Date: 03/08/2018
SHA-1: de0fa8a3df69f50cbc0684dfb1b911ad497856d7
JIRA: https://issues.apache.org/jira/browse/JAMES-2514
Required: Yes
Concerned products: Cassandra Guice products
James Cassandra Guice now officially uses Cassandra 3.11.3 as a storage backend. After performing the upgrade, the team did perform some breaking changes, detailed below. James Cassandra Guice products are no more tested against Cassandra 2.2.x. Thus we strongly advise our users to upgrade.
Replace in default compaction strategies “DateTieredCompactionStrategy” by “TimeWindowCompactionStrategy”.
This means you can no more start James on top of an empty Cassandra 2.2.x cluster, but existing deployments should not be impacted.
We will assume that Cassandra had been installed with a debian package. Upgrade procedure stays similar in other cases.
/etc/apt/sources.list.d/cassandra.list to match 311x repositorydeb http://downloads.apache.org/cassandra/debian 311x main
$ apt-get update $ apt-get install cassandra=3.11.3
Edit /etc/cassandra/cassandra.yaml and ensure to really specify the interface cassandra is listening on as seeds.
4.1. Drain data & stop
$ nodetool drain $ nodetool stop
4.2. start Cassandra
$ nodetool upgradesstables apache_james