| <?xml version="1.0" encoding="UTF-8"?> |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| <!-- Generated by Apache Maven Doxia at 2021-11-12 --> |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> |
| <title>Apache James Project – Managing Guice distributed James</title> |
| <style type="text/css" media="all"> |
| @import url("../css/james.css"); |
| @import url("../css/maven-base.css"); |
| @import url("../css/maven-theme.css"); |
| @import url("../css/site.css"); |
| @import url("../js/jquery/css/custom-theme/jquery-ui-1.8.5.custom.css"); |
| @import url("../js/jquery/css/print.css"); |
| @import url("../js/fancybox/jquery.fancybox-1.3.4.css"); |
| </style> |
| <script type="text/javascript" src="../js/jquery/js/jquery-1.4.2.min.js"></script> |
| <script type="text/javascript" src="../js/jquery/js/jquery-ui-1.8.5.custom.min.js"></script> |
| <script type="text/javascript" src="../js/fancybox/jquery.fancybox-1.3.4.js"></script> |
| <link rel="stylesheet" href="../css/print.css" type="text/css" media="print" /> |
| <meta name="Date-Revision-yyyymmdd" content="20211112" /> |
| <meta http-equiv="Content-Language" content="en" /> |
| |
| <!-- Google Analytics --> |
| <script type="text/javascript"> |
| |
| var _gaq = _gaq || []; |
| _gaq.push(['_setAccount', 'UA-1384591-1']); |
| _gaq.push(['_trackPageview']); |
| |
| (function() { |
| var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; |
| ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; |
| var s = document.getElementsByTagName('script').item(0); s.parentNode.insertBefore(ga, s); |
| })(); |
| |
| </script> |
| </head> |
| <body class="composite"> |
| <div id="banner"> |
| <a href="../index.html" id="bannerLeft" title="james-logo.png"> |
| |
| |
| <img src="../images/logos/james-logo.png" alt="James Project" /> |
| </a> |
| <a href="https://www.apache.org/index.html" id="bannerRight"> |
| |
| |
| <img src="images/logos/asf_logo_small.png" alt="The Apache Software Foundation" /> |
| </a> |
| <div class="clear"> |
| <hr/> |
| </div> |
| </div> |
| <div id="breadcrumbs"> |
| |
| |
| <div class="xleft"> |
| <span id="publishDate">Last Published: 2021-11-12</span> |
| </div> |
| <div class="xright"> <a href="../index.html" title="Home">Home</a> |
| | |
| <a href="../documentation.html" title="James">James</a> |
| | |
| <a href="../mime4j/index.html" title="Mime4J">Mime4J</a> |
| | |
| <a href="../jsieve/index.html" title="jSieve">jSieve</a> |
| | |
| <a href="../jspf/index.html" title="jSPF">jSPF</a> |
| | |
| <a href="../jdkim/index.html" title="jDKIM">jDKIM</a> |
| |
| |
| </div> |
| <div class="clear"> |
| <hr/> |
| </div> |
| </div> |
| <div id="leftColumn"> |
| <div id="navcolumn"> |
| |
| |
| <h5>James components</h5> |
| <ul> |
| <li class="collapsed"> |
| <a href="../documentation.html" title="About James">About James</a> |
| </li> |
| <li class="expanded"> |
| <a href="../server/index.html" title="Server">Server</a> |
| <ul> |
| <li class="none"> |
| <a href="../server/advantages.html" title="Advantages">Advantages</a> |
| </li> |
| <li class="none"> |
| <a href="../server/objectives.html" title="Objectives">Objectives</a> |
| </li> |
| <li class="expanded"> |
| <a href="../server/quick-start.html" title="User Manual">User Manual</a> |
| <ul> |
| <li class="collapsed"> |
| <a href="../server/features.html" title="1. Features">1. Features</a> |
| </li> |
| <li class="none"> |
| <a href="../server/packaging.html" title="2. Packaging">2. Packaging</a> |
| </li> |
| <li class="collapsed"> |
| <a href="../server/install.html" title="3. Install James">3. Install James</a> |
| </li> |
| <li class="collapsed"> |
| <a href="../server/config.html" title="4. Configure James">4. Configure James</a> |
| </li> |
| <li class="expanded"> |
| <a href="../server/manage.html" title="5. Manage">5. Manage</a> |
| <ul> |
| <li class="none"> |
| <a href="../server/manage-cli.html" title="Command line">Command line</a> |
| </li> |
| <li class="none"> |
| <a href="../server/metrics.html" title="Metrics">Metrics</a> |
| </li> |
| <li class="none"> |
| <a href="../server/manage-webadmin.html" title="WebAdmin">WebAdmin</a> |
| </li> |
| <li class="none"> |
| <strong>Distributed James</strong> |
| </li> |
| </ul> |
| </li> |
| <li class="collapsed"> |
| <a href="../server/monitor.html" title="6. Monitor">6. Monitor</a> |
| </li> |
| <li class="collapsed"> |
| <a href="../server/upgrade.html" title="7. Upgrade">7. Upgrade</a> |
| </li> |
| <li class="collapsed"> |
| <a href="../server/dev.html" title="8. Developers Corner">8. Developers Corner</a> |
| </li> |
| </ul> |
| </li> |
| <li class="none"> |
| <a href="../mail.html#James_Mailing_lists" title="Mailing Lists">Mailing Lists</a> |
| </li> |
| <li class="none"> |
| <a href="../server/release-notes.html" title="Release Notes">Release Notes</a> |
| </li> |
| <li class="none"> |
| <a href="../server/apidocs/index.html" title="Javadoc">Javadoc</a> |
| </li> |
| <li class="none"> |
| <a href="https://issues.apache.org/jira/browse/JAMES" title="Issue Tracker">Issue Tracker</a> |
| </li> |
| <li class="none"> |
| <a href="https://github.com/apache/james-project" title="Sources">Sources</a> |
| </li> |
| <li class="none"> |
| <a href="../server/rfcs.html" title="RFCs">RFCs</a> |
| </li> |
| <li class="none"> |
| <a href="../download.cgi#Apache_James_Server" title="Download releases">Download releases</a> |
| </li> |
| </ul> |
| </li> |
| <li class="collapsed"> |
| <a href="../mailet/index.html" title="Mailets">Mailets</a> |
| </li> |
| <li class="collapsed"> |
| <a href="../mailbox/index.html" title="Mailbox">Mailbox</a> |
| </li> |
| <li class="collapsed"> |
| <a href="../protocols/index.html" title="Protocols">Protocols</a> |
| </li> |
| <li class="collapsed"> |
| <a href="../mpt/index.html" title="MPT">MPT</a> |
| </li> |
| </ul> |
| <h5>Apache Software Foundation</h5> |
| <ul> |
| <li> |
| <strong> |
| <a title="ASF" href="http://www.apache.org/">ASF</a> |
| </strong> |
| </li> |
| <li> |
| <a title="Get Involved" href="http://www.apache.org/foundation/getinvolved.html">Get Involved</a> |
| </li> |
| <li> |
| <a title="FAQ" href="http://www.apache.org/foundation/faq.html">FAQ</a> |
| </li> |
| <li> |
| <a title="License" href="http://www.apache.org/licenses/" >License</a> |
| </li> |
| <li> |
| <a title="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a> |
| </li> |
| <li> |
| <a title="Thanks" href="http://www.apache.org/foundation/thanks.html">Thanks</a> |
| </li> |
| <li> |
| <a title="Security" href="http://www.apache.org/security/">Security</a> |
| </li> |
| </ul> |
| <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"> |
| <img class="poweredBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /> |
| </a> |
| |
| |
| </div> |
| </div> |
| <div id="bodyColumn"> |
| <div id="contentBox"> |
| <h1>Managing Guice distributed James</h1> |
| <p>This guide aims to be an entry-point to the James documentation for user managing a distributed Guice James server.</p> |
| <p>It includes:</p> |
| <ul> |
| |
| <li>Simple architecture explanations</li> |
| <li>Propose some diagnostics for some common issues</li> |
| <li>Present procedures that can be set up to address these issues</li> |
| </ul> |
| <p>In order to not duplicate information, existing documentation will be linked.</p> |
| <p>Please note that this product is under active development, should be considered experimental and thus targets advanced users.</p><section> |
| <h2><a name="Table_of_content"></a>Table of content</h2> |
| <ul> |
| |
| <li><a href="#Overall_architecture">Overall architecture</a></li> |
| <li><a href="#Basic_Monitoring">Basic Monitoring</a></li> |
| <li><a href="#Cassandra_table_level_configuration">Cassandra table level configuration</a></li> |
| <li><a href="#Deleted_Messages_Vault">Deleted Messages Vault</a></li> |
| <li><a href="#Elasticsearch_Indexing">ElasticSearch Indexing</a></li> |
| <li><a href="#Mailbox_Event_Bus">Mailbox Event Bus</a></li> |
| <li><a href="#Mail_Processing">Mail Processing</a></li> |
| <li><a href="#Mail_Queue">Mail Queue</a></li> |
| <li><a href="#Setting_Cassandra_user_permissions">Setting Cassandra user permissions</a></li> |
| <li><a href="#Solving_cassandra_inconsistencies">Solving cassandra inconsistencies</a></li> |
| <li><a href="#Updating_Cassandra_schema_version">Updating Cassandra schema version</a></li> |
| </ul></section><section> |
| <h2><a name="Overall_architecture"></a>Overall architecture</h2> |
| <p>Guice distributed James server intends to provide a horizontally scalable email server.</p> |
| <p>In order to achieve this goal, this product leverages the following technologies:</p> |
| <ul> |
| |
| <li><b>Cassandra</b> for meta-data storage</li> |
| <li><b>ObjectStorage</b> (S3) for binary content storage</li> |
| <li><b>ElasticSearch</b> for search</li> |
| <li><b>RabbitMQ</b> for messaging</li> |
| </ul> |
| <p>A <a class="externalLink" href="https://github.com/apache/james-project/blob/master/server/apps/distributed-app/docker-compose.yml">docker-compose</a> file is available to allow you to quickly deploy locally this product.</p></section><section> |
| <h2><a name="Basic_Monitoring"></a>Basic Monitoring</h2> |
| <p>A toolbox is available to help an administrator diagnose issues: - <a href="#Structured_logging_into_Kibana">Structured logging into Kibana</a> - <a href="#Metrics_graphs_into_Grafana">Metrics graphs into Grafana</a> - <a href="#Webadmin_Healthchecks">WebAdmin HealthChecks</a></p><section> |
| <h3><a name="Structured_logging_into_Kibana"></a>Structured logging into Kibana</h3> |
| <p>Read this page regarding <a href="monitor-logging.html#Guice_products_and_logging">setting up structured logging</a>.</p> |
| <p>We recommend to closely monitoring <b>ERROR</b> and <b>WARNING</b> logs. Those logs should be considered not normal.</p> |
| <p>If you encounter some suspicious logs: - If you have any doubt about the log being caused by a bug in James source code, please reach us via<br /> |
| the bug tracker, the user mailing list or our Gitter channel (see our <a class="externalLink" href="http://james.apache.org/#second">community page</a>) - They can be due to insufficient performance from tier applications (eg Cassandra timeouts). In such case we advise you to conduct a close review of performances at the tier level.</p> |
| <p>Leveraging filters in Kibana discover view can help filtering out “already known” frequently occurring logs.</p> |
| <p>When reporting ERROR or WARNING logs, consider adding the full logs, and related data (eg the raw content of a mail triggering an issue) to the bug report in order to ease resolution.</p></section><section> |
| <h3><a name="Metrics_graphs_into_Grafana"></a>Metrics graphs into Grafana</h3> |
| <p>James keeps tracks of various metrics and allow to easily visualize them.</p> |
| <p>Read this page for <a href="metrics.html">explanations on metrics</a>.</p> |
| <p>Here is a list of <a class="externalLink" href="https://github.com/apache/james-project/tree/master/grafana-reporting">available metric boards</a></p> |
| <p>Configuration of <a href="config-elasticsearch.html">ElasticSearch metric exporting</a> allows a direct display within <a class="externalLink" href="https://grafana.com/">Grafana</a></p> |
| <p>Monitoring these graphs on a regular basis allows diagnosing early some performance issues.</p> |
| <p>If some metrics seem abnormally slow despite in depth database performance tuning, feedback is appreciated as well on the bug tracker, the user mailing list or our Gitter channel (see our <a class="externalLink" href="http://james.apache.org/#second">community page</a>) . Any additional details categorizing the slowness are appreciated as well (details of the slow requests for instance).</p></section><section> |
| <h3><a name="WebAdmin_HealthChecks"></a>WebAdmin HealthChecks</h3> |
| <p>James webadmin API allows to run healthChecks for a quick health overview.</p> |
| <p>Here is related <a href="manage-webadmin.html#HealthCheck">webadmin documentation</a></p> |
| <p>Here are the available checks alongside the insight they offer:</p> |
| <ul> |
| |
| <li><b>Cassandra backend</b>: Cassandra storage. Ensure queries can be executed on the connection James uses.</li> |
| <li><b>ElasticSearch Backend</b>: ElasticSearch storage. Triggers an ElasticSearch health request on indices James uses.</li> |
| <li><b>EventDeadLettersHealthCheck</b>: EventDeadLetters checking.</li> |
| <li><b>RabbitMQ backend</b>: RabbitMQ messaging. Verifies an open connection and an open channel are well available.</li> |
| <li><b>Guice application lifecycle</b>: Ensures James Guice successfully started, and is up. Logs should contain explanations if James did not start well.</li> |
| <li><b>MessageFastViewProjection</b>: Follows MessageFastViewProjection cache miss rates and warns if it is below 10%. If this projection is missing, this results in performance issues for JMAP GetMessages list requests. WebAdmin offers a <a href="manage-webadmin.html#Recomputing_Global_JMAP_fast_message_view_projection">global</a> and <a href="manage-webadmin.html#Recomputing_Global_JMAP_fast_message_view_projection">per user</a> projection re-computation. Note that as computation is asynchronous, this projection can be slightly out of sync on a normally behaving server.</li> |
| </ul></section></section><section> |
| <h2><a name="Mail_Processing"></a>Mail Processing</h2> |
| <p>Mail processing allows to take asynchronously business decisions on received emails.</p> |
| <p>Here are its components:</p> |
| <ul> |
| |
| <li>The spooler takes mail out of the mailQueue and executes mail processing within the mailet container.</li> |
| <li>The mailet container synchronously executes the user defined logic. This ‘logic’ is written through the use of mailet, matcher and processor.</li> |
| <li>A mailet represents an action: mail modification, envelop modification, a side effect, or stop processing.</li> |
| <li>A matcher represents a condition to execute a mailet.</li> |
| <li>A processor is a flow of pair of matcher and mailet executed sequentially. The ToProcessor mailet is a goto instruction to start executing another processor</li> |
| <li>A mail repository allows storage of a mail as part of its processing. Standard configuration relies on the following mail repository: |
| <ul> |
| |
| <li>cassandra://var/mail/error/ : unexpected errors that occurred during mail processing. Emails impacted by performance related exceptions, or logical bug within James code are typically stored here. These mails could be reprocessed once the cause of the error is fixed. The Mail.error field can help diagnose the issue. Correlation with logs can be achieved via the use of the Mail.name field.</li> |
| <li>cassandra://var/mail/address-error/ : mail addressed to a non-existing recipient of a handled local domain. These mails could be reprocessed once the user is created, for instance.</li> |
| <li>cassandra://var/mail/relay-denied/ : mail for whom relay was denied: missing authentication can, for instance, be a cause. In addition to prevent disasters upon miss configuration, an email review of this mail repository can help refine a host spammer blacklist.</li> |
| <li>cassandra://var/mail/rrt-error/ : runtime error upon Recipient Rewritting occurred. This is typically due to a loop. We recommend verifying user mappings via <a href="manage-webadmin.html#User_Mappings">User Mappings webadmin API</a> then once identified break the loop by removing some Recipient Rewrite Table entry via the <a href="manage-webadmin.html#Removing_an_alias_of_an_user">Delete Alias</a>, <a href="manage-webadmin.html#Removing_a_group_member">Delete Group member</a>, <a href="manage-webadmin.html#Removing_a_destination_of_a_forward">Delete forward</a>, <a href="manage-webadmin.html#Remove_an_address_mapping">Delete Address mapping</a>, <a href="manage-webadmin.html#Removing_a_domain_mapping">Delete Domain mapping</a> or <a href="manage-webadmin.html#Removing_a_regex_mapping">Delete Regex mapping</a> APIs (as needed). The Mail.error field can help diagnose the issue as well. Then once the root cause has been addressed, the mail can be reprocessed.</li> |
| </ul> |
| </li> |
| </ul> |
| <p>Read <a href="config-mailetcontainer.html">this</a> to discover mail processing configuration, including error management.</p> |
| <p>Currently, an administrator can monitor mail processing failure through ERROR log review. We also recommend watching in Kibana INFO logs using the org.apache.james.transport.mailets.ToProcessor value as their logger. Metrics about mail repository size, and the corresponding Grafana boards are yet to be contributed.</p> |
| <p>WebAdmin exposes all utilities for <a href="manage-webadmin.html#Reprocessing_mails_from_a_mail_repository">reprocessing all mails in a mail repository</a> or <a href="manage-webadmin.html#Reprocessing_a_specific_mail_from_a_mail_repository">reprocessing a single mail in a mail repository</a>.</p> |
| <p>Also, one can decide to <a href="manage-webadmin.html#Removing_all_mails_from_a_mail_repository">delete all the mails of a mail repository</a> or <a href="manage-webadmin.html#Removing_a_mail_from_a_mail_repository">delete a single mail of a mail repository</a>.</p> |
| <p>Performance of mail processing can be monitored via the <a class="externalLink" href="https://github.com/apache/james-project/blob/master/grafana-reporting/MAILET-1490071694187-dashboard.json">mailet grafana board</a> and <a class="externalLink" href="https://github.com/apache/james-project/blob/master/grafana-reporting/MATCHER-1490071813409-dashboard.json">matcher grafana board</a>.</p></section><section> |
| <h2><a name="Mailbox_Event_Bus"></a>Mailbox Event Bus</h2> |
| <p>James relies on an event bus system to enrich mailbox capabilities. Each operation performed on the mailbox will trigger related events, that can be processed asynchronously by potentially any James node on a distributed system.</p> |
| <p>Many different kind of events can be triggered during a mailbox operation, such as:</p> |
| <ul> |
| |
| <li>MailboxEvent: event related to an operation regarding a mailbox:</li> |
| <li>MailboxDeletion: a mailbox has been deleted</li> |
| <li>MailboxAdded: a mailbox has been added</li> |
| <li>MailboxRenamed: a mailbox has been renamed</li> |
| <li>MailboxACLUpdated: a mailbox got its rights and permissions updated</li> |
| <li>MessageEvent: event related to an operation regarding a message:</li> |
| <li>Added: messages have been added to a mailbox</li> |
| <li>Expunged: messages have been expunged from a mailbox</li> |
| <li>FlagsUpdated: messages had their flags updated</li> |
| <li>MessageMoveEvent: messages have been moved from a mailbox to an other</li> |
| <li>QuotaUsageUpdatedEvent: event related to quota update</li> |
| </ul> |
| <p>Mailbox listeners can register themselves on this event bus system to be called when an event is fired, allowing to do different kind of extra operations on the system, like:</p> |
| <ul> |
| |
| <li>Current quota calculation</li> |
| <li>Message indexation with ElasticSearch</li> |
| <li>Mailbox annotations cleanup</li> |
| <li>Ham/spam reporting to SpamAssassin</li> |
| <li>…</li> |
| </ul> |
| <p>It is possible for the administrator of James to define the mailbox listeners he wants to use, by adding them in the <a class="externalLink" href="https://github.com/apache/james-project/blob/master/server/apps/distributed-app/sample-configuration/listeners.xml">listeners.xml</a> configuration file. It’s possible also to add your own custom mailbox listeners. This enables to enhance capabilities of James as a Mail Delivery Agent. You can get more information about those <a href="config-listeners.html">here</a>.</p> |
| <p>Currently, an administrator can monitor listeners failures through ERROR log review. Metrics regarding mailbox listeners can be monitored via <a class="externalLink" href="https://github.com/apache/james-project/blob/master/grafana-reporting/MailboxListeners-1528958667486-dashboard.json">mailbox_listeners grafana board</a> and <a class="externalLink" href="https://github.com/apache/james-project/blob/master/grafana-reporting/MailboxListeners%20rate-1552903378376.json">mailbox_listeners_rate grafana board</a>.</p> |
| <p>Upon exceptions, a bounded number of retries are performed (with exponential backoff delays). If after those retries the listener is still failing to perform its operation, then the event will be stored in the <a href="manage-webadmin.html#Event_Dead_Letter">Event Dead Letter</a>. This API allows diagnosing issues, as well as redelivering the events.</p> |
| <p>To check that you have undelivered events in your system, you can first run the associated with <a href="manage-webadmin.html#Event_Dead_Letter">event dead letter health check</a> .You can explore Event DeadLetter content through WebAdmin. For this, <a href="manage-webadmin.html#Listing_mailbox_listener_groups">list mailbox listener groups</a> you will get a list of groups back, allowing you to check if those contain registered events in each by <a href="manage-webadmin.html#Listing_failed_events">listing their failed events</a>.</p> |
| <p>If you get failed events IDs back, you can as well <a href="manage-webadmin.html#Getting_event_details">check their details</a>.</p> |
| <p>An easy way to solve this is just to trigger then the <a href="manage-webadmin.html#Redeliver_all_events">redeliver all events</a> task. It will start reprocessing all the failed events registered in event dead letters.</p> |
| <p>If for some other reason you don’t need to redeliver all events, you have more fine-grained operations allowing you to <a href="manage-webadmin.html#Redeliver_group_events">redeliver group events</a> or even just <a href="manage-webadmin.html#Redeliver_a_single_event">redeliver a single event</a>.</p></section><section> |
| <h2><a name="ElasticSearch_Indexing"></a>ElasticSearch Indexing</h2> |
| <p>A projection of messages is maintained in ElasticSearch via a listener plugged into the mailbox event bus in order to enable search features.</p> |
| <p>You can find more information about ElasticSearch configuration <a href="config-elasticsearch.html">here</a>.</p><section> |
| <h3><a name="Usual_troubleshooting_procedures"></a>Usual troubleshooting procedures</h3> |
| <p>As explained in the <a href="#Mailbox_Event_Bus">Mailbox Event Bus</a> section, processing those events can fail sometimes.</p> |
| <p>Currently, an administrator can monitor indexation failures through ERROR log review. You can as well <a href="manage-webadmin.html#Listing_failed_events">list failed events</a> by looking with the group called org.apache.james.mailbox.elasticsearch.v7.events.ElasticSearchListeningMessageSearchIndex$ElasticSearchListeningMessageSearchIndexGroup. A first on-the-fly solution could be to just <a href="#Mailbox_Event_Bus">redeliver those group events with event dead letter</a>.</p> |
| <p>If the event storage in dead-letters fails (for instance in the face of Cassandra storage exceptions), then you might need to use our WebAdmin reIndexing tasks.</p> |
| <p>From there, you have multiple choices. You can <a href="manage-webadmin.html#ReIndexing_all_mails">reIndex all mails</a>, <a href="manage-webadmin.html#ReIndexing_a_mailbox_mails">reIndex mails from a mailbox</a> or even just <a href="manage-webadmin.html#ReIndexing_a_single_mail">reIndex a single mail</a>.</p> |
| <p>When checking the result of a reIndexing task, you might have failed reprocessed mails. You can still use the task ID to <a href="manage-webadmin.html#Fixing_previously_failed_ReIndexing">reprocess previously failed reIndexing mails</a>.</p></section><section> |
| <h3><a name="On_the_fly_ElasticSearch_Index_setting_update"></a>On the fly ElasticSearch Index setting update</h3> |
| <p>Sometimes you might need to update index settings. Cases when an administrator might want to update index settings include:</p> |
| <ul> |
| |
| <li>Scaling out: increasing the shard count might be needed.</li> |
| <li>Changing string analysers, for instance to target another language</li> |
| <li>etc.</li> |
| </ul> |
| <p>In order to achieve such a procedure, you need to:</p> |
| <ul> |
| |
| <li><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/6.3/indices-create-index.html">Create the new index</a> with the right settings and mapping</li> |
| <li>James uses two aliases on the mailbox index: one for reading (mailboxReadAlias) and one for writing (mailboxWriteAlias). First <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/6.3/indices-aliases.html">add an alias</a> mailboxWriteAlias to that new index, so that now James writes on the old and new indexes, while only keeping reading on the first one</li> |
| <li>Now trigger a <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/6.3/docs-reindex.html">reindex</a> from the old index to the new one (this actively relies on _source field being present)</li> |
| <li>When this is done, add the mailboxReadAlias alias to the new index</li> |
| <li>Now that the migration to the new index is done, you can <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/6.3/indices-delete-index.html">drop the old index</a></li> |
| <li>You might want as well modify the James configuration file <a class="externalLink" href="https://github.com/apache/james-project/blob/master/server/apps/distributed-app/sample-configuration/elasticsearch.properties">elasticsearch.properties</a> by setting the parameter elasticsearch.index.mailbox.name to the name of your new index. This is to avoid that James re-creates index upon restart</li> |
| </ul> |
| <p><i>Note</i>: keep in mind that reindexing can be a very long operation depending on the volume of mails you have stored.</p></section></section><section> |
| <h2><a name="Solving_cassandra_inconsistencies"></a>Solving cassandra inconsistencies</h2> |
| <p>Cassandra backend uses data duplication to workaround Cassandra query limitations. However, Cassandra is not doing transaction when writing in several tables, this can lead to consistency issues for a given piece of data. The consequence could be that the data is in a transient state (that should never appear outside of the system).</p> |
| <p>Because of the lack of transactions, it’s hard to prevent these kind of issues. We had developed some features to fix some existing cassandra inconsistency issues that had been reported to James.</p> |
| <p>Here is the list of known inconsistencies: - <a href="#Jmap_message_fast_view_projections">Jmap message fast view projections</a> - <a href="#Mailboxes">Mailboxes</a> - <a href="#Mailboxes_counters">Mailboxes Counters</a> - <a href="#Messages">Messages</a> - <a href="#Quotas">Quotas</a> - <a href="#Rrt_RecipientRewriteTable_mapping_sources">RRT (RecipientRewriteTable) mapping sources</a></p><section> |
| <h3><a name="Jmap_message_fast_view_projections"></a>Jmap message fast view projections</h3> |
| <p>When you read a Jmap message, some calculated properties are expected to be fast to retrieve, like preview, hasAttachment. James achieves it by pre-calculating and storing them into a caching table (message_fast_view_projection). Missing caches are populated on message reads and will temporary decrease the performance.</p><section> |
| <h4><a name="How_to_detect_the_outdated_projections"></a>How to detect the outdated projections</h4> |
| <p>You can watch the MessageFastViewProjection health check at <a href="manage-webadmin.html#Check_all_components">webadmin documentation</a>. It provides a check based on the ratio of missed projection reads.</p></section><section> |
| <h4><a name="How_to_solve"></a>How to solve</h4> |
| <p>Since the MessageFastViewProjection is self healing, you should be concerned only if the health check still returns degraded for a while, there’s a possible thing you can do is looking at James logs for more clues.</p></section></section><section> |
| <h3><a name="Mailboxes"></a>Mailboxes</h3> |
| <p>mailboxPath and mailbox tables share common fields like mailboxId and mailbox name. A successful operation of creating/renaming/delete mailboxes has to succeed at updating mailboxPath and mailbox table. Any failure on creating/updating/delete records in mailboxPath or mailbox can produce inconsistencies.</p><section> |
| <h4><a name="How_to_detect_the_inconsistencies"></a>How to detect the inconsistencies</h4> |
| <p>If you found the suspicious MailboxNotFoundException in your logs. Currently, there’s no dedicated tool for that, we recommend scheduling the SolveInconsistencies task below for the mailbox object on a regular basis, avoiding peak traffic in order to address both inconsistencies diagnostic and fixes.</p></section><section> |
| <h4><a name="How_to_solve"></a>How to solve</h4> |
| <p>An admin can run offline webadmin <a href="manage-webadmin.html#Fixing_mailboxes_inconsistencies">solve Cassandra mailbox object inconsistencies task</a> in order to sanitize his mailbox denormalization.</p> |
| <p>In order to ensure being offline, stop the traffic on SMTP, JMAP and IMAP ports, for example via re-configuration or firewall rules.</p></section></section><section> |
| <h3><a name="Mailboxes_Counters"></a>Mailboxes Counters</h3> |
| <p>James maintains a per mailbox projection for message count and unseen message count. Failures during the denormalization process will lead to incorrect results being returned.</p><section> |
| <h4><a name="How_to_detect_the_inconsistencies"></a>How to detect the inconsistencies</h4> |
| <p>Incorrect message count/message unseen count could be seen in the Mail User Agent (IMAP or JMAP). Invalid values are reported in the logs as warning with the following class org.apache.james.mailbox.model.MailboxCounters and the following message prefix: Invalid mailbox counters.</p></section><section> |
| <h4><a name="How_to_solve"></a>How to solve</h4> |
| <p>Execute the [recompute Mailbox counters task](manage-webadmin.html#Recomputing mailbox counters). This task is not concurrent-safe. Concurrent increments & decrements will be ignored during a single mailbox processing. Re-running this task may eventually return the correct result.</p></section></section><section> |
| <h3><a name="Messages"></a>Messages</h3> |
| <p>Messages are denormalized and stored in both imapUidTable (source of truth) and messageIdTable. Failure in the denormalization process will cause inconsistencies between the two tables.</p><section> |
| <h4><a name="How_to_detect_the_inconsistencies"></a>How to detect the inconsistencies</h4> |
| <p>User can see a message in JMAP but not in IMAP, or mark a message as ‘SEEN’ in JMAP but the message flag is still unchanged in IMAP.</p></section><section> |
| <h4><a name="How_to_solve"></a>How to solve</h4> |
| <p>Execute the <a href="manage-webadmin.html#Fixing_messages_inconsistencies">solve Cassandra message inconsistencies task</a>. This task is not concurrent-safe. User actions concurrent to the inconsistency fixing task could result in new inconsistencies being created. However the source of truth imapUidTable will not be affected and thus re-running this task may eventually fix all issues.</p></section></section><section> |
| <h3><a name="Quotas"></a>Quotas</h3> |
| <p>User can monitor the amount of space and message count he is allowed to use, and that he is effectively using. James relies on an event bus and Cassandra to track the quota of an user. Upon Cassandra failure, this value can be incorrect.</p><section> |
| <h4><a name="How_to_detect_the_inconsistencies"></a>How to detect the inconsistencies</h4> |
| <p>Incorrect quotas could be seen in the Mail User Agent (IMAP or JMAP).</p></section><section> |
| <h4><a name="How_to_solve"></a>How to solve</h4> |
| <p>Execute the [recompute Quotas counters task](manage-webadmin.html#Recomputing current quotas for users). This task is not concurrent-safe. Concurrent operations will result in an invalid quota to be persisted. Re-running this task may eventually return the correct result.</p></section></section><section> |
| <h3><a name="RRT_.28RecipientRewriteTable.29_mapping_sources"></a>RRT (RecipientRewriteTable) mapping sources</h3> |
| <p>rrt and mappings_sources tables store information about address mappings. The source of truth is rrt and mappings_sources is the projection table containing all mapping sources.</p><section> |
| <h4><a name="How_to_detect_the_inconsistencies"></a>How to detect the inconsistencies</h4> |
| <p>Right now there’s no tool for detecting that, we’re proposing a <a class="externalLink" href="https://issues.apache.org/jira/browse/JAMES-3069">development plan</a>. By the mean time, the recommendation is to execute the SolveInconsistencies task below in a regular basis.</p></section><section> |
| <h4><a name="How_to_solve"></a>How to solve</h4> |
| <p>Execute the Cassandra mapping SolveInconsistencies task described in <a href="manage-webadmin.html#Operations_on_mappings_sources">webadmin documentation</a></p></section></section></section><section> |
| <h2><a name="Setting_Cassandra_user_permissions"></a>Setting Cassandra user permissions</h2> |
| <p>When a Cassandra cluster is serving more than a James cluster, the keyspaces need isolation. It can be achieved by configuring James server with credentials preventing access or modification of other keyspaces.</p> |
| <p>We recommend you to not use the initial admin user of Cassandra and provide a different one with a subset of permissions for each application.</p><section> |
| <h3><a name="Prerequisites"></a>Prerequisites</h3> |
| <p>We’re gonna use the Cassandra super users to create roles and grant permissions for them. To do that, Cassandra requires you to login via username/password authentication and enable granting in cassandra configuration file.</p> |
| <p>For example:</p> |
| |
| <div class="source"> |
| <div class="source"> |
| <pre>echo -e "\nauthenticator: PasswordAuthenticator" >> /etc/cassandra/cassandra.yaml |
| echo -e "\nauthorizer: org.apache.cassandra.auth.CassandraAuthorizer" >> /etc/cassandra/cassandra.yaml |
| </pre></div></div> |
| </section><section> |
| <h3><a name="Prepare_Cassandra_roles_.26_keyspaces_for_James"></a>Prepare Cassandra roles & keyspaces for James</h3><section> |
| <h4><a name="Create_a_role"></a>Create a role</h4> |
| <p>Have a look at <a class="externalLink" href="http://cassandra.apache.org/doc/3.11.11/cql/security.html">cassandra documentation</a> section CREATE ROLE for more information</p> |
| <p>E.g.</p> |
| |
| <div class="source"> |
| <div class="source"> |
| <pre>CREATE ROLE james_one WITH PASSWORD = 'james_one' AND LOGIN = true; |
| </pre></div></div> |
| </section><section> |
| <h4><a name="Create_a_keyspace"></a>Create a keyspace</h4> |
| <p>Have a look at <a class="externalLink" href="http://cassandra.apache.org/doc/3.11.11/cql/ddl.html">cassandra documentation</a> section CREATE KEYSPACE for more information</p></section><section> |
| <h4><a name="Grant_permissions_on_created_keyspace_to_the_role"></a>Grant permissions on created keyspace to the role</h4> |
| <p>The role to be used by James needs to have full rights on the keyspace that James is using. Assuming the keyspace name is james_one_keyspace and the role be james_one.</p> |
| |
| <div class="source"> |
| <div class="source"> |
| <pre>GRANT CREATE ON KEYSPACE james_one_keyspace TO james_one; // Permission to create tables on the appointed keyspace |
| GRANT SELECT ON KEYSPACE james_one_keyspace TO james_one; // Permission to select from tables on the appointed keyspace |
| GRANT MODIFY ON KEYSPACE james_one_keyspace TO james_one; // Permission to update data in tables on the appointed keyspace |
| </pre></div></div> |
| |
| <p><b>Warning</b>: The granted role doesn’t have the right to create keyspaces, thus, if you haven’t created the keyspace, James server will fail to start is expected.</p> |
| <p><b>Tips</b></p> |
| <p>Since all of Cassandra roles used by different James are supposed to have a same set of permissions, you can reduce the works by creating a base role set like typical_james_role with all of necessary permissions. After that, with each James, create a new role and grant the typical_james_role to the newly created one. Note that, once a base role set is updated ( granting or revoking rights) all granted roles are automatically updated.</p> |
| <p>E.g.</p> |
| |
| <div class="source"> |
| <div class="source"> |
| <pre>CREATE ROLE james1 WITH PASSWORD = 'james1' AND LOGIN = true; |
| GRANT typical_james_role TO james1; |
| |
| CREATE ROLE james2 WITH PASSWORD = 'james2' AND LOGIN = true; |
| GRANT typical_james_role TO james2; |
| </pre></div></div> |
| </section><section> |
| <h4><a name="Revoke_harmful_permissions_from_the_created_role"></a>Revoke harmful permissions from the created role</h4> |
| <p>We want a specific role that cannot describe or query the information of other keyspaces or tables used by another application. By default, Cassandra allows every role created to have the right to describe any keyspace and table. There’s no configuration that can make effect on that topic. Consequently, you have to accept that your data models are still being exposed to anyone having credentials to Cassandra.</p> |
| <p>For more information, have a look at <a class="externalLink" href="http://cassandra.apache.org/doc/3.11.11/cql/security.html">cassandra documentation</a> section REVOKE PERMISSION.</p> |
| <p>Except for the case above, the permissions are not auto available for a specific role unless they are granted by GRANT command. Therefore, if you didn’t provide more permissions than <a href="#Grant_permissions_on_created_keyspace_to_the_role">granting section</a>, there’s no need to revoke.</p></section></section></section><section> |
| <h2><a name="Cassandra_table_level_configuration"></a>Cassandra table level configuration</h2> |
| <p>While <i>Distributed James</i> is shipped with default table configuration options, these settings should be refined depending of your usage.</p> |
| <p>These options are: - The <a class="externalLink" href="https://cassandra.apache.org/doc/latest/operating/compaction.html">compaction algorithms</a> - The <a class="externalLink" href="https://cassandra.apache.org/doc/latest/operating/bloom_filters.html">bloom filter sizing</a> - The <a class="externalLink" href="https://cassandra.apache.org/doc/latest/operating/compression.html?highlight=chunk%20size">chunk size</a> - The <a class="externalLink" href="https://www.datastax.com/blog/2011/04/maximizing-cache-benefit-cassandra">caching options</a></p> |
| <p>The compaction algorithms allow a tradeoff between background IO upon writes and reads. We recommend: - Using <b>Leveled Compaction Strategy</b> on read intensive tables subject to updates. This limits the count of SStables being read at the cost of more background IO. High garbage collections can be caused by an inappropriate use of Leveled Compaction Strategy. - Otherwise use the default <b>Size Tiered Compaction Strategy</b>.</p> |
| <p>Bloom filters help avoiding unnecessary reads on SSTables. This probabilistic data structure can tell an entry absence from a SSTable, as well as the presence of an entry with an associated probability. If a lot of false positives are noticed, the size of the bloom filters can be increased.</p> |
| <p>As explained in <a class="externalLink" href="https://thelastpickle.com/blog/2018/08/08/compression_performance.html">this post</a>, chunk size used upon compression allows a tradeoff between reads and writes. A smaller size will mean decreasing compression, thus it increases data being stored on disk, but allow lower chunks to be read to access data, and will favor reads. A bigger size will mean better compression, thus writing less, but it might imply reading bigger chunks.</p> |
| <p>Cassandra enables a key cache and a row cache. Key cache enables to skip reading the partition index upon reads, thus performing 1 read to the disk instead of 2. Enabling this cache is globally advised. Row cache stores the entire row in memory. It can be seen as an optimization, but it might actually use memory no longer available for instance for file system cache. We recommend turning it off on modern SSD hardware.</p> |
| <p>A review of your usage can be conducted using <a class="externalLink" href="https://cassandra.apache.org/doc/latest/tools/nodetool/nodetool.html">nodetool</a> utility. For example nodetool tablestats {keyspace} allows reviewing the number of SSTables, the read/write ratios, bloom filter efficiency. nodetool tablehistograms {keyspace}.{table} might give insight about read/write performance.</p> |
| <p>Table level options can be changed using <b>ALTER TABLE</b> for example with the <a class="externalLink" href="https://cassandra.apache.org/doc/latest/tools/cqlsh.html">cqlsh</a> utility. A full compaction might be needed in order for the changes to be taken into account.</p></section><section> |
| <h2><a name="Mail_Queue"></a>Mail Queue</h2> |
| <p>An email queue is a mandatory component of SMTP servers. It is a system that creates a queue of emails that are waiting to be processed for delivery. Email queuing is a form of Message Queuing – an asynchronous service-to-service communication. A message queue is meant to decouple a producing process from a consuming one. An email queue decouples email reception from email processing. It allows them to communicate without being connected. As such, the queued emails wait for processing until the recipient is available to receive them. As James is an Email Server, it also supports mail queue as well.</p><section> |
| <h3><a name="Why_Mail_Queue_is_necessary"></a>Why Mail Queue is necessary</h3> |
| <p>You might often need to check mail queue to make sure all emails are delivered properly. At first, you need to know why email queues get clogged. Here are the two core reasons for that:</p> |
| <ul> |
| |
| <li>Exceeded volume of emails</li> |
| </ul> |
| <p>Some mailbox providers enforce email rate limits on IP addresses. The limits are based on the sender reputation. If you exceeded this rate and queued too many emails, the delivery speed will decrease.</p> |
| <ul> |
| |
| <li>Spam-related issues</li> |
| </ul> |
| <p>Another common reason is that your email has been busted by spam filters. The filters will let the emails gradually pass to analyze how the rest of the recipients react to the message. If there is slow progress, it’s okay. Your email campaign is being observed and assessed. If it’s stuck, there could be different reasons including the blockage of your IP address.</p></section><section> |
| <h3><a name="Why_combining_Cassandra.2C_RabbitMQ_and_Object_storage_for_MailQueue"></a>Why combining Cassandra, RabbitMQ and Object storage for MailQueue</h3> |
| <ul> |
| |
| <li>RabbitMQ ensures the messaging function, and avoids polling.</li> |
| <li>Cassandra enables administrative operations such as browsing, deleting using a time series which might require fine performance tuning (see <a class="externalLink" href="http://cassandra.apache.org/doc/latest/operating/index.html">Operating Casandra documentation</a>).</li> |
| <li>Object Storage stores potentially large binary payload.</li> |
| </ul> |
| <p>However the current design do not implement delays. Delays allow to define the time a mail have to be living in the mailqueue before being dequeued and is used for example for exponential wait delays upon remote delivery retries, or SMTP traffic rate limiting.</p></section><section> |
| <h3><a name="Fine_tune_configuration_for_RabbitMQ"></a>Fine tune configuration for RabbitMQ</h3> |
| <p>In order to adapt mail queue settings to the actual traffic load, an administrator needs to perform fine configuration tunning as explained in <a class="externalLink" href="https://github.com/apache/james-project/blob/master/src/site/xdoc/server/config-rabbitmq.xml">rabbitmq.properties</a>.</p> |
| <p>Be aware that MailQueue::getSize is currently performing a browse and thus is expensive. Size recurring metric reporting thus introduces performance issues. As such, we advise setting mailqueue.size.metricsEnabled=false.</p></section><section> |
| <h3><a name="Managing_email_queues"></a>Managing email queues</h3> |
| <p>Managing an email queue is an easy task if you follow this procedure:</p> |
| <ul> |
| |
| <li>First, <a href="manage-webadmin.html#Listing_mail_queues">List mail queues</a> and <a href="manage-webadmin.html#Getting_a_mail_queue_details">get a mail queue details</a>.</li> |
| <li>And then <a href="manage-webadmin.html#Listing_the_mails_of_a_mail_queue">List the mails of a mail queue</a>.</li> |
| <li>If all mails in the mail queue are needed to be delivered you will <a href="manage-webadmin.html#Flushing_mails_from_a_mail_queue">flush mails from a mail queue</a>.</li> |
| </ul> |
| <p>In case, you need to clear an email queue because there are only spam or trash emails in the email queue you have this procedure to follow:</p> |
| <ul> |
| |
| <li>All mails from the given mail queue will be deleted with <a href="manage-webadmin.html#Clearing_a_mail_queue">Clearing a mail queue</a>.</li> |
| </ul></section></section><section> |
| <h2><a name="Updating_Cassandra_schema_version"></a>Updating Cassandra schema version</h2> |
| <p>A schema version indicates you which schema your James server is relying on. The schema version number tracks if a migration is required. For instance, when the latest schema version is 2, and the current schema version is 1, you might think that you still have data in the deprecated Message table in the database. Hence, you need to migrate these messages into the MessageV2 table. Once done, you can safely bump the current schema version to 2.</p> |
| <p>Relying on outdated schema version prevents you to benefit from the newest performance and safety improvements. Otherwise, there’s something very unexpected in the way we manage cassandra schema: we create new tables without asking the admin about it. That means your James version is always using the last tables but may also take into account the old ones if the migration is not done yet.</p><section> |
| <h3><a name="How_to_detect_when_we_should_update_Cassandra_schema_version"></a>How to detect when we should update Cassandra schema version</h3> |
| <p>When you see in James logs org.apache.james.modules.mailbox.CassandraSchemaVersionStartUpCheck showing a warning like Recommended version is versionX, you should perform an update of the Cassandra schema version.</p> |
| <p>Also, we keep track of changes needed when upgrading to a newer version. You can read this <a class="externalLink" href="https://github.com/apache/james-project/blob/master/upgrade-instructions.md">upgrade instructions</a>.</p></section><section> |
| <h3><a name="How_to_update_Cassandra_schema_version"></a>How to update Cassandra schema version</h3> |
| <p>These schema updates can be triggered by webadmin using the Cassandra backend. Following steps are for updating Cassandra schema version:</p> |
| <ul> |
| |
| <li>At the very first step, you need to <a href="manage-webadmin.html#Retrieving_current_Cassandra_schema_version">retrieve current Cassandra schema version</a></li> |
| <li>And then, you <a href="manage-webadmin.html#Retrieving_latest_available_Cassandra_schema_version">retrieve latest available Cassandra schema version</a> to make sure there is a latest available version</li> |
| <li>Eventually, you can update the current schema version to the one you got with <a href="manage-webadmin.html#Upgrading_to_the_latest_version">upgrading to the latest version</a></li> |
| </ul> |
| <p>Otherwise, if you need to run the migrations to a specific version, you can use <a href="manage-webadmin.html#Upgrading_to_a_specific_version">Upgrading to a specific version</a></p></section></section><section> |
| <h2><a name="Deleted_Messages_Vault"></a>Deleted Messages Vault</h2> |
| <p>Deleted Messages Vault is an interesting feature that will help James users have a chance to:</p> |
| <ul> |
| |
| <li>retain users deleted messages for some time.</li> |
| <li>restore & export deleted messages by various criteria.</li> |
| <li>permanently delete some retained messages.</li> |
| </ul> |
| <p>If the Deleted Messages Vault is enabled when users delete their mails, and by that we mean when they try to definitely delete them by emptying the trash, James will retain these mails into the Deleted Messages Vault, before an email or a mailbox is going to be deleted. And only administrators can interact with this component via <a href="manage-webadmin.html#deleted-messages-vault">WebAdmin REST APIs</a>.</p> |
| <p>However, mails are not retained forever as you have to configure a retention period before using it (with one-year retention by default if not defined). It’s also possible to permanently delete a mail if needed and we recommend the administrator to <a href="#Cleaning_expired_deleted_messages">run it</a> in cron job to save storage volume.</p><section> |
| <h3><a name="How_to_configure_deleted_messages_vault"></a>How to configure deleted messages vault</h3> |
| <p>To setup James with Deleted Messages Vault, you need to follow those steps:</p> |
| <ul> |
| |
| <li>Enable Deleted Messages Vault by configuring Pre Deletion Hooks.</li> |
| <li>Configuring the retention time for the Deleted Messages Vault.</li> |
| </ul><section> |
| <h4><a name="Enable_Deleted_Messages_Vault_by_configuring_Pre_Deletion_Hooks"></a>Enable Deleted Messages Vault by configuring Pre Deletion Hooks</h4> |
| <p>You need to configure this hook in <a class="externalLink" href="https://github.com/apache/james-project/blob/master/server/apps/distributed-app/sample-configuration/listeners.xml">listeners.xml</a> configuration file. More details about configuration & example can be found at <a class="externalLink" href="http://james.apache.org/server/config-listeners.html">Pre Deletion Hook Configuration</a></p></section><section> |
| <h4><a name="Configuring_the_retention_time_for_the_Deleted_Messages_Vault"></a>Configuring the retention time for the Deleted Messages Vault</h4> |
| <p>In order to configure the retention time for the Deleted Messages Vault, an administrator needs to perform fine configuration tunning as explained in <a class="externalLink" href="https://github.com/apache/james-project/blob/master/server/apps/distributed-app/sample-configuration/deletedMessageVault.properties">deletedMessageVault.properties</a>. Mails are not retained forever as you have to configure a retention period (by retentionPeriod) before using it (with one-year retention by default if not defined).</p></section></section><section> |
| <h3><a name="Restore_deleted_messages_after_deletion"></a>Restore deleted messages after deletion</h3> |
| <p>After users deleted their mails and emptied the trash, the admin can use <a href="manage-webadmin.html#deleted-messages-vault">Restore Deleted Messages</a> to restore all the deleted mails.</p></section><section> |
| <h3><a name="Cleaning_expired_deleted_messages"></a>Cleaning expired deleted messages</h3> |
| <p>You can delete all deleted messages older than the configured retentionPeriod by using <a href="manage-webadmin.html#deleted-messages-vault">Purge Deleted Messages</a>. We recommend calling this API in CRON job on 1st day each month.</p></section></section> |
| </div> |
| </div> |
| <div class="clear"> |
| <hr/> |
| </div> |
| <div id="footer"> |
| <div class="xright">Copyright © 2006-2021 |
| <a href="https://www.apache.org/">The Apache Software Foundation</a>. |
| All Rights Reserved. |
| |
| </div> |
| <div class="clear"> |
| <hr/> |
| </div> |
| </div> |
| </body> |
| </html> |
