| <!DOCTYPE html><html><head><meta charset="utf-8"><title>Apache Pony Mail (Incubating)</title> |
| <link rel="stylesheet" type="text/css" href="/css/default.css"/> |
| <link rel="stylesheet" type="text/css" href="/css/fa/fa.css"/> |
| <link rel="apple-touch-icon" sizes="57x57" href="/icons/apple-icon-57x57.png"> |
| <link rel="apple-touch-icon" sizes="60x60" href="/icons/apple-icon-60x60.png"> |
| <link rel="apple-touch-icon" sizes="72x72" href="/icons/apple-icon-72x72.png"> |
| <link rel="apple-touch-icon" sizes="76x76" href="/icons/apple-icon-76x76.png"> |
| <link rel="apple-touch-icon" sizes="114x114" href="/icons/apple-icon-114x114.png"> |
| <link rel="apple-touch-icon" sizes="120x120" href="/icons/apple-icon-120x120.png"> |
| <link rel="apple-touch-icon" sizes="144x144" href="/icons/apple-icon-144x144.png"> |
| <link rel="apple-touch-icon" sizes="152x152" href="/icons/apple-icon-152x152.png"> |
| <link rel="apple-touch-icon" sizes="180x180" href="/icons/apple-icon-180x180.png"> |
| <link rel="icon" type="image/png" sizes="192x192" href="/android-icon-192x192.png"> |
| <link rel="icon" type="image/png" sizes="32x32" href="/icons/favicon-32x32.png"> |
| <link rel="icon" type="image/png" sizes="96x96" href="/icons/favicon-96x96.png"> |
| <link rel="icon" type="image/png" sizes="16x16" href="/icons/favicon-16x16.png"> |
| <link rel="manifest" href="/icons/manifest.json"> |
| <meta name="msapplication-TileColor" content="#ffffff"> |
| <meta name="msapplication-TileImage" content="/icons/ms-icon-144x144.png"> |
| <meta name="theme-color" content="#ffffff"> |
| </head><body> |
| <div id="titlebar"> |
| <a href="/contribute.html"><img align='left' style="width: 130px; height: 125px; position: relative; left: -6px; top: -6px; border: 0;" src="/images/devme.png" alt="Fork/Hack on Pony Mail"></a> |
| <a href='/'><img src="/images/ponymail.svg" style="width: 110px; margin-left: -10px; margin-right: 20px; height: auto;" align="left"/></a> |
| <h1><a id="title" href="/" style="color: #FFF !important;">Apache Pony Mail™ (Incubating)</a></h1> |
| <div id="menubar"> |
| <ul> |
| <li><a href="/docs.html"><i class="fa fa-book"></i><span>Documentation</span></a></li> |
| <li><a href="/source.html"><i class="fa fa-git-square"></i><span>Source</span></a></li> |
| <li><a href="/downloads.html"><i class="fa fa-cloud-download"></i><span>Download</span></a></li> |
| <li><a href="/support.html"><i class="fa fa-question-circle"></i><span>Get support</span></a></li> |
| <li><a href="/contribute.html"><i class="fa fa-share-alt"></i><span>Contribute</span></a></li> |
| <li><a href="/about.html"><i class="fa fa-users"></i><span>About</span></a></li> |
| </ul> |
| </div> |
| </div> |
| <h1 id='archivingnewemailstoponymail'>Archiving New Emails to Pony Mail<a href='#archivingnewemailstoponymail' style='color: rgba(0,0,0,0);'>¶</a></h1> |
| <p>This document exists to extend the <a href="installing.html">general install guide</a> provide examples on how to archive emails.</p> |
| <p><strong>Note:</strong> If you plan on <a href="importing.html">importing old emails from an archive</a>, |
| please set up the archiver <strong>first</strong> so as to create an overlap of new emails |
| coming in and old emails being imported. The system is designed to handle this |
| without creating duplicate entries in the archive.</p> |
| <h2 id='mailman2xexample'>Mailman 2.x example:<a href='#mailman2xexample' style='color: rgba(0,0,0,0);'>¶</a></h2> |
| <p>Set up a Pony Mail mail account/alias on a machine. This can be your local mail |
| server, it can be the machine that Pony Mail is on (install sendmail or postfix |
| etc there), or it can be any other machine with access to the ElasticSearch |
| database that Pony Mail uses.</p> |
| <h3 id='prerequisites'>Pre-requisites<a href='#prerequisites' style='color: rgba(0,0,0,0);'>¶</a></h3> |
| <p>If this is not the machine Pony Mail was installed on, you'll need to copy the |
| tools/ directory from your Pony Mail installation to this machine and adjust |
| ponymail.cfg to point to the right place for the database. You will also need |
| Python 3 and the helper libraries installed |
| (<code>pip3 install elasticsearch formatflowed netaddr</code>)</p> |
| <h3 id='createanalias'>Create an alias:<a href='#createanalias' style='color: rgba(0,0,0,0);'>¶</a></h3> |
| <p>Set up a mail alias for public and private lists in <code>/etc/aliases</code> or similar method, |
| and point them at the archiver script in tools/:</p> |
| <pre> |
| # You may need to add "--altheader delivered-to" to these commands, it varies |
| foo-public: "|/usr/bin/python3 /path/to/tools/archiver.py" |
| foo-private: "|/usr/bin/python3 /path/to/tools/archiver.py --private" |
| </pre> |
| |
| <p>Once done, run <code>newaliases</code> to update your alias DB.</p> |
| <h3 id='subscribethealiasestoyourmailinglists'>Subscribe the aliases to your mailing lists<a href='#subscribethealiasestoyourmailinglists' style='color: rgba(0,0,0,0);'>¶</a></h3> |
| <p>Use the mailman UI or CLI to subscribe foo-public@ to your public lists and |
| foo-private to your private lists. Don't worry, the contents of private lists |
| are hidden by default till the correct AAA scripting is set up.</p> |
| <h2 id='ezmlmexample'>ezmlm example:<a href='#ezmlmexample' style='color: rgba(0,0,0,0);'>¶</a></h2> |
| <p>First, see the general introduction in the MM2 example, as this applies here as well.</p> |
| <h3 id='createanalias'>Create an alias:<a href='#createanalias' style='color: rgba(0,0,0,0);'>¶</a></h3> |
| <p>Set up a dot-forward file for a public and a private alias:</p> |
| <pre> |
| .qmail-archive-public: |
| "|/usr/bin/python3 /path/to/tools/archiver.py" |
| |
| .qmail-archive-private: |
| "|/usr/bin/python3 /path/to/tools/archiver.py --private" |
| </pre> |
| |
| <h3 id='subscribethealiasestoyourmailinglists'>Subscribe the aliases to your mailing lists<a href='#subscribethealiasestoyourmailinglists' style='color: rgba(0,0,0,0);'>¶</a></h3> |
| <p>Use the ezmlm CLI to subscribe your new aliases to the lists: |
| <code>ezmlm-sub foolist/ archive-public@yourhost.tld</code> |
| <code>ezmlm-sub secretlist/ archive-private@yourhost.tld</code></p> |
| <h2 id='settingupaaa'>Setting up AAA<a href='#settingupaaa' style='color: rgba(0,0,0,0);'>¶</a></h2> |
| <p>If you have an custom OAuth2 provider and a binary approach to private access |
| (either/or), you can enable private access to people by having a key/value pair |
| called <code>isMember</code> set to <code>true</code> in your JSON response from the OAuth server, |
| provided it is set as an authority in config.lua. This will provide anyone |
| defined as a member via OAuth full access to all private lists.</p> |
| <p>If you use LDAP, you can modify the LDAP queries in the example AAA file to suit |
| your organization.</p> |
| <h2 id='usingtherightidgenerator'>Using the right ID generator<a href='#usingtherightidgenerator' style='color: rgba(0,0,0,0);'>¶</a></h2> |
| <p>Pony Mail relies on ID generators to produce a unique document ID for each email. |
| Depending on the setup of your installation, we recommend using one of two generators:</p> |
| <ul> |
| <li>If you have a single-node setup, we recommend using the <code>full</code> generator. This generator |
| uses the full message body, including MTA trails (Received headers etc), to generate an ID.</li> |
| <li>If you have a clustered/redundant setup (multiple archiver nodes), we recommend using the |
| <code>cluster</code> generator. This will use as many unique traits in an email as possible, but |
| discard the MTA trail (which differs from machine to machine).</li> |
| </ul> |
| <p>We do not recommend using the older generators (<code>medium</code> and <code>short</code>) |
| for new installations, as they are theoretically prone to create |
| duplicate IDs (See issues #176 #177 #178).</p> |
| <h2 id='importingarchivinghtmlonlyemails'>Importing/Archiving HTML-only emails<a href='#importingarchivinghtmlonlyemails' style='color: rgba(0,0,0,0);'>¶</a></h2> |
| <p>Should you need to import HTML-only emails into the archive, you may enable this |
| with the <code>--html2text</code> command line arg. This requires that the <code>html2text</code> Python3 package |
| is installed beforehand.</p> |
| <h2 id='munginglistids'>Munging list IDs<a href='#munginglistids' style='color: rgba(0,0,0,0);'>¶</a></h2> |
| <p>If you need to rewrite list IDs on the fly as emails come in, you can use the debug.cropout |
| setting for this (in <code>ponymail.cfg</code>). </p> |
| <p>You can either use it to just crop away something: |
| <pre> |
| [debug] |
| # Remove 'foo' from all list IDs |
| cropout: foo |
| </pre></p> |
| <p>Or you can use it as a regex substitution: |
| <pre> |
| [debug] |
| #Replace '<em>.bar.tld' with '</em>.blorg.bar.tld' |
| cropout: <([a-z]+).bar.tld> <\1.blorg.bar.tld> |
| </pre></p> |
| <div style="display: inline-block; background: #BBB; margin: -10px; padding: 10px;"> |
| <h4><a id="disclaimer"></a>Disclaimer</h4> |
| <div style="width: 65%; float: left;"> |
| <p style="line-height: 12pt;"> |
| Apache Pony Mail (Incubating) is an effort undergoing incubation at |
| The Apache Software Foundation (ASF), sponsored by the <a href="https://incubator.apache.org"> |
| Apache Incubator</a>. Incubation is required of all newly accepted projects |
| until a further review indicates that the infrastructure, |
| communications, and decision making process have stabilized in a |
| manner consistent with other successful ASF projects. While |
| incubation status is not necessarily a reflection of the |
| completeness or stability of the code, it does indicate that the |
| project has yet to be fully endorsed by the ASF. |
| </p> |
| </div> |
| <div style="width: 20%; float: left; padding-top: 16px; margin-left: -8px;"> |
| <a href="https://incubator.apache.org"><img src="https://incubator.apache.org/images/incubator_feather_egg_logo_sm.png"/></a> |
| </div> |
| <div style="width: 15%; float: left; line-height: 15px; padding-left: 22px;"> |
| <a class="item" target="_blank" href="https://incubator.apache.org/">Apache Incubator</a><br/> |
| <a class="item" target="_blank" href="https://www.apache.org/">About the ASF<span></span></a><br/> |
| <a class="item" target="_blank" href="https://www.apache.org/foundation/policies/conduct.html">Code of Conduct<span></span></a><br/> |
| <a class="item" target="_blank" href="https://www.apache.org/foundation/thanks.html">Thanks<span></span></a><br/> |
| <a class="item" target="_blank" href="https://www.apache.org/foundation/sponsorship.html">Become a Sponsor<span></span></a><br/> |
| <a class="item" target="_blank" href="https://www.apache.org/security/">Security<span></span></a><br/> |
| <a class="item" target="_blank" href="https://www.apache.org/licenses/LICENSE-2.0">License<span></span></a><br/> |
| </div> |
| </div> |
| </body></html> |
