| Note for Users Upgrading to SpamAssassin 4.0.0 |
| ---------------------------------------------- |
| |
| Apache SpamAssassin 4.0.0 represents years of work by the project with |
| numerous improvements, new rule types, and internal native handling |
| of messages in international languages. We highly recommend looking |
| through this file and all of the .pre files to evaluate your |
| configuration thoroughly. Plugins have been added, removed, and |
| improved throughout. |
| |
| - All rules, functions, command line options and modules that contain |
| "whitelist" or "blacklist" have been renamed to contain more |
| racially neutral "welcomelist" and "blocklist" terms. This allows |
| acronyms like WL and BL to remain the same. Previous options will |
| continue work at least until version 4.1.0 is released. If you have |
| local settings including scores or meta rules referring to old rule |
| names, these should be changed and "enable_compat |
| welcomelist_blocklist" added in init.pre. See: |
| https://wiki.apache.org/spamassassin/WelcomelistBlocklist (Bug 7826) |
| |
| - Meta rules no longer use priority values, they are evaluated |
| dynamically when the rules they depend on are finished. (Bug 7735) |
| |
| - API: New $pms->rule_ready() function. Any asynchronous eval-function |
| must now return undef (instead of 0 or 1), if rule result is not |
| ready when exiting the function. $pms->rule_ready($rulename) or |
| $pms->got_hit(...) must be called when the result has arrived. If |
| these are not used, it can break depending meta rule evaluation. |
| |
| - Setting normalize_charset is now enabled by default. Note that rules |
| should not expect specific non-UTF8 or UTF8 encoding in |
| body. Matching is done against the raw data which may vary depending |
| on normalize_charset setting and whether decoding to UTF8 was |
| successful. See: |
| https://wiki.apache.org/spamassassin/WritingRulesAdvanced |
| |
| - DKIM plugin has added support for ARC signature verification |
| |
| - The DecodeShortURL plugin has been added and decodes URIs from URL |
| shorteners that may be used to evade scanning |
| |
| - Strings can now be captured from rules and later reused using the |
| special %{TAGNAME} syntax |
| |
| - The Bayes stopwords, or noise words, are now configurable in order |
| to optimize Bayes usage for non-English languages. Stopwords for 16 |
| foreign languages have been included. See 60_bayes_stopwords.cf in |
| the rules files. See Mail::SpamAssassin::Plugin::Bayes and the |
| bayes_stopword_languages option if you wish to use a different |
| stopword list. This is highly recommended if you are using Bayes and |
| you are processing messages in languages other than English. |
| |
| - The OLEVBMacro plugin has been improved to identify more macros |
| while also extracting uris from the attachments for automatic |
| inclusion in RBL lookups |
| |
| - Internationalized domain name (IDN) support has been added and |
| requires Net::LibIDN2 or Net::LibIDN module with a new |
| Util::idn_to_ascii() function. (Bug 7215) |
| |
| - Improved internal header address (From/To/Cc) parser, now also |
| handles multiple addresses and includes optional support for |
| external Email::Address::XS parser, which can handle nested comments |
| and other oddities. |
| |
| - Header :addr :name modifiers now return all addresses. Options of |
| :first :last select only first (topmost) or last header to process |
| when there are multiple headers with the same name. :addr and :name |
| may still return multiple values from a single header. |
| |
| - API: $pms->get() can and should now be called in list |
| context. Scalar context continues to return multiple values newline |
| separated, but this should be considered deprecated. |
| |
| - New ExtractText plugin that extracts text from documents or images |
| to feed the data into SpamAssassin for standard processing with |
| existing rules, URIs extracted from documents will fall into normal |
| RBL lookups. |
| |
| - New "nolog" tflag added to hide info coming from rules in |
| SpamAssassin reports |
| |
| - All log output (stderr, file, syslog) is now escaped properly for \r |
| \n \t \\, control chars, DEL, and UTF-8 sequences presented as |
| \x{XX}. Whitespace is not normalized anymore like in versions prior |
| to 4.0.0. |
| |
| - API: Logger::add() has new optional 'escape' parameter. New |
| Logger::escape_str() function. |
| |
| - API: New $pms->add_uri_detail_list() function. Also new |
| uri_detail_list types: unlinked, schemeless |
| |
| - Util::split_domain, trim_domain, and is_domain_valid functions have |
| a new optional argument ($is_ascii) |
| |
| - Header names support new :host :domain :ip :revip modifiers |
| |
| - AskDNS: tag HEADER(hdrname) supported to query any header content |
| similarly to header rules |
| |
| - The HashCash module and support has been removed completely, as it |
| has been long since deprecated |
| |
| - URILocalBL: uri_block_cc/uri_block_cont now support negation (Bug |
| 7528) |
| |
| - URILocalBL: IPv6 lookups for hosts is now support, if provided by |
| your database |
| |
| - DNS and other asynchronous lookups such as Pyzor and DCC are now |
| only launched when priority -100 is reached. This allows short |
| circuiting at a lower priority without sending unneeded DNS queries |
| and starting process forms. (Bug 5930) |
| |
| - API: New plugin method callback method check_dnsbl added to launch |
| network lookups at priority -100 and check_post_dnsbl to harvest own |
| network lookups |
| |
| - API: New plugin callback method check_cleanup for cleaning up |
| things... |
| |
| - FreeMail: new options freemail_import_welcomelist_auth and |
| freemail_import_def_welcomelist_auth added (Bug 6451) |
| |
| - New internal Mail::SpamAssassin::GeoDB module that provides a |
| unified interface to modules MaxMind::DB::Reader (GeoIP2), Geo::IP, |
| IP::Country::DB_File, and IP::Country::Fast. |
| |
| This is utilized by RelayCountry and URILocalBL with settings |
| geodb_module, geodb_options, and geodb_search_path. |
| |
| Deprecated settings still work such as country_db_type, |
| country_db_path, uri_country_db_path, and uri_country_db_isp_path |
| but will print a warning to migrate to geodb_module/options. |
| |
| - Razor2 razor_fork option added to create separate Razor2 processes |
| and read in the results later asynchronously, increasing throughput, |
| and automatically adjusting rule priorities to -100. |
| |
| - DCC checks are now done asynchronously if using dccifd, improving |
| throughput. With dccifd, rule priorities are automatically adjusted |
| to -100. Commercial reputation rules can be ignored with the option |
| "use_dcc_rep 0" to save a few CPU cycles. |
| |
| - Pyzor pyzor_fork option added to create separate Pyzor processes and |
| read in the results later asynchronously, increasing throughput, and |
| automatically adjusting rule priorities to -100. Renamed pyzor_max |
| setting to pyzor_count_min. Added pyzor_welcomelist_min and |
| pyzor_welcomelist_factor setting. Also try to improve false |
| positives by ignoring "empty body" messages. |
| |
| - API: deprecated $pms->register_async_rule_start() and |
| $pms->register_async_rule_finish() calls though left in for |
| backwards compatibility. Plugins should only use |
| $pms->bgsend_and_start_lookup(), which handles required things |
| Automatically. Direct calls to bgsend or start_lookup should not be |
| used. $pms->bgsend_and_start_lookup() should always contain |
| $ent->{rulename} for correct meta dependency handling. Deprecated |
| start_lookup, get_lookup, lookup_ns, harvest_until_rule_completes, |
| and is_rule_complete. |
| |
| - SPF: Mail::SPF is now the only supported perl module and |
| Mail::SPF::Query is deprecated along with the settings |
| do_not_use_mail_spf, and do_not_use_mail_spf_query. SPF lookups are |
| not done asynchronously so using an MTA filter such as pypolicyd-spf |
| or spf-engine can generate Received-SPF for SpamAssassin to parse. |
| |
| - "ALL" pseudo-header now returns decoded headers, so it's usage is |
| consistent with single header matching. Using the :raw option mimics |
| the previous behavior of with undecoded and folded headers. |
| |
| - New dns_block_rule option handles blocked DNSBLs (Bug 6728) |
| |
| - ASN: Support GeoDB for ASN lookups (asn_use_geodb, asn_prefer_geodb, |
| asn_use_dns). |
| |
| - ASN: Default sa-update ruleset doesn't make ASN lookups or add |
| headers anymore. Configure desired methods, asn_use_geodb or |
| asn_use_dns, and add_header clauses manually as described in the |
| plugin documentation. Usage of asn_use_geodb without DNS is |
| recommended unless ASNCIDR is needed. Do not use rules that check |
| metadata X-ASN header! Only the new eval function check_asn() |
| described in plugin manual works reliably. |
| |
| - sa-update: New --score-multiplier, --score-limit, and --forcemirror |
| options added. |
| #1 forcemirror: forces sa-update to use a specific mirror server, |
| #2 score-multiplier: adjust all scores from update channel by a |
| given multiplier to quickly level set scores to match your |
| preferred threshold |
| #3 score-limit adjusts all scores from update channel over a |
| specified limit to a new limit |
| |
| - New dns_options "nov4" and "nov6" added. IMPORTANT:; You must set |
| nov6 if your DNS resolver is filtering IPv6 AAAA replies. |
| |
| - API: Added Message::get_pristine_body_digest(), |
| Message::get_msgid(), and Message::generate_msgid() |
| functions. Removed deprecated private Plugin::Bayes::get_msgid() |
| function. |
| |
| - Bayes and TxRep seen Message-ID tracking hashing method changed. No |
| actions are required. If re-learning some old messages, they might |
| be learned twice but old IDs should expire automatically. |
| |
| - report_charset defaults now to UTF-8. |
| |
| - Meta rules inherit net tflag setting from dependencies (Bug 7735) |
| |
| - BodyEval: Added plaintext_body_sig_ratio eval rules for the first |
| text/plain MIME part's body and signature length ratio. |
| |
| - API: Now supports multiple calls of $pms->test_log() for |
| rules. Added $pms->check_cleanup() to finalize tags, reports, |
| etc. Deprecated internal $pms->{test_log_msgs}, renamed to |
| $pms->{test_logs}. Deprecated $pms->clear_test_state() as it is not |
| needed anymore. $pms->test_log() now accepts $rulename as second |
| argument. |
| |
| - URIDNSBL: urirhsbl/urirhssub rules support "notrim" tflag to force |
| querying the full hostname instead of just the domain. This works |
| best if the specific uribl supports this mode. (Bug 7835) |
| |
| - Removed deprecated --auth-ident and --ident-timeout options from |
| spamd |
| |
| - MIMEHeader: support matching ALL header, tflags range, and tflags |
| concat |
| |
| - Autolearn: add new tflags autolearn_header/autolearn_body. These can |
| force a rule to count as header or body points accordingly. (Bug |
| 7907) |
| |
| - SSL client certificate support for spamc/spamd is now easier. New |
| spamc options --ssl-cert, --ssl-key, --ssl-ca-file, and |
| --ssl-ca-path. New spamd options --ssl-verify, --ssl-ca-file, and |
| --ssl-ca-path (Bug 7267) |
| |
| - ArchiveIterator now automatically uncompressed all gzip, bzip2, xz, |
| lz4, lzip, and lzo-compressed files (Bug 7598). These apply to |
| spamassassin and sa-learn commands also. |
| |
| - New DMARC policy check plugin. |
| |
| - New project maintained DecodeShortURLs plugin which may not be |
| directly compatible with rules from other third party plugins. See |
| The plugin documentation for configuration and rule format. |
| |
| - Installing module Net::CIDR::Lite allows the use of dash-separated |
| IP range format (e.g. 192.168.1.1-192.168.255.255) for NetSet tables |
| including internal_networks, trusted_networks, msa_networks, and |
| uri_local_cidr. |
| |
| - The HashBL plugin in v342.pre is now enabled by default. |
| |
| - HeaderEval check_for_unique_subject_id() function is deprecated. |
| |
| (end of UPGRADE) |