======================= GNU Mailman 3 changes ======================= Copyright (C) 1998-2022 by the Free Software Foundation, Inc. 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA Here is a history of user visible changes to Mailman. .. _news-3.3.10: 3.3.10 ====== (xxxx-xx-xx) Bugs fixed ---------- * The ``mailman members`` command will now fall back to the member's user record display_name if the address record has none. (Closes #1107) * The user_name_or_email substitution in regular header/footer templates now works (Closes #1108) * The output from the ``mailman lists`` command no longer has trailing blanks and the command with the ``-d`` option doesn't fail with very long list ids. (Closes #1117 and #1118) * The text of the AlreadySubscribedError is now translatable. (Closes #1121) * The ``mailman create`` command now has a ``--style-name NAME`` option. (Closes #1136) * The ``mailman import21`` command adjust spaces in ``subject_prefix`` for Mailman 2.1 compatibility. (Closes #1135) New Features ------------ * One can now add entries like ``@list@example.com`` to the ``accept_these_nonmembers`` attribute of a list to accept nonmember posts from members of another list. (Closes #794) * REST API now allows forwarding the held message to a new address when handling the message. (Fixes #174) * Add the ability to set web URLs by setting domain's ``base_url`` parameter which allows using parameters like ``$held_message_url`` in the templates. (See #981) Other ----- * The Mailman master process that starts the runners will now pass a number of environment variables to the runner processes that alter the behavior of the Python interpreter. See section ENVIRONMENT VARIABLES in `python(1)`. Packaging --------- * Migrate to using the new standard `pyproject.toml` for Python packaging instead of setup.py. This also moves from setuptools to pdm to be used as the build backend. .. _news-3.3.9: 3.3.9 ===== (2023-10-20) Configuration ------------- * **BREAKING CHANGE**: ``logging.http`` section has now been split into two parts, ``logging.http`` is for the Mailman internal code and ``logging.gunicorn`` is for Gunicorn specifically since they have different logging formats. This should fix suprious errors with ``KeyError: t`` that was often confusing for users. (Fixes #1049) * Python 3.9 is now the minimum supported version of Python. Bugs fixed ---------- * ``config.mta.remove_dkim_headers`` now applies to messages to -owner. (Closes #1053) * The master process now robustly detects when runner processes terminate and restarts them (Closes #898). * Setting Postfix transport_file_type: regex without setting something for postmap_command: no longer throws an exception. (Closes #1058) * When bounce processing disables delivery for a user, the user's score is reset so it will be zero if delivery is enabled. (Closes #1061) * The ``mailman shell`` command now works and displays the banner with ``use_ipython: yes``. (Closes #1062) * Bogus Message-ID headers that have been observed in the wild are now fixed. (Closes #1065) * Raising StopIteration in a workflow process no longer creates a mailman.error log entry. (Closes #1059) * The task runner now evicts old, processed bounce events. (Closes #1067) * We now require authheaders>=0.15.2. (Closes #1064) * Digests no longer contain a bogus Archived-At header. (Closes #1069) * Content filtering will no longer add its report to a non-mixed multipart message but will wrap the message and add the report to the wrapper. (Closes #1071) * The administrivia rule will now decode encoded message bodies. (Closes #1072) * Email address validation now accepts quoted local parts and @ in local part. (Closes #1077) * The default for max_recipients has been reduced to 10 to avoid possible lost mail when a large number of invalid recipients causes the MTA to disconnect. (Closes #1083) * Messages preserved or forwarded by content filtering are now the original message rather than the content filtered one. (Closes #1089) * List welcome and goodbye messages to users with English preferred_language now have their Subject: RFC 2047 encoded as needed. (Closes #1098) * Bounce processing notifications due to a DSN for multiple users now have the DSN attached to each notice. (Closes #1101) New Features ------------ * There is a new ``dmarc_addresses`` list attribute which is a list of email addresses and patterns matching email addresses. If a post is From: an address matching one of these, dmarc mitigations will be applied regardless of the From: domain's dmarc policy. (Closes #1084) Other ----- * Typo corrected under "Building the documentation" section of ``contribute.rst``. (Fixes #1075) .. _news-3.3.8: 3.3.8 ===== (2023-01-04) Bugs fixed ---------- * The deprecated SQLAlchemy query.values() method is replaced by query.with_entities(). This fixes an issue with MariaDB that truncated the generated postfix_lmtp file. (Closes #1044) * An uncaught passlib.exc.UnknownHashError in utilities/passwords.py is now caught and handled appropriately. (Closes #1046) * Rejection notices for a message with an RFC 2047 encoded Subject: now display the decoded subject. (Closes #672) * Don't RFC 2047 encode display names in UserNotifications. Allow sending utf-8 encoded notifications. (Closes #673) * Don't replace non-ascii in subjects in notifications. (Closes #673) Command line ------------ * The ``mailman members`` command no longer RFC 2047 encodes non-ascii display names. (Closes #1048) Performance ----------- * Improve the performance of Held message handling by reducing the total number of database calls required. (Fixes #1026 and #1045) Python Support -------------- * Add support for Python 3.10. (Closes #936) * Add support for Python 3.11 (See !1076) .. _news-3.3.7: 3.3.7 ===== (2022-11-10) **BREAKING CHANGE**: When using PostgreSQL database with Mailman, starting with Mailman Core 3.3.7 (due to bump in SQLAlchemy 1.4+), the urls in ``[database]`` section will require ``postgresql`` as the scheme. Previously, ``postgres`` was also accepted. This is an example config:: [database] class: mailman.database.postgresql.PostgreSQLDatabase url: postgresql://myuser:mypassword@mypghost/mailman Dependencies ------------ * Add support for SQLAlchemy 1.4+ (Closes #964) Python Support -------------- * Drop support for Python 3.6 since SQLAlchemy 2.0 will drop support for it and it is EOL by now. (See !1035) Bugs fixed ---------- * The fix for #994 in 3.3.6 blocked too many subscription attempts. This is now corrected and another test added. .. _news-3.3.6: 3.3.6 ===== (2022-10-26) Bugs fixed ---------- * The ``mailman members`` command reports incompatible options that would otherwise throw exceptions. * Require authheaders >=0.14.0 and adjust tests accordingly. (Closes #954) * Handling a held message will not remove it from the message store if there are other requests for it. (Closes #955) * ARC signing now signs after the message has been personalized and decorated. (Closes #896) * The prior fix for expanding replacements in the ``list:user:notice:goodbye`` template was incomplete. This is fixed. (Closes #922) * Decoration of an html message containing non-ascii resulted in a message that couldn't by flattened as_bytes and whose as_string representation contained non-ascii. This is fixed. (Closes #965 and #967) * The REST API will now accept JSON encoded data with boolean values for boolean attributes. (Closes #970) * Fix a bug where loading a plugin which raises an exception in ``pre_hook`` can cause Mailman to crash trying to disable that plugin. (Closes #724) * Fix a bug where the ``PIDWatcher()`` iterates over internal dict while it can be updated during that time. (Closes #724) * The ``mailman import21`` now always imports owners/moderators with ``DeliveryStatus.enabled``. (Closes #977) * A specific nonmember moderation action will be checked for and applied before testing the legacy ``*_these_nonmembers`` settings. (Closes #978) * Invalid regexps in ``*_these_nonmembers`` are properly logged and can't be set via REST. (Closes #974) * Override the as_bytes() method in the mailman.email.message.Message class to workaround https://bugs.python.org/issue41307. (Closes #979 and #980) * The nonmember moderation rule now checks all senders, not just the first. (Closes #986) * The i18n gettext replacements now all use curly brace notation to avoid issues whith translations that might follow them immediately with a period. The various mailman.po files have been updated accordingly. (Closes #987) * The ``mailman import21`` command no longer creates unnecessary entries in the ``template`` table. (Closes #988) * DMARC munge from mitigation will now find a nonmember poster's display name. (Closes #989) * Subscription workflow will now find an existing User when subscription is for an Address and vice versa. (Closes #994) * The subject of Welcome and Goodbye messages is now translated. (Closes #993) * Emails to a list's ``-join``, ``-leave`` or ``-confirm`` address that result in some errors will now produce a response. (Closes #991) * RFC2047 encoded Subject: and From: headers reported in email command replies are now decoded. (Closes #999) * The ``mailman import21`` command now converts more template replacements and reports to stderr any remaining unconverted replacements. (See #870) * The postfix generated virtual mappings now have correct spacing with alias_domains. (Closes #1001 and #1013) * Updated outgoing runner's fake DSN to use list addresses rather than mailman@example.com in From: and To:. (Closes #1018) * Subject prefixing on lists with non-ascii preferred language no longer drops parts of folded subject headers. (Closes #1021) * The ``verp_confirmations`` setting has now been implemented with default of ``yes``. (Closes #1023) * The ``anonymous_list_keep_headers`` setting now only applies to X-* headers and will not cause non X-* headers to be removed. (Closes #1031) * The ``mailman.mta.connection.sendmail()`` method now accepts a string, bytes or a Message object and ``BaseDelivery`` now passes a Message object. (See #1022) * The ``mailman.app.inject.inject_text()`` function now creates the Message object from bytes. (Closes #1024) * The ``mailman import21`` command now adds ``^listname@`` to the list's ``acceptable_aliases`` for compatiblity with Mailman 2.1's behavior. (Closes #1032) * An ``AttributeError`` thrown in the task runner, possibly due to a race condition, is now avoided. (Closes #1028) * Email confirmation of a subscription request for a member is no longer shunted. (Closes #1035) REST ---- * Expose bounce related parameters for Member objects. Command line ------------ * The ``mailman members`` command now has a ``--count-only`` option. (Closes #1030) New Features ------------ * The LMTP service now only accept ``RCPT TO`` commands if the given recipient address is acceptable (See !1043). .. _news-3.3.5: 3.3.5 ===== (2021-09-28) Bugs fixed ---------- * Bounce runner now properly commits database transactions. (Closes #850) * Pending subscriptions now have a lifetime equal to the configured ``pending_request_life`` rather than 3650 days. (Closes #729) * Held messages which are handled are now removed from the message store and the pending db. (Closes #257) * Admin notices sent when a list member's delivery is disabled by bounce or when a member's bounce score is incremented, now contain a copy of the triggering DSN. (Closes #737) * Handling a held message now also removes the poster's pending ``held message`` from the pending db. (Closes #257) * Deleting a pending request now also deletes any associated workflow. (See #853) * A new runner which is not a queue runner and which doesn't override its ``run`` or ``_one_itiration`` methods can now be implemented. (Closes #866) * The ``FileNotFound`` exception thrown in CacheManager on attempt to delete an entry for a missing file is now caught. (Closes #867) * Pending probe bounce tokens now have a lifetime of 10 days. (Closes #869) * Improve the performance of ``/users`` API when paginating by doing the pagination in database layer. (Closes #876) * Attempts to get a message from the message store with a missing file are now handled. (Closes #877) * The task runner no longer prematurely deletes saved DSNs. (Closes #878) * Bounce probe messages now contain the DSN as advertised. (Closes #880) * The avoid_duplicates handler properly handles headers that are returned as email.header.Header instances rather than strings. (Closeds #881) * The mta.deliver module properly handles headers that are returned as email.header.Header instances rather than strings. (Closeds #882) * Places where ``to_list`` in the message metadata were increctly referenced as ``tolist`` have been corrected. (Closes #883) * It is now possible to confirm held messages by email as the admin notice says. (Closes #169) * Some exceptions in ARC signing of some posts from HyperKitty and some from prod.outlook.com are now handled without shunting the message. (Closes #885) * Command runner now will decode the message body before processing it. (Closes #859) * The ``mailinglist`` table ``info``, ``autoresponse_owner_text``, ``autoresponse_postings_text`` and ``autoresponse_request_text`` columns are changed to Text. (Closes #840, #886 and #925) * The mailing list administrators roster ``get_member()`` method now returns the owner if the target is both an owner and moderator. (Closes #888) * Command runner now handles RFC 2047 encoded command with non-ascii prefix. (Closes #858) * Improve performance of the bounce runner by decreasing the number of database queries executed. * The master watcher will now restart a process that exits. (See #887) * When using MySQL, most database columns that expect user input will now accept emojis and other 4-byte unicode characters by using the utf8mb4 character set instead of 3-byte. (Closes #891) * LMTP runner now sets unixfrom in incoming messages. (Closes #904) * The ``Message`` ``senders()`` method now RFC2047-decodes and unfolds headers. (Closes #903) * The ``Message`` ``senders()`` method now gets all the addresses from the configured headers. (Closes #905) * Require flufl.bounce >=3.0.2, which improves parsing of bounce messages. (Closes #892) * DMARC policy discovery ignores domains with multiple DMARC records per RFC 7849. (Closes #907) * Held messages missing from the message store can now be accepted and/or forwarded. (closes #914) * Dispose SQLAlchemy connections after a new worker process is forked. (Closes #854) * Exceptions in bounce processing member removal don't roll back successful removals. (Closes #909) * Only remove or send warnings to list members when processing bounces. (Closes #910) * Replacements are now expanded in the ``list:user:notice:goodbye`` template. (Closes #922) * The nntp runner no longer folds long headers in newsgroup posts. (Closes #919) * RFC 2047 encoded subject headers will now be recoded in the character set of the list's preferred_language during subject prefixing if possible. (Closes #923) * DMARC mitigation wrap message now ensures existing cc and reply-to headers are included in the wrapper. (Closes #926) * The tagger handler now stringifies any Header instances. (Closes #928) * Invitation and (un)subscription confirmation subjects are now translated to the list's preferred language. (Closes #930) * The ``mailman syncmembers`` command now catches and reports a ``SubscriptionPendingError``. (Closes #929) * Trailing spaces are no longer removed from header and footer templates. (Closes #932) * The email ``join`` command now handles RFC2047 encoded display names with embedded commas. (Closes #933) * The ``gatenews`` command now parses messages with email.message_from_bytes without specifying policy. (Closes #934) * Improve error message from REST API for invalid email address (Fixes #872) * Nonmember posts gated from usenet bypass nonmember moderation. (Closes #937) * VERPed vacation responses and similar are no longer scored as bounces. (Closes #939) * If a message is held by a ``HeaderMatchRule``, the header name is now included in the reason for hold. (Closes #785) Command line ------------ * The ``notify`` subcommand is now more tolerant of certain database anomalies. (Closes #861) * The ``notify`` subcommand now handles unicode errors in decoding RFC 2047 encoded subject headers. (Closes #915) * The ``mailman`` command will refuse to run as root unless the new ``--run-as-root`` option is specified. (Closes #776 and #920) REST ---- * Fixed an exception on retreiving held messages if the held request exists, but the message is missing. (Closes #856) * Add a new ``/users/find`` endpoint to allow searching for users by matching display_name and user's email address. (Closes #874) New Features ------------ * There is a new ``bounce_notify_admin_on_bounce_increment`` list setting and a corresponding ``list:admin:notice:increment`` template for the notice. This setting, if True, will cause a notice to be sent to the list admins when a member's bounce score is incremented, but doesn't disable delivery. * There is a new ``moderator_request_life`` setting in the ``[mailman]`` section of mailman.cfg that controls the lifetime of pending moderator requests as opposed to pending user confirmations. Default is 180 days. * There is a new ``task`` runner to do periodic tasks. The current implementation evicts expired pendings and cache entries and removes orphaned workflows and orphaned message store messages. It does these tasks at intervals defined by the new ``run_tasks_every`` setting in the ``[mailman]`` section of mailman.cfg. Default is 1 hour. (Closes #467 and #853) * There is a new ``list:admin:notice:pending`` template for the notice from the ``mailman notify`` command. (Closes #890) * The ``nntp`` runner will now run the ``gatenews`` subcommand at intervals defined by a new ``gatenews_every`` setting in the ``[nntp]`` section of mailman.cfg. Default is 5 every minutes. Security -------- * Check the REST API password in a way that is resistant to timing attacks. (CVE-2021-34337, Closes #911) .. _news-3.3.4: 3.3.4 ===== (2021-03-21) Bugs ---- * Require aiosmtpd >= 1.4.1 to allow address local parts longer than 64 bytes in the lmtp runner. (Closes #836) * The email join command now honors the digest= option. (Closes #19) * Folded ``Message-ID`` headers no longer cause folded smtp.log messages. (Closes #844) * Removal of headers from posts to anonymous lists that can reveal the poster or the poster's domain has been improved. (Closes #848) Command line ------------ * The ``notify`` subcommand will now decode an RFC 2047 encoded Subject: for the report of held messages. (Closes #831) * The deprecated options ``--add``, ``--del`` and ``--sync`` are now disabled on the ``mailman members`` command. REST ---- * Improve the speed of Members lookup via REST API. (Related to #700) * Allow specifying ``delivery_mode`` and ``delivery_status`` when creating a new Member. (Closes #828) * Allow filtering members in ``/members/find`` API using ``delivery_status``, ``delivery_mode`` and ``moderation_action``. (See #827) New Features ------------ * There is a new setting ``filter_report`` in the ``[mailman]`` section of mailman.cfg. If this is set to ``yes``, a report of changes by content filtering will be added to the outgoing message. (Closes #833) * There is a new setting ``check_max_size_on_filtered_message`` in the ``[mailman]`` section of mailman.cfg. If this is set to ``yes`` and the list does content filtering, the ``max_message_size`` hold will be based on the size of the content filtered message. (Closes #377) * There is a new setting ``anonymous_list_keep_headers`` in the ``[mailman]`` section of mailman.cfg. This is part of improved removal of headers from posts to anonymous lists. This setting is a space separated list of regexp patterns. After anonomyzing removes the headers which are known to reveal the poster or poster's domain, it then removes all headers whose names do not match (case-insensitively) one of these patterns. The default setting keeps non X- headers, those X- headers added by Mailman and any X-Spam- headers. Other ----- * Add better error message for preferred email edit when not verified. (Closes #706) * Reduce the number of database calls in UserManager. (Closes #700) .. _news-3.3.3: 3.3.3 ===== (2021-02-02) Bugs ---- * Handle some UnicodeEncodeErrors in creating digests. (Closes #560) * Increased the size of the data column in the workflowstate table. (Closes #793) * Implemented a ``scrubber`` for plain text digests. (Closes #473) * The ``mailman gatenews`` command now adds ``original_size`` as a message attribute. (Extends fix for #762) * Handle FileNotFoundError when creating digest.mmdf file without a parent directory present. (Closes #699) * Fixed an issue where content filtering can throw UnicodeEncodeError when converting HTML to plain text. (Closes #798) * A bounce for a non-existent list is now handled. (Closes #799) * RFC 2047 From: headers in emailed ``join`` commands are now decoded. (Closes #802) * The ``mailman addmembers`` command now catches and reports a ``SubscriptionPendingError``. (Closes #805) * RFC 2369 ``List-Owner`` header is now added when these headers are included. (Closes #809) * Header filters will now properly match RFC 2047 encoded headers. (Closes #815) * Mailman's ``vette`` log discard messages now include the reasons. (Closes #816) * Increase the default REST API (gunicorn) timeout to 360 seconds from 30 as several API endpoinds can be very slow. (Closes #770) * Header filter rules are now properly processed after deletions and/or reordering. (Closes #818) * Folded ``To: list-confirm+token@...`` headers are now parsed correctly. (Closes: #819) Command line ------------ * A new ``findmember`` ``mailman`` subcommand has been added to find to which lists and with which roles an address matching a given pattern belongs. * A new ``changeaddress`` ``mailman`` subcommand has been added to enable site admins to change a user's address. New Features ------------ * There is a new setting ``hold_digest`` in the ``[mailman]`` section of mailman.cfg. If this is set to ``yes``, posts with a digest like Subject: header or which quote the digest masthead will be held for moderation. REST ---- * List configuration option ``send_goodbye_message`` is now exposed through the REST API. (See !737) * Allow updating an Addresses' display_name attribute. (Closes #786) * Allow specifying a reason when rejecting a subscription request. (Closes #767) * REST API now exposes unsubscription requests that can be handled by Moderator. (Closes #768) 3.3.2 ===== (2020-11-6) Bugs ---- * When importing 2.1 lists, ignore CR characters added by browsers in headers and footers. (Closes #701) * RFC 2369 headers are now added to notification messages. (Closes #710) * Bounce probes are now encoded in the correct charset. (Closes #712) * We now unfold address containing headers before parsing in avoid_duplicates. (Closes #725) * The ``dmarc`` rule no longer misses if DNS returns a name containing upper case. (Closes #726) * Fixed ``mailman.email.message.Message.as_string`` to not return unicode surrogates. (Closes #732) * Bounce probes can now be sent to a member subscribed as a User. (Closes #739) * A member subscribed as a User can now be unsubscribed. (Closes #734) * When a handler rejects or discards a message, it won't be delivered or archived anyway. (Closes #752) * Messages forwarded when content filtering removes the entire message and ``filter_action`` is forward now go to owners as well as moderators. (Closes #753) * Handle TOCTOU conditions when there are two simultaneous subscription requests for the same email address. (Closes #748) * Removed unnecessary call to ``Lock().disown()`` from ``bin/master.py``. (Closes #754) * Fixed an invalid logging call in bin/master.py. (Closes #756) * VERPed list welcome messages now have a correct envelope sender. (Closes #757) * Messages in digests now contain a ``Message: N`` header. (Closes #764) * The LMTP runner will now add a ``Message-ID:`` header if missing. (Closes #448 and #490) * The ``mailman gatenews`` command now adds ``original_size`` to the msgdata. (Closes #762) Command line ------------ * New ``addmembers``, ``delmembers`` and ``syncmembers`` ``mailman`` subcommands have been added. These provide more options and controls than the corresponding ``mailman members`` modes which are now deprecated. (Closes #686) * The ``mailman conf`` command without a ``-s/--section`` argument will now show sections defined only in mailman.cfg in addition to those from schema.cfg. (Closes #736) * Added a ``charset`` option to the ``import21`` subcommand. (Closes #769) * The ``import21`` subcommand will now truncate long SAUnicode values if the database is MySQL. (Closes #772) * The ``import21`` subcommand no longer adds the entire legacy ``*_these_nonmembers`` list and then removes the non-regexps. It now just adds the regexps. (Closes #773) REST ---- * Add a new endpoint ``/lists//requests/count`` which returns total number of pending requests. (Closes #713) * Subscription requests API now allows filtering requests with ``token_owner`` parameter. (Closes #714) * Add ``subscription_mode`` to ``Member`` resource so API clients can differentiate between subscription via address and via primary address of an user. (Closes #707) * Add ``/list//held/count`` to get a count of total held messages. (Closes #713) * Add ``fields`` as an optional parameter in all the Member's APIs to customize which fields are included in the resource. (Closes #716) * Expose ``emergency`` field of MailingList via REST API. (Closes #719) Localization ------------ * Italian translations of the templates have been added. * The default charset for many languages is changed to utf-8. (Closes #747) Others ------ * Add a new ``archive_rendering_mode`` attribute to ``MailingList`` to configure what kind of rendering should Archivers use to render Emails. (Closes #720) * An ``address=`` option has been added to the email ``join`` command to allow requesting subscription of other than the sender of the email. (Closes #721) * Added the ability to invite members to a list. (Closes #510 and #730) * Made (un)subscription confirmation email subjects user friendly and translatable. (Closes #541) * Implemented a new email ``who`` command to obtain list membership. * Add support for dnspython>=2.0. (Closes #743) * Added information about ``pass_types``, ``filter_extensions`` and ``pass_extensions`` to the content filtering doc. (Closes #775) 3.3.1 ===== (2020-04-19) Bugs ---- * Allow ``action`` for header matches to be None in REST interface. (Closes #671) * It is now possible to add the list posting address with nonmember role, e.g. to give it a moderaction of discard. (Closes #633) * The issue of posting a message without a To: header to a fully personalized list throwing a KeyError has been fixed. (Closes #639) * Confirmation or moderator approval of a subscription from an address banned subsequent to subscribing will no longer succeed. (Closes #642) * Fixed an issue caused by the fix for #642 where email confirmation of a banned subscription address would cause a shunted message. (Closes #643) * Fixed a bug where the ``postfix_vmap`` file would merge two columns for long domain names. (Closes #651) * Fix failing test cases due to changed API in a new version of dkimpy. (Closes #655) * Mailman now auto-geneartes alias maps when it starts for the first time. (Closes #469) * Sending a bounce probe now resets ``bounce_score``. (Closes #660) * Bounce probes which bounce are now properly recognized. (Closes #661) * Residual bounces are logged but are not processed further. (Closes #662) * Disabling delivery by bounce properly initializes some bounce info so warnings can be sent and the member eventually removed. (Closes #663) * Bouncing member is not removed until ``bounce_you_are_disabled_warnings_interval`` after the last warning was sent. (Closes #664) * Fixed recipient address in delivery disabled by bounce notice to the user. (Closes #665) * Residial bounces are now marked as processed. (Closes #669) * Find members API now searches for the Member's display name too. (Closes #667) * Content filtering ``convert_html_to_plaintext`` no longer creates messages that can't be flattened as_bytes. (Closes #677) * Removed an incorrect failing assertion when confirming a confirm_then_moderate unsubscription. (Closes #553) * Fix a bug due to incompatible API of a new version of ``importlib_resources`` library which causes Templates loading to fail. (Closes #691) * Fix a bug due to incompatible API of a new version of ``zope.intefaces``. (See !606) * Import21 no longer tries to import banned addresses. (Closes #688) Command line ------------ * A new ``mailman members --sync`` command has been added to help you synchronize subscribed mailing list members against a text list of subscription addresses. (see !545) REST ---- * Expose ``max_days_to_hold``, ``accept_these_nonmembers``, ``reject_these_nonmembers``, ``discard_these_nonmembers``, ``hold_these_nonmembers``, ``personalize``, and ``unsubscription_policy`` attributes of a MailingList through API. (See !570) * Add a new ``send_welcome_message`` flag when subscribing users to override MailingList's default setting. (Closes #362) * Expose content filtering settings for MailingList. (Closes #694) * Add ``advertised`` attribute to ``MailingList`` object so Postorius doesn't have to make multiple calls for Index Page. (See !608) * Expose ``filter_action`` attribute of MailingList through API. (See !609) * Unsubscribing a user by calling ``DELETE`` on a Member resources now honors Lists's ``unsubscription_policy`` and also sends out notifications to user and admins if list is configured to do so. (Closes #759) Features -------- * Add support for processing of email bounce events. Thanks to Aaryan Bhagat for working on this as a part of his GSoC project and Thanks to Google for sponsoring the project as a part of GSoC.(See !584) 3.3.0 -- "Tom Sawyer" ===================== (2019-09-04) Bugs ---- * A list member with ``receive_list_copy`` set to ``False`` who is addressed in ``CC`` will now be removed from ``CC`` even if this results in no ``CC``. (Closes #575) * ``X-Mailman-Approved-At`` header now has correct timezone. (Closes #584) * A spurious ``invalid FilterAction: discard. Treating as discard`` log message is eliminated. (Closes #592) * A post from a nonmember matching the legacy ``accept_these_nonmembers`` is now subject to subsequent rules rather than accepted immediately. (Closes #587) * Email attempts to subscribe a user who is banned or has a subscription already pending are now handled properly. (Closes #577 and #583) * It is no longer possible to add the list's posting address with any role to a list. (Closes #599) * Fixed the nntp runner which was calling the ``nntplib.NNTP.post()`` method with a string object instead of bytes. (Closes #613) * A post with a Reply-To: the list address will no longer be shunted. (Closes #614) * Encoded (base64 or quoted-printable) HTML message bodies are now decoded for ``html_to_plaintext``. (Closes #616) * Non-VERPed temporary failure DSNs are no longer reported as unrecognized bounces. (Closes #622) * Fix ``mailman import21`` command to work in Python 3.7.4 and Python 3.8b4 (Closes #625) Command line ------------ * The ``mailman import21`` command now leaves only regexps in the legacy ``*_these_nonmembers`` list attributes. (Closes #588) * The ``mailman import21`` command now imports nonmember accept actions as ``Action.defer`` rather than ``Action.accept``. (Closes #579) * The ``mailman import21`` command now correctly imports ``*_these_nonmembers`` actions for nonmembers following a member in the list. (Closes #580) * The progress meter while ``mailman import21`` is importing rosters has been shortened so it no longer wraps and scrolls. (Closes #589) * The ``mailman import21`` command no longer sends an email to existing owners for each imported member. (Closes #605) * A ``mailman notify`` command has been implemented to be run by cron to send periodic notices of held requests to list owners and moderators. (Closes #258) * The ``mailman import21`` command now imports ``private_roster``. (Closes #607) * A ``mailman gatenews`` command has been implemented to be run by cron to gate messages from usenet to those lists which have the gateway configured. * The ``mailman members --add`` command no longer prints a stack trace if an invalid email address is given. (See !544) * A new ``mailman members --remove`` command has been added to help you mass-unsubscribe subscribers from a mailing list. REST ---- * Expose a user's preferred address using REST API. (Closes #240) * Expose `header_matches/find` API to find a list of ``HeaderMatches`` belonging to a MailingList. (See !497) * Allow adding a comment when rejecting held messages. (Closes #594) * Fix a 500 error when creating a user with invalid email address. (Closes #263) Localization ------------ * Add French translations for mail templates. (See !522) Other ----- * Add a new attribute ``tag`` to ``HeaderMatch`` to find and manage a set of rules. (See !497) * Expired cached entries will be deleted efficiently. (Closes #462) * REST Runner now uses Gunicorn to run WSGI server instead of the standard library wsgiref for better performance. * Add support for SMPTS/STARTTLS for connections to MTA. (See !508) * The last remnants of the mailing list attribute ``nntp_host`` have been removed. (Closes #611) * Email confirmation requests now include an ``Auto-Submitted`` header to prevent robotic replies per ``RFC 3834``. (Closes #23) * The mail->news gateway no longer munges Message-ID headers unless the original Message-ID is rejected by the ``nntp_host``. (Closes #24) 3.2.2 ===== (2019-03-23) REST ---- * Failed validation to new Mailing List creation and Domain creation no longer returns 500 errors. * Expose ``preferred_language`` for MailingList through REST API. * Self-link of banned emails will be url-encoded when the email is actually a regex, mainly to prevent misplaced `?` in the link. * REST API now accepts inputs in JSON format. (See !451) * REST API errors are now always formatted as JSON. (See !451) * Failed request validations now return more verbose errors, like ``Enum`` types return all acceptable values. (See !451) * REST API accepts request with content type None (See !479) Command line ------------ * The ``mailman import21`` command now displays import progress. (Closes #561) * An issue with ``mailman subcommand --help`` hanging has been worked around. (Closes #520) 3.2.1 ===== (2019-02-22) Command line ------------ * The ``mailman import21`` command properly converts all acceptable_aliases to regexps. (Closes #496) * The ``mailman import21`` command correctly converts autorespond_* settings. (Closes #505) * The various ``mailman`` commands now issue a help hint in case of error. Tests have been updated to recognize this. (Closes #519) Bugs ---- * Open non-regular log files (e.g. fifos) in write mode instead of append mode. (See !407) * Autoresponses to posts and -owner and -request messages now work. (Closes #504) * Message parts are now properly decoded when trying to remove an Approved: header. (Closes #518) * Outgoing SMTP connections are now closed following message delivery regardless of the max_sessions_per_connection setting. (Closes #529) * Messages with Subject: Re: only are now processed correctly. (Closes #545) LOCALIZATION ------------ * Added German translations for mail templates. Thanks to the "Institut für Gebrauchsgrafik" for providing them. (Closes #534) REST ---- * Allow setting ``max_num_recipients`` for a mailing list. (Closes #508) * Add a per-list visibility option for Members. (Closes #108) Other ----- * Email commands are now case insensitive. (Closes #353) * Implicit commands sent to -join, -leave and -confirm addresses no longer generate a Results email. (Closes #516) * An ``explicit_header_only`` option has been added to Reply-To: munging to avoid putting the list address in Cc:. (Closes #531) * The default list welcome message has been cleaned up a bit. (Closes #530) * A substitution variable ``user_name_or_email`` has been added to ``list:member:regular:header`` and ``list:member:regular:footer`` templates when personalized deliveries are enabled. (Closes #533) * The template search algorithm was fixed to look for in-tree templates for each language in the search list. (Closes #535) * Set default subscription policy to "confirm then moderate" for private mailing list style. (Closes #536) 3.2.0 -- "La Villa Strangiato" ============================== (2018-07-10) Bugs ---- * A missing html_to_plain_text_command is now properly detected and logged. (Closes #345) * Syntactically invalid sender addresses are now ignored. (Closes #229) * An AttributeError: 'str' object has no attribute 'decode' exception in subject prefixing is fixed. (Closes #359) * Messages with no syntactically valid senders are now automatically discarded. (Closes #369) * Various message holds and rejects that gave 'N/A' as a reason now give an appropriate reason. (Closes #368) * Bounce messages are now composed for proper translations. * Messages with badly encoded ``Subject`` headers no longer crash the message moderation REST API. (Closes #383) * Generated ``regexp`` tables for Postfix now account for possible ``+extra`` additions to the ``-bounces`` and ``-confirm`` addresses. (Closes #401) * Address confirmation notices are now properly encoded in the character set of the list's preferred_language. (Closes #395 and #413) * Thanks to Jim Popovitch, certain failures in DNS lookups of DMARC policy will now result in mitigations being applied. (Closes #415) * Messages without a sender can no longer bypass the ``nonmember-moderation`` rule. (Closes #414) * Invalid regexps in header_matches rules are properly logged and can't be set via REST. (Closes #418) * A list whose name is one of the ``admin``, ``bounces``, ``confirm``, etc. subaddresses can now be posted to. (Closes #433) * The ``admin`` subaddress, a synonym for ``bounces`` and deprecated since Mailman 2.1, has been removed. (Closes #435) * Better support for changing the ``list_name`` property. (Closes #428) * Raw Message text is now coerced to pure ascii before sending, and https://bugs.python.org/issue27321 is now defended against by overriding Message.as_string() to cover more cases than before. (Closes #406) * The override of Message.as_string() has been extended to catch another observed exception. (Closes #441) * The email.message.UserNotification() constructor will now not fail if given a ``text`` argument containing a character not in the charset of the ``lang`` argument. (Closes #437) * A new SQLAlchemy column type ``SAUnicodeXL`` has been implemented to support large columns in MySQL and is used for the ``value`` column of the ``pendedkeyvalue`` table. (Closes #385) * Messages with ``Subject`` headers encoded in an unknown character set no longer throw ``LookupError`` in subject prefixing. (Closes #445) * A list's ``last_post_at`` is now properly updated. (Closes #453) * Fixed an AttributeError in mailman/handlers/decorate.py when member.subscriber is an ``IUser`` instance rather than an ``IAddress`` instance. (Closes #449) * Message decoration has been removed from the posting pipeline and added to bulk delivery to avoid duplicate decoration of personalized and VERPed messages. (Closes #450) * The ``mailman inject`` command now reads standard input as documented if there is no ``-f`` or ``--filename`` option. (Closes #459) * Mailman doesn't chmod existing configuration directories on startup (Closes #439) * Calling ``set`` on a template evicts the local cache if the template was cached before. (Closes #463) * The override of Message.as_string() has been extended to catch yet another observed exception. (Closes #470) * Fixed a typo in the help for ``mailman digests --periodic``. (Closes #472) * Character encoding errors in adding headers and footers to multipart messages are detected and ``replaced``. (Closes #409) * The help for ``mailman qfile`` has been fixed to reference ``mailman qfile`` rather than the non-existent ``bin/dumpdb``. (Closes #475) * The dmarc rule has been fixed to not throw a TypeError when msg.get('from') returns a header instance. (Closes #477) * ``mailman`` command without any sub-command now invokes ``mailman help`` by default. (Closes #479) * The ``alias_domain`` attribute can become a null string. ``mta/postfix.py`` has been fixed to generate proper mappings in this case. (Closes #485) * Fix a bug where duplicate address records can be created when a user adds an existing address with a different case. (Closes #476) * Several additional templates can now be set in the ``var/templates`` hierarchy. (Closes #486) Command line ------------ * Adopt the ``click`` package for command line parsing. This makes the command line more consistent and pluggable. Also, many CLIs that accepted a "fqdn list name" (i.e. the posting address of a mailing list), now also accept a ``List-ID``. Every attempt has been made to keep the CLI backward compatible, but there may be subtle differences. (Closes #346) * If no listname is given, running ``mailman withlist -r`` must name a function taking no arguments. This can be used to introspect Mailman outside of the context of a mailing list. * Fix ``mailman withlist`` command parsing. (Closes #319) * Add a new option ``--periodic`` to ``mailman digests`` command. It sends out digests for only those Mailing Lists that have ``digest_send_periodic`` option set to ``True``. (Closes #384) * The ``mailman import21`` command now imports DMARC mitigations. * The ``mailman import21`` command no longer creates templates with non-standard names, (Closes #482) Configuration ------------- * The ``[mailman]pre_hook`` and ``[mailman]post_hook`` variables are deprecated. They can still be specified but they will not be run. * The ``[paths.*]ext_dir`` variable has been removed. * A new logger has been added called ``logging.plugins``. * The ``[styles]paths`` variable has been removed; you can now specify additional styles using the new plugin architecture. * Mailman now also searches at ``/etc/mailman3/mailman.cfg`` for the configuration file. Database -------- * The fix for #313 which ported an upstream SQLAlchemy fix to Mailman has been refactored now that SQLAlchemy 1.2 has been released. Interfaces ---------- * Broaden the semantics for ``IListManager.get()``. This API now accepts both ``List-ID``s and fully qualified list names, since that's the most common use case. There's now a separate ``.get_by_fqdn()`` which only accepts the latter and mirrors the already existing ``.get_by_list_id()``. * A new template ``list:user:notice:rejected`` has been added for customizing the bounce message rejection notice. * The ``acceptable_alias`` and ``require_explicit_destination`` attributes have been added to the ``IMailingList`` definition. (Closes #483) Other ----- * Add a new plugin architecture, which allows third parties to add initialization hooks, REST endpoints, and additional components. Given by Jan Jancar. * Drop support for Python 3.4. (Closes #373) * Bump minimum requirements for aiosmtpd (>= 1.1) and flufl.lock (>= 3.1). * Add '.pc' (patch directory) to list of ignored patterns when building the documentation with Sphinx. * Domain objects now have an ``alias_domain`` attribute which is normally ``None``, but which can be set to an alternate domain. This will enable the Postfix MTA to generate ``virtual_alias_maps`` which will map list addresses in the domain to addresses in the alternate domain, and will modify the other generated mappings to use the ``alias_domain``. The ``lmtp`` runner will also accept the alias_domain as an alias for the ``email_host``. * ``ICacheManager`` now allows evicting a single file from cache. REST ---- * Allow a mailing list's acceptable aliases to be cleared by calling ``DELETE`` on the list's ``config/acceptable_aliases`` resource. (Closes #394) * Allow setting ``max_message_size`` for a mailing list. (Closes #417) * Added new attribute in ``lists/styles`` resource ``styles`` that contains the list of all the styles in Core along with their description. ``style_names`` attribute is now deprecated and will be removed in future versions. * Add ``display_name`` for ``member`` in order to expose ``member.display_name``. (Closes #398) * Expose ``MailingList.respond_to_post_request`` through REST API. (Closes #420) * Add a new API ``lists/find`` which returns all the lists related to a subscriber. It optionally allows filtering based on a role. (See !388) * ``IAcceptableAliasSet`` resource now interprets an empty string ('') as an empty list ([]). This can be used to clear the list of acceptable aliases of a MailingList in a PATCH or PUT request, without having to use a DELETE request. * Expose ``MailingList.require_explicit_destination`` through REST API. (Closes #484) 3.1.0 -- "Between The Wheels" ============================= (2017-05-25) Bugs ---- * When the mailing list's ``admin_notify_mchanges`` is True, the list owners now get the subscription notification. (Closes: #1) * Fix the traceback that occurred when trying to convert a ``text/html`` subpart to plaintext via the ``mimedel`` handler. Now, a configuration variable ``[mailman]html_to_plain_text_command`` in the ``mailman.cfg`` file defines the command to use. It defaults to ``lynx``. (Closes: #109) * Confirmation messages should not be ``Precedence: bulk``. (Closes #75) * Fix constraint violations on mailing list deletes affecting PostgreSQL. Given by Abhilash Raj. (Closes #115) * ``mailman`` command with no subcommand now prints the help text. Given by Abhilash Raj. (Closes #137) * The MHonArc archiver must set stdin=PIPE when calling the subprocess. Given by Walter Doekes. * For now, treat ``DeliveryMode.summary_digests`` the same as ``.mime_digests``. (Closes #141). Also, don't enqueue a particular digest if there are no recipients for that digest. * For Python versions earlier than 3.5, use a compatibility layer for a backported smtpd module which can accept non-UTF-8 data. (Closes #140) * Bulk emails are now decorated with headers and footers. Given by Aurélien Bompard. (Closes #145) * Core no longer depends on the standalone ``mock`` module. (Closes: #146) * The logging of moderation reasons has been fixed. Given by Aurélien Bompard. * Collapse multiple ``Re:`` in Subject headers. Given by Mark Sapiro. (Closes: #147) * Added Trove classifiers to setup.py. (Closes: #152) * Fix the processing of subscription confirmation messages when the mailing list is set to confirm-then-moderate. (Closes #114) * Fix ``UnicodeEncodeError`` in the hold chain when sending the authorization email to the mailing list moderators. (Closes: #144) * Fix traceback in approved handler when the moderator password is None. Given by Aurélien Bompard. * Fix IntegrityErrors raised under PostreSQL when deleting users and addresses. Given by Aurélien Bompard. * Allow mailing lists to have localhost names with a suffix matching the subcommand extensions. Given by Aurélien Bompard. (Closes: #168) * Don't traceback if a nonexistent message-id is deleted from the message store. Given by Aurélien Bompard, tweaked by Barry Warsaw. (Closes: #167) * Fix a bug in ``SubscriptionService.find_members()`` when searching for a subscribed address that is not linked to a user. Given by Aurélien Bompard. * Fix a REST server crash when trying to subscribe a user without a preferred address. (Closes #185) * Fix membership query when multiple users are subscribed to a mailing list. Reported by Darrell Kresge. (Closes: #190) * Prevent moderation of messages held for a different list. (Closes: #161) * When approving a subscription request via the REST API, for a user who is already a member, return an HTTP 409 Conflict code instead of the previous server traceback (and resulting HTTP 500 code). (Closes: #193) * In decoration URIs (e.g. ``IMailingList.header_uri`` and ``.footer_uri``) you should now use the mailing list's List-ID instead of the fqdn-listname. The latter is deprecated. (Closes #196) * Trying to subscribe an address as a list owner (or moderator or nonmember) which is already subscribed with that role produces a server error. Originally given by Anirudh Dahiya. (Closes #198) * Cross-posting messages held on both lists no longer fails. (Closes #176) * Don't let unknown charsets crash the "approved" rule. Given by Aurélien Bompard. (Closes #203) * Don't let crashes in IArchiver plugins break handlers or runners. (Closes #208) * Fix "None" as display name in welcome message. Given by Aditya Divekar. (Closes #194) * Fix ``mailman shell`` processing of ``$PYTHONSTARTUP``. (Closes #224) * Fix query bug for ``SubscriptionService.find_members()`` leading to the incorrect number of members being returned. Given by Aurélien Bompard. (Closes #227) * Fix header match rule suffix inflation. Given by Aurélien Bompard. (Closes #226) * MIME digests now put the individual message/rfc822 messages inside a multipart/digest subpart. (Closes #234) * Nonmember subscriptions are removed when one of the addresses controlled by a user is subscribed as a member. Given by Aditya Divekar. (Closes #237) * Email address validation is now more compliant with RFC 5321. (Closes #266) * A mailing list's ``description`` must not contain newlines. Given by Aurélien Bompard. (Closes: #273) * Allow MailingList.info to be set using the REST API. Given by Aurélien Bompard. * Extend header filters to also check sub-part headers. (Closes #280) * Allow REST API to PUT and PATCH domain attributes. Allows Postorius domain edit to work. (Closes: #290) * Prevent posting from banned addresses. Given by Aurélien Bompard. (Closes: #283) * Remove the digest mbox files after the digests are sent. Given by Aurélien Bompard. (Closes: #259) * Transmit the moderation reason and expose it in the REST API as the ``reason`` attribute. Given by Aurélien Bompard. * Don't return a 500 error from the REST API when trying to handle a held message with defective content. Given by Abhilash Raj. (Closes: #256) * Delete subscription requests when a mailing list is deleted. Given by Abhilash Raj. (Closes: #214) * Messages were shunted when non-ASCII characters appeared in a mailing list's description. Given by Mark Sapiro. (Closes: #215) * Fix confirmation of unsubscription requests. (Closes: #294) * Fix ``mailman stop`` not stopping some runners due to PEP 475 interaction. (Closes: #255) * Update documentation links for ``config.cfg`` settings. (Closes: #306) * Disallow problematic characters in listnames. (Closes: #311) * Forward port several content filtering fixes from the 2.1 branch. (Closes: #330, #331, #332 and #334) Configuration ------------- * Mailing lists can now have their own header matching rules, although site-defined rules still take precedence. Importing a Mailman 2.1 list with header matching rules defined will create them in Mailman 3, albeit with a few unsupported corner cases. Definition of new header matching rules is not yet exposed through the REST API. Given by Aurélien Bompard. * The default languages from Mailman 2.1 have been ported over. Given by Aurélien Bompard. * There is now a configuration setting to limit the characters that can be used in list names. Command line ------------ * ``mailman create `` will now create missing domains by default. The ``-d``/``--domain`` option is kept for backward compatibility, but now there is a ``-D``/``--no-domain`` option to prevent missing domains from being create, forcing an error in those cases. Given by Gurkirpal Singh. (Closes #39) * ``mailman`` subcommands now properly commit any outstanding transactions. (Closes #223) * ``mailman digests`` has grown ``--verbose`` and ``-dry-run`` options. * ``mailman shell`` now supports readline history if you set the ``[shell]history_file`` variable in mailman.cfg. Also, many useful names are pre-populated in the namespace of the shell. (Closes: #228) Database -------- * MySQL is now an officially supported database. Given by Abhilash Raj. * Fix a problem with tracebacks when a PostgreSQL database is power cycled while Mailman is still running. This ports an upstream SQLAlchemy fix to Mailman in lieu of a future SQLAlchemy 1.2 release. (Closes: #313) Interfaces ---------- * Implement reasons for why a message is being held for moderator approval. Given by Aurélien Bompard, tweaked by Barry Warsaw. * The default ``postauth.txt`` and ``postheld.txt`` templates now no longer include the inaccurate admindb and confirmation urls. * Messages now include a ``Message-ID-Hash`` as the replacement for ``X-Message-ID-Hash`` although the latter is still included for backward compatibility. Also be sure that all places which add the header use the same algorithm. (Closes #118) * ``IMessageStore.delete_message()`` no longer raises a ``LookupError`` when you attempt to delete a nonexistent message from the message store. * ``ISubscriptionService.find_members()`` accepts asterisks as wildcards in the ``subscriber`` argument string. Given by Aurélien Bompard. * ``ISubscriptionService`` now supports mass unsubscribes. Given by Harshit Bansal. Message handling ---------------- * New DMARC mitigations have been added. Given by Mark Sapiro. (Closes #247) * New placeholders have been added for message headers and footers. You can use a placeholder of the format ``$_url`` to insert the permalink to the message in the named archiver, for any archiver enabled for the mailing list. Given by Abhilash Raj. * The default posting chain has been modified so that the header-match chain and nonmember-moderation rule are processed before "hold" rules are processed. This allows for better anti-spam defenses and rejecting non-member posts instead of always holding them for moderator review. Given by Aurélien Bompard. (Closes #163) * Bounces can now contain rejection messages. Given by Aurélien Bompard. * The ``moderation_action`` for members and nonmember can now be ``None`` which signals falling back to the appropriate list default action, e.g. ``default_member_action`` and ``default_nonmember_action``. Given by Aurélien Bompard. (Closes #189) * Ensure that postings from alternative emails aren't held for moderator approval. For example, if a user is subscribed with one email but posts with a second email that they control, the message should be processed as a posting from a member. Given by Aditya Divekar. (Closes #222) * The default message footer has been improved to include a way to unsubscribe via the ``-leave`` address. Given by Francesco Ariis. REST ---- * REST API version 3.1 introduced. Mostly backward compatible with version 3.0 except that UUIDs are represented as hex strings instead of 128-bit integers, since the latter are not compatible with all versions of JavaScript. (Closes #121) * REST clients must minimally support HTTP/1.1. (Closes #288) * Experimental Gunicorn support. See ``contrib/gunicorn.py`` docstring for details. With assistance from Eric Searcy. (Closes #287) * The new template system is introduced for API 3.1. See ``src/mailman/rest/docs/templates.rst`` for details. (Closes #249) * When creating a user via REST using an address that already exists, but isn't linked, the address is linked to the new user. Given by Aurélien Bompard. * The REST API incorrectly parsed ``is_server_owner`` values when given explicitly in the POST that creates a user. (Closes #136) * A new top-level resource ``/owners`` can be used to get the list of server owners as ``IUser`` s. (Closes #135) * By POSTing to a user resource with an existing unlinked address, you can link the address to the user. Given by Abhilash Raj. * Fix pagination values ``start`` and ``total_size`` in the REST API. Given by Aurélien Bompard. (Closes: #154) * JSON representations for held message now include a ``self_link``. * When ``[devmode]enabled`` is set, the JSON output is sorted. Given by Aurélien Bompard. * A member's moderation action can be changed via the REST API. Given by Aurélien Bompard. * Fixed a number of corner cases for the return codes when PUTing or PATCHing list configuration variables. (Closes: #182) * Expose ``digest_send_periodic``, ``digest_volume_frequency``, and ``digests_enabled`` (renamed from ``digestable``) to the REST API. (Closes: #159) * Expose the "bump digest" and "send digest" functionality though the REST API via the ``/lists//digest`` end-point. GETting this resource returns the ``next_digest_number`` and ``volume`` as the same values accessible through the list's configuraiton resource. POSTing to the resource with either ``send=True``, ``bump=True``, or both invokes the given action. * Global and list-centric bans can now be managed through the REST API. Given by Aurélien Bompard. * ``/members/find`` accepts GET query parameters in addition to POST arguments. Given by Aurélien Bompard. * Header match rules for individual mailing lists are now exposed in the REST API. Given by Aurélien Bompard. (Closes: #192) * Expose ``goodbye_message_uri`` in the REST API. Given by Harshit Bansal. * New subscription requests are rejected if there is already one pending. With thanks to Anirudh Dahiya. (Closes #199) * Expose the system pipelines and chains via ``/system/pipelines`` and ``/system/chains`` respectively. Given by Simon Hanna. (Closes #66) * Support mass unsubscription of members via ``DELETE`` on the ``/lists//roster/member`` resource. Given by Harshit Bansal. (Closes #171) * It is now possible to merge users when creating them via REST. When you POST to ``/users/
/addresses`` and the address given in the ``email`` parameter already exists, instead of getting a 400 error, if you set ``absorb_existing=True`` in the POST data, the existing user will be merged into the newly created on. Given by Aurélien Bompard. * Port to Falcon 1.0 (Closes #20) * A member's ``moderation_action`` can be reset, allowing fallback to the list's ``default_member_action`` by setting the attribute to the empty string in the REST API. Given by Aurélien Bompard. * A list's ``moderator_password`` can be set via the REST API. Given by Andrew Breksa. (Closes #207) * The ban manager now returns a pageable, sorted sequence. Given by Amit and Aurélien Bompard. (Closes #284) * Query parameters now allow you to filter mailing lists by the ``advertised`` boolean parameter. Given by Aurélien Bompard. * Only the system-enabled archivers are returned in the REST API. Given by Aurélien Bompard. * **Backward incompatibility: mild** Held message resources now have an ``original_subject`` key which is the raw value of the ``Subject:`` header (i.e. without any RFC 2047 decoding). The ``subject`` key is RFC 2047 decoded. Given by Simon Hanna. (Closes #219) Other ----- * Add official support for Python 3.5 and 3.6. (Closes #295) * A handful of unused legacy exceptions have been removed. The redundant ``MailmanException`` has been removed; use ``MailmanError`` everywhere. * Drop the use of the ``lazr.smtptest`` library, which is based on the asynchat/asyncore-based smtpd.py stdlib module. Instead, use the asyncio-based `aiosmtpd `_ package. * Improvements in importing Mailman 2.1 lists, given by Aurélien Bompard. * The ``prototype`` archiver is not web accessible so it does not have a ``list_url`` or permalink. Given by Aurélien Bompard. * Large performance improvement in ``SubscriptionService.find_members()``. Given by Aurélien Bompard. * Rework the digest machinery, and add a new ``digests`` subcommand, which can be used from the command line or cron to immediately send out any partially collected digests, or bump the digest and volume numbers. * The mailing list "data directory" has been renamed. Instead of using the fqdn listname, the subdirectory inside ``[paths]list_data_dir`` now uses the List-ID. * The ``mailman members`` command can now be used to display members based on subscription roles. Also, the positional "list" argument can now accept list names or list-ids. * Unsubscriptions can now be confirmed and/or moderated. (Closes #213) 3.0.0 -- "Show Don't Tell" ========================== (2015-04-28) Architecture ------------ * Domains now have a list of owners, which are ``IUser`` objects, instead of the single ``contact_address`` they used to have. ``IUser`` objects now also have a ``is_server_owner`` flag (defaulting to False) to indicate whether they have superuser privileges. Give by Abhliash Raj, with fixes and refinements by Barry Warsaw. (LP: #1423756) * Mailing list subscription policy work flow has been completely rewritten. It now properly supports email verification and subscription confirmation by the user, and approval by the moderator using unique tokens. ``IMailingList`` objects now have a ``subscription_policy`` attribute. (LP: #1095552) * Port the REST machinery to Falcon 0.3. (LP: #1446881) Bugs ---- * Fix calculation of default configuration file to use when the ``$var_dir`` is created by ``mailman start``. (LP: #1411435) * When creating a user with an email address, do not create the user record if the email address already exists. Given by Andrew Stuart. (LP: #1418280) * When deleting a user via REST, make sure all linked addresses are deleted. Found by Andrew Stuart. (LP: #1419519) * When trying to subscribe an address to a mailing list through the REST API where a case-differing version of the address is already subscribed, return a 409 error instead of a 500 error. Found by Ankush Sharma. (LP: #1425359) * ``mailman lists --domain`` was not properly handling its arguments. Given by Manish Gill. (LP: #1166911) * When deleting a user object, make sure their preferences are also deleted. Given by Abhishek. (LP: #1418276) * Be sure a mailing list's acceptable aliases are deleted when the mailing list itself is deleted. (LP: #1432239) * The built-in example ``IArchiver`` implementations now explicitly return None. (LP: #1203359) * The test suite now runs successfully again with PostgreSQL. Given by Aurélien Bompard. (LP: #1435941) Configuration ------------- * When specifying a file system path in the [paths.*] section, $cfg_file can be used to expand into the path of the ``-C`` option if given. In the default ``[paths.dev]`` section, ``$var_dir`` is now specified relative to ``$cfg_file`` so that it won't accidentally be relative to the current working directory, if ``-C`` is given. * ``$cwd`` is now an additional substitution variable for the ``mailman.cfg`` file's ``[paths.*]`` sections. A new ``[paths.here]`` section is added, which puts the ``var_dir`` in ``$cwd``. It is made the default layout. Documentation ------------- * Improve the documentation describing how to run Alembic to add new schema migrations. Given by Abhilash Raj. REST ---- * **Backward incompatible change**: The JSON representation for pending mailing list subscription hold now no longer includes the ``password`` key. Also, the ``address`` key has been renamed ``email`` for consistent terminology and other usage. * You can now view the contents of, inject messages into, and delete messages from the various queue directories via the ``/queues`` resource. * You can now DELETE an address. If the address is linked to a user, the user is not delete, it is just unlinked. * A new API is provided to support non-production testing infrastructures, allowing a client to cull all orphaned UIDs via ``DELETE`` on ``/reserved/uids/orphans``. Note that *no guarantees* of API stability will ever be made for resources under ``reserved``. (LP: #1420083) * Domains can now optionally be created with owners; domain owners can be added after the fact; domain owners can be deleted. Also, users now have an ``is_server_owner`` flag as part of their representation, which defaults to False, and can be PUT and PATCH'd. Given by Abhilash Raj, with fixes and refinements by Barry Warsaw. (LP: #1423756) 3.0 beta 5 -- "Carve Away The Stone" ==================================== (2014-12-29) Bugs ---- * Fixed Unicode errors in the digest runner and when sending messages to the site owner as a fallback. Given by Aurélien Bompard. (LP: #1130957). * Fixed Unicode errors when a message being added to the digest has non-ascii characters in its payload, but no Content-Type header defining a charset. Given by Aurélien Bompard. (LP: #1170347) * Fixed messages without a `text/plain` part crashing the `Approved` rule. Given by Aurélien Bompard. (LP: #1158721) * Fixed getting non-ASCII filenames from RFC 2231 i18n'd messages. Given by Aurélien Bompard. (LP: #1060951) * Fixed `AttributeError` on MIME digest messages. Given by Aurélien Bompard. (LP: #1130696) Commands -------- * The `mailman conf` command no longer takes the `-t/--sort` option; the output is always sorted. Configuration ------------- * The ``[database]migrations_path`` setting is removed. Database -------- * The ORM layer, previously implemented with Storm, has been replaced by SQLAlchemy, thanks to the fantastic work by Abhilash Raj and Aurélien Bompard. Alembic is now used for all database schema migrations. * The new logger `mailman.database` logs any errors at the database layer. Development ----------- * Python 3.4 is now the minimum requirement. * You no longer have to create a virtual environment separately when running the test suite. Just use `tox`. * You no longer have to edit `src/mailman/testing/testing.cfg` to run the test suite against PostgreSQL. See `src/mailman/docs/START.rst` for details. Interfaces ---------- * The RFC 2369 headers added to outgoing messages are now added in sorted order. * Several changes to the internal API: - `IListManager.mailing_lists` is guaranteed to be sorted in List-ID order. - `IDomains.mailing_lists` is guaranteed to be sorted in List-ID order. - Iteration over domains via the `IDomainManager` is guaranteed to be sorted by `IDomain.mail_host` order. - `ITemporaryDatabase` interface and all implementations are removed. REST ---- * The Falcon Framework has replaced restish as the REST layer. This is an internal change only. * The JSON representation `http_etag` key uses an algorithm that is insensitive to Python's dictionary sort order. * The address resource now has an additional '/user' sub-resource which can be used to GET the address's linked user if there is one. This sub-resource also supports POST to link an unlinked address (with an optional 'auto_create' flag), and PUT to link the address to a different user. It also supports DELETE to unlink the address. (LP: #1312884) Given by Aurélien Bompard based on work by Nicolas Karageuzian. * The ``/3.0/system`` path is deprecated; use ``/3.0/system/versions`` to get the system version information. * You can access the system configuration via the resource path ``/3.0/system/configuration/
``. This returns a dictionary with the keys being the section's variables and the values being their value from ``mailman.cfg`` as verbatim strings. You can get a list of all section names via ``/3.0/system/configuration`` which returns a dictionary containing the ``http_etag`` and the section names as a sorted list under the ``sections`` key. The system configuration resource is read-only. * Member resource JSON now include the ``member_id`` as a separate key. 3.0 beta 4 -- "Time and Motion" =============================== (2014-04-22) Development ----------- * Mailman 3 no longer uses ``zc.buildout`` and tests are now run by the ``nose2`` test runner. See ``src/mailman/docs/START.rst`` for details on how to build Mailman and run the test suite. Also, use ``-P`` to select a test pattern and ``-E`` to enable stderr debugging in runners. * Use the ``enum34`` package instead of ``flufl.enum``. * Use ``setuptools`` instead of ``distribute``, since the latter is defunct. REST ---- * Add ``reply_to_address`` and ``first_strip_reply_to`` as writable attributes of a mailing list's configuration. (LP: #1157881) * Support pagination of some large collections (lists, users, members). [Florian Fuchs] (LP: #1156529) * Expose ``hide_address`` to the ``.../preferences`` REST API. [Sneha Priscilla.] (LP: #1203519) * Mailing lists can now individually enable or disable any archiver available site-wide. [Joanna Skrzeszewska] (LP: #1158040) * Addresses can be added to existing users, including display names, via the REST API. [Florian Fuchs] * Fixed a crash in the REST server when searching for nonmembers via ``/find`` which we've never seen before, because those members only have an address record, not a user record. This requires a small change in the API where the JSON response's ``address`` key now contains the URL to the address resource, the new ``email`` key contains the email address as a string, and the ``user`` key is optional. Commands -------- * `mailman conf` now has a `-t/--sort` flag which sorts the output by section and then key. [Karl-Aksel Puulmann and David Soto] (LP: 1162492) * Greatly improve the fidelity of the Mailman 2.1 list importer functionality (i.e. ``mailman import21``). [Aurélien Bompard]. Configuration ------------- * Add support for the Exim 4 MTA. [Stephen Turnbull] * When creating the initial file system layout in ``var``, e.g. via ``bin/mailman info``, add an ``var/etc/mailman.cfg`` file if one does not already exist. Also, when initializing the system, look for that file as the configuration file, just after ``./mailman.cfg`` and before ``~/.mailman.cfg``. (LP: #1157861) Database -------- * The `bounceevent` table now uses list-ids to cross-reference the mailing list, to match other tables. Similarly for the `IBounceEvent` interface. * Added a `listarchiver` table to support list-specific archivers. Bugs ---- * Non-queue runners should not create ``var/queue`` subdirectories. [Sandesh Kumar Agrawal] (LP: #1095422) * Creation of lists with upper case names should be coerced to lower case. (LP: #1117176) * Fix REST server crash on `mailman reopen` due to no interception of signals. (LP: #1184376) * Add `subject_prefix` to the `IMailingList` interface, and clarify the docstring for `display_name`. (LP: #1181498) * Fix importation from MM2.1 to MM3 of the archive policy. [Aurélien Bompard] (LP: #1227658) * Fix non-member moderation rule to prefer a member sender if both members and non-members are in the message's sender list. [Aurélien Bompard] (LP: #1291452) * Fix IntegrityError (against PostgreSQL) when deleting a list with content filters. [Aurélien Bompard] (LP: #1117174) * Fix test isolation bug in ``languages.rst``. [Piotr Kasprzyk] (LP: #1308769) 3.0 beta 3 -- "Here Again" ========================== (2012-12-31) Compatibility ------------- * Python 2.7 is now required. Python 2.6 is no longer officially supported. The code base is now also `python2.7 -3` clean, although there are still some warnings in 3rd party dependencies. (LP: #1073506) REST ---- * **API change**: The JSON representation for held messages no longer includes the `data` key. The values in this dictionary are flatted into the top-level JSON representation. The `key` key is remove since it's redundant. Use `message_id` for held messages, and `address` for held subscriptions/unsubscriptions. The following `_mod_*` keys are inserted without the `_mod_` prefix: - `_mod_subject` -> `subject` - `_mod_hold_date` -> `hold_date` - `_mod_reason` -> `reason` - `_mod_sender` -> `sender` - `_mod_message_id` -> `message_id` * List styles are supported through the REST API. Get the list of available styles (by name) via `.../lists/styles`. Create a list in a specific style by using POST data `style_name=