[Dune-devel] Listerver archives cleaned up

[Jö Fahlke]

TLDR: Listserver archives have been cleaned up to make them regeneretable with
current mailman.  Should be mostly, but not completely seamless in regard to
message/attachment URLs.

Hi,

The listserver archives had been rotting for a long time.  Mailman keeps the
list mails in an mbox-format file, and generates the .html archives and the
.txt.gz archives that are accessible from the web interface from that.  Those
archives are generated incrementally, meaning whenever a mail comes in, there
can be differences in handling depending on the mailman version.

As a result it had become impossible to regenerated the archives as intended
from the mbox files, and keep mail URLs stable across such regeneration.
Likewise, the URLs of scrubbed attachments have often changed.  Finally, there
were some attempts a censoring spam by removing it from the mbox and editing
the html files, but without regenerating the archives completely, this also
leads to differences in messge numbering and thus changed URLs.

In this cleanup, I fixed the message numbering issues by inserting dummy
message at apropriate places in the mbox.  The differences in attachment
naming I mostly solved with HTTP redirects.

So this should mostly be seamless.

Though there were a few places where it was impossible to use redirects for
attachments due to clashing old and new names.  Then there were a few
instances where current mailman takes different decisions on whether to
include certain message parts in the html archive, so mail might look
differently.

This was done for all mailinglists on the server.

Details are at [2], for those that have access.

If you notice anything off, let me know -- preferrably via the phone number
listed at [1] (if you have access there), but a direct email to
jorrit at jorrit.de should also work, though I'm a bit slow to read them these
days.

[1]: https://gitlab.dune-project.org/infrastructure/dune-lists-doc
[2]: https://gitlab.dune-project.org/infrastructure/dune-lists-doc/-/merge_requests/50

Regards,
Jö.

-- 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.dune-project.org/pipermail/dune-devel/attachments/20211016/41219de0/attachment-0001.sig>