Version 4.6

Released: January 11, 2024

Changes

Appearance

The user interface templates were converted to a widget component system.

The UI component work is meant to allow restyling and even remaking the user interface much easier.

The blocks content and content_plain were unified.

Templates were moved to sub-folders and their names shorten based on the folder location commonality.

Cabinets

Top-level cabinets now display the amount of documents contained. Because the widget for top-level cabinets is different from the one used for child cabinets, the document column is only shown for top-level cabinets.

Docker

The Docker Compose index mirroring profile was improved. The variable MAYAN_MIRROR_INDEX_NAME was added to make it much easier to mount different indexes without modifying the Docker Compose file.

By just setting the variable MAYAN_MIRROR_INDEX_NAME to the internal name of the index and enabling the index_mirror profile, the Docker Compose file takes care of all configuration, even the Docker volume mounting to the parent OS.

The Docker Compose file was also converted into a platform template. This means that the final Docker Compose file that is published is generated by Mayan EDMS itself, reducing the probability of errors, especially when updating the recommended default versions of the different container images.

The base Debian image was updated from 12.1-slim to 12.2-slim.

Documents

The event “document version edited” is now committed when a document version pages are remapped. This allow triggering workflows when a document version is finally ready after upload.

Dependencies

With Django 3.2 LTS approaching end-of-life, Mayan EDMS was updated to use Django 4.2 LTS. Django 4.2 LTS will reach end-of-life on Apr 2026 providing Mayan EDMS with 3 years of security updates.

With the updated Django, it was also possible to use the new PostgreSQL Python client library 3.x version. The library psycopg2 was updated from version 2.x to version 3.x.

Updated the dependency versions:

  • jQuery from 3.6.0 to 3.7.1.

  • Update use of psycopg2 version to 2.x to psycopg version 3.1.14.

  • AMQP from 5.1.0 to 5.2.0.

  • PIP from 23.2.1 to 23.3.1.

  • django-test-migrations from 1.1.0 to 1.3.0.

  • redis from 4.6.0 to 5.0.1.

  • wheel from 0.41.0 to 0.42.0.

  • bleach from 6.0.0 to 6.1.0.

  • django-auth-ldap from 4.4.0 to 4.6.0.

  • mozilla-django-oidc from 2.0.0 to 3.0.0.

  • pyotp from 2.6.0 to 2.9.0.

  • django-solo from 2.0.0 to 2.1.0.

  • django from 3.2 to 4.2.8.

  • django-mptt from 0.14.0 to 0.15.0.

  • sh from 2.0.4 to 2.0.6.

  • django-debug-toolbar from 3.2.4 to 4.2.0.

  • django-rosetta from 0.9.9 to 0.10.0.

  • django-silk from 5.0.3 to 5.0.3.

  • ipython from 8.14.0 to 8.18.1.

  • Pillow from 10.0.0 to 10.1.0.

  • pypdf from 3.14.0 to 3.17.1.

  • qrcode from 7.3.1 to 7.4.2.

  • node-semver from 0.8.1 to 0.9.0.

  • python_gnupg from 0.4.8 to 0.4.9.

  • graphviz from 0.17 to 0.20.1.

  • dateparser from 1.1.8 to 1.2.0.

  • pytz from 2022 to 2023.3.post1.

  • gevent from 22.10.2 to 23.9.1.

  • greenlet from 2.0.2 to 3.0.1.

  • sentry-sdk from 1.29.0 to 1.38.0.

  • whitenoise from 6.5.0 to 6.6.0.

  • django-cors-headers from 4.2.0 to 4.3.1.

  • jsonschema from 4.18.0 to 4.20.0.

  • CairoSVG from 2.5.2 to 2.7.1.

  • boto3 from 1.28.16 to 1.33.7.

  • django-storages from 1.13.2 to 1.14.2.

  • extract-msg from 0.37.1 to 0.46.2.

  • pycryptodome from 3.18.0 to 3.19.0.

  • celery from 5.3.5 to 5.3.6.

  • coverage from 5.5 to 6.5.0.

  • coveralls from 3.2.0 to 3.3.1.

  • psutil from 5.8.0 to 5.9.6.

  • django-widget-tweaks from 1.4.12 to 1.5.0.

The program ruff was added to the development dependencies to perform code style checking along with flake8.

The support Django series version was added to the setup generation script to ensure builds are always matched to their supported Django version.

The minimum and recommended requirements were updated. This change reflects the primary focus of Mayan EDMS as an advanced enterprise document management system.

The new minimum requirements are:

  • RAM: 4 GB.

  • CPU: dual core, 64 bit, 1 GHz, Intel Core 2 Duo or similar.

  • Database manager: PostgreSQL 13.11.

  • Platform: GNU/Linux distributions or operating systems that support mature virtualization and mature Docker with Docker Compose support.

The new recommended requirements are:

  • RAM: 16 GB or more.

  • CPU: 64 bit, 8 cores or more, 2 GHz or faster, third generation “IvyBridge” of the Intel Core processors or better.

  • Storage: solid state drive.

File metadata

The original purpose of this app was to do basic introspection of the document. With the addition of multi-file documents, the file metadata app now serves a major role when gathering information about the files that compose a single document.

To accomplish this, the file metadata app was refactored to bring the code up to the standards of the project. Many shortcomings were fixed and new functionality was introduced.

It is now possible to execute multiple inspection drivers for the same MIME type. A single document file will be inspected by all drivers that understand the type of file that composes the document.

File metadata drivers are now automatically detected and imported. This is important for developers and third-party apps. It is no longer necessary to import the driver and call the register method.

The file metadata attribute names are now normalized and saved as a new field called “Internal name”. This avoids issues when the file metadata attribute names have spaces in them or special characters. When accessing the file metadata attribute in a template the internal name must be used.

A tool view was added showing all detected file metadata drivers installed.

Previously drivers were processed in series. Drivers for document files are now processed in parallel. This shortens the time when the results are available at the expense of increased system load.

Antivirus scanning was added as a file metadata driver for ClamAV. No action is taken if an infection is detected. Instead, all attributes of the scan are saved and made available to the template system. Users can then choose on the course of action via a workflow depending on their policies. Some possible ways to manage infected files via workflow are:

  • Tag infected documents

  • Place infected documents in a cabinet or an index

  • Delete infected document files or the entire document

  • Notify administrators

The latest ClamAV database is included in the Docker image and is automatically updated on each release by the build pipeline.

Since antivirus scanning is a CPU-heavy operation, the file metadata queue was moved to the D worker.

License

Mayan EDMS is once again licensed under the GPL, but this time under version 2.0.

Lock manager

Unique test lock names are now generated when testing the lock manager backend. This avoids unintended lock errors when stopping and starting the stack in a short time.

Mailer

The mailer app was refactored to bring the code quality up to standards and to add new functionality.

The API for creating, editing, and deleting mailing profiles was added.

Another API was added to send object links via email or the object itself as an attachment.

Instead of making this API a sub URL of the document, document file, or document version, the decision was made to make it object agnostic. This means that only two endpoints needed to be created to allow for the email functionality.

The mailing app can now track which types of objects have been registered for emailing. For example, attempting to email a tag will raise the corresponding error informing that tags are not registered for emailing.

Along this line, the mailing app was updated with the capacity to email any type of object, not just document-related objects. As Mayan EDMS is also used as a development platform for other projects, like contract management, these custom implementations can now email their implementation-specific objects using the same app and API.

Each object type that supports emailing just needs to provide two functions:

  • How to extract the MIME type

  • How to extract the content to be emailed

Naming conventions in the mailer were normalized. This replaces uses of mailer and user mailer with mailing profile.

Settings

The smart settings app was refactored to improve existing functions but also to add new functionality.

Setting value changes no longer take effect immediately or trigger saving the configuration file. Instead a banner will show that the settings have been changed and need to be saved explicitly via a new action option.

A new column was added to show which settings have been updated. A new link was added to allow reverting any pending updated setting values.

Setting post edit functions are now execute during startup and not after editing the setting. This behavior is more inline with how settings changes are expected to work.

Django 4.2 changes how HTTPS operates from behind a reverse proxy. To support this, the following Django settings were exposed by Mayan EDMS’s settings system:

  • CSRF_TRUSTED_ORIGINS

  • CSRF_COOKIE_SECURE

  • CSRF_USE_SESSIONS

Task manager

The worker OS nice levels were updated from literals to config constants. This allows re-configuring the worker OS priorities at build time much easier.

Other

  • Override the test runner’s logging setup to avoid having its output being concatenated when calling management commands and makefile targets.

  • Support source column resolution for non model subclasses.

  • Normalize the permission system to work with single permissions per filter or check.

  • Lower the severity of searching indexing problems to INFO. This reduces user confusion between normal messages when processing the asynchronous task queue and actual coding errors.

  • Optimize document type retention policy queries.

  • Minor optimization to the ACL calculation queries.

  • Optimize the file cache eviction selection.

  • Check if database tables are available before preloading and caching permissions and file metadata drivers.

  • Use the correct post_load_modules method to execute initialization code after app module loading.

  • Reorganize make files. Remove unused and outdated targets. Move all Docker related targets to the Docker make file. Improve staging targets.

  • Fix double separator in the user menu.

  • Theme updates.

Removals

  • None

Backward incompatible changes

  • Existing templates that access the file metadata attributes need to be updated to use the new normalized version of the attribute.

Deprecations

  • The Keycloak Docker Compose service will be removed from the Mayan EDMS Docker Compose file into a separate Docker Compose file.

Issues closed