Object Storage¶

As introduced in File Storage, Mayan EDMS takes ownership of document backing files and stores them using the configured storage backend. Another supported backend is the s3boto3 backend. Provided by the Django Storages project, this backend allows Mayan EDMS to use any S3 (Simple Storage Service) compatible service as storage for documents, caches and tempoary files. This backend works not only with AWS, but any other S3-compatible API such as Minio (which can provide self-hosted S3 storage). Using S3 storage can help Mayan EDMS scale to millions of documents.

The following steps will configure Mayan EDMS to use a S3 style storage for documents on a Mayan EDMS instance

Warning

If migrating from a different storage backend, ensure Mayan EDMS is stopped first and that every file from the existing storage directory is copied to the new storage location before starting Mayan EDMS configured with with the new backend. Also, ensure you have made a backup before beginning this process.

Note

The below configuration only updates the primary document_storage location. You will need to add additional variables if you also want to update the location of the Caches, Temporary Location, Shared Folders, Etc. Anything that is currently configured with the django.core.files.storage.FileSystemStorage backend can have its backend changed, but this is not advisable for caches and tempoary files locations due to performance reasons.

Docker Deployments¶

Add the following environment variables to your docker-compose.yml file in the environment section:

MAYAN_PIP_INSTALLS: 'django-storages boto3'
MAYAN_DOCUMENTS_STORAGE_BACKEND: storages.backends.s3boto3.S3Boto3Storage
MAYAN_DOCUMENTS_STORAGE_BACKEND_ARGUMENTS: "{'access_key':'yourS3accesskey','secret_key':'yourS3secretkey','bucket_name':'yourS3bucketname','default_acl':'private'}"

If deploying Mayan EDMS without Docker Compose, add the environment variable to the complete docker run command:

-e MAYAN_PIP_INSTALLS='django-storages boto3' \
-e MAYAN_DOCUMENTS_STORAGE_BACKEND=storages.backends.s3boto3.S3Boto3Storage \
-e MAYAN_DOCUMENTS_STORAGE_BACKEND_ARGUMENTS="{'access_key':'yourS3accesskey','secret_key':'yourS3secretkey','bucket_name':'yourS3bucketname','default_acl':'private'}"

Once added and the values changed to match the desired bucket, run:

docker-compose restart

For non Compose docker deployments, remove and re-deploy the container if not using docker-compose.

Note

The default_acl value is needed to prevent the backend from writing to S3 with a public-readable ACL. This is blocked by default on some providers such as AWS resulting in permission denied errors when trying to upload a document

Note

The default_acl option has been removed from django-storages in version 1.10 (2020-08-30). Instead use object_parameters with the options mentioned in https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_object and update your MAYAN_DOCUMENTS_STORAGE_BACKEND_ARGUMENTS setting accordingly.

Direct Deployments¶

For environments installed using the direct deployment method:

Install the required dependencies:

sudo -u mayan /opt/mayan-edms/bin/pip install django-storages boto3

Then, edit the supervisor file at /etc/supervisor/conf.d/mayan.conf and add these two extra settings above the MAYAN_DATABASES option, ensuring that the lines remain indented as before.

An example database option is shown for guidance; do not replace the existing MAYAN_DATABASES line, add the new variables above it.

MAYAN_DOCUMENTS_STORAGE_BACKEND=storages.backends.s3boto3.S3Boto3Storage,
MAYAN_DOCUMENTS_STORAGE_BACKEND_ARGUMENTS="{'access_key':'yourS3accesskey','secret_key':'yourS3secretkey','bucket_name':'yourS3bucketname','default_acl':'private'}",
MAYAN_DATABASES="{default: {ENGINE: django.db.backends.postgresql, HOST: 127.0.0.1, NAME: mayan, PASSWORD: mayanuserpass, USER: mayan}}"

To complete the change, restart Supervisor on all systems running Mayan EDMS:

sudo systemctl restart supervisor

Non-AWS S3 Deployments¶

If you are using Minio or another non-AWS based S3-compatible storage solution (such as Ceph with a S3 Gateway), there will need to be additional variables entries needed for the MAYAN_DOCUMENTS_STORAGE_BACKEND_ARGUMENTS line.

The default_acl configuration may also need to be ommitted if the platform does not support AWS-style ACLs, but try first with the default_acl value set for security:

MAYAN_DOCUMENTS_STORAGE_BACKEND_ARGUMENTS="{'access_key':'yourS3accesskey','secret_key':'yourS3secretkey','bucket_name':'yourS3bucketname','default_acl':'private','endpoint_url':'https://address-of-s3-server:9000','verify':'False'}"
Note

The verify option validates for trusted TLS certificates. Change this to True if your S3-compatible server is using trusted signed certificates. Leave it at false for self-signed certificates, and don’t forget to change the https to http for non-TLS setups

You may also need to add the variables region_name and/or location depending on your S3 server configuration.

endpoint_url: “https://<my internal NAS FQDN>:9000”, verify: False