Proper way to clean media_store_path?

[sunsingerus]

I have the following setup:

    ## Media Store ##
    #enable_media_repo: false
    media_store_path: "/data/media_store"
    media_storage_providers:
    - module: s3_storage_provider.S3StorageProviderBackend
      store_local: True
      store_remote: True
      store_synchronous: True
      config:
        bucket: synapsebucket
        endpoint_url: http://synapse-minio-hl-svc:9000
        access_key_id: accesskey
        secret_access_key: secretkey
    uploads_path: "/data/uploads"

With this setup, files are kept both in /data/media_store and in synapsebucket bucket now.

As far as I understand, synapse upload files into /data/media_store and synapse-s3-storage-provider copies these files to synapsebucket bucket.
However, files in /data/media_store are not required, after being copied to bucket, but are still in there.

How should /data/media_store be cleaned properly?
Should I setup a cronjob and just rm old files in /data/media_store?
Or may be there are more proper ways to do this?

I have some vague idea that s3_media_upload is somehow related to this, but do not grasp the whole picture. I do not use s3_media_upload at the moment and files are successfully copied into bucket by synapse-s3-storage-provider already.

The main (and only problem at the moment) is unused files kept in /data/media_store.

Please, advise! Thanks in advance!

[djschilling]

An answer to this would be great.

A few further questions:

Can someone explain the three options store_local , store_remote , store_synchronous .
I would expect store_local to behave that the files are not saved anymore to disk but only to s3. But thats not whats happening.

Is there a way to say that i just want to use S§ and not the local file system for media?

@sunsingerus maybe you figured these things out?

[HarHarLinks]

I think store_local, store_remote, and store_synchronous are actually used by synapse to decide on whether to dispatch storing to either local or remote storage provider. See the "documentation" in the example config:
https://github.com/matrix-org/synapse/blob/883ac4b1bb7c520e928e8a42d7700de7f0d56055/docs/sample_config.yaml#L700

[puyoxyz]

did you ever figure it out?

[sorcer1122]

I played with this and simply emptying local media storage with rm -r /var/lib/matrix-synapse/media does the trick. It would be interesting to know how to disable local media storage, so that all files would go straight to S3 bucket.

[vfosnar]

Going through the source code of Synapse it looks like it can't be disabled in config. MediaStorage.store_file calls self.write_to_file which always writes to the main media repository. [1]

If I understand it correctly this can be fixed by refactoring the code to only use storage providers for file storage and make the main media repository an optional storage provider.

[col-panic]

An answer to this would be great.

A few further questions:

Can someone explain the three options store_local , store_remote , store_synchronous . I would expect store_local to behave that the files are not saved anymore to disk but only to s3. But thats not whats happening.

Is there a way to say that i just want to use S§ and not the local file system for media?

@sunsingerus maybe you figured these things out?

See https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#media_storage_providers for the respective documentation