Remove files from MongoDB after the associated document is deleted

[vegidio]

Disclaimer: this is a feature request, but I'm willing to contribute by coding it myself since I already coded a similar solution in many of my projects that already use the Parse Server.

---

Is your feature request related to a problem? Please describe.
I've been using Parse for quite some time in different projects, but when I create a document with File fields, this becomes a problem when the document or when the field itself is removed/replaced using one of the Parse SDKs (JS, Android, etc) because the reference to the file is removed, but the actual file remains there. After some time the database will be full of orphan files that only occupy space.

Describe the solution you'd like
To solve this problem I created a couple of triggers in my Parse projects (Parse.Cloud.afterSave, Parse.Cloud.afterDelete, etc) to properly remove the files whenever the document that they are associated with is deleted or when the field is remove/updated.

The way I implemented this was by creating an environment variable called DELETE_ORPHANS_FROM where I put the list of classes that should have the files removed when a document is deleted/updated. Like this: DELETE_ORPHANS_FROM=Post,Comment. So in my cloud code I read the content of this environment variable and send a Parse.Cloud.httpRequest to delete any orphan file in the classes that I listed there.

The orphan files in the classes that are not listed in this variable are of course untouched. This gives flexibility to the users if for some reason they want to keep the orphan files in certain classes.

Additional context
Before I start working on this I would like to hear from the maintainers and other members of the community what they think of this new feature. Maybe there is a reason why it was not implemented yet. Or maybe the solution that I'm proposing is not optimal, despite the fact that it works in my projects

Please let me know your thoughts and depending on the feedback I will start working on this.

[mtrezza]

I think your suggested feature would be a useful addition to Parse Server.

There have been many questions posted about the fact that files are not deleted from the file storage when the referencing Parse Object is deleted. I have come across similar solutions as you suggest in your implementation that works as expected. Adding this as a feature would certainly be welcomed by the community.

I think the 3 most commonly expected behaviors would be:

• a) Don't delete a file when the referencing object is deleted (what we have currently)
• b) Delete a file when the referencing object is deleted (for 1-1 relations, easier to implement)
• c) Delete a file when all referencing objects are deleted (for 1-n relations, harder to implement)

Actually c) covers b) but is more difficult to implement. If you want to to start with b), it would make sense to explicitly define the classes (ideally in regex) in the Parse Server configuration, so that projects that use a 1-1 relations in some classes and a 1-n relations in others can use the feature at least for some of their classes.

If you want to shoot straight for c) you would most likely implement a file reference counter so that Parse Server knows when a file is not referenced anymore and deletes it.

Whichever solution you go for, we'd be happy to review your PR.

[vegidio]

Thanks for the input, I appreciate it.

Indeed option c) seems to be best one to implement, but let me dive into the code first and see what I find there. I will probably start working on this next week.

[mtrezza]

Great, let me know whenever you need any guidance.

[Moumouls]

Really interesting addition to clean up GridFS or some linked object storages !

[mtrezza]

Really interesting addition to clean up GridFS or some linked object storages !

A "clean-up", as I understand it, would also delete unreferenced files retroactively from the file storage. That means scanning through the files in the storage and delete the unreferenced ones. That's a (one time) script with its own complexities therefore I am not sure this should be mixed into this PR, unless someone comes up with an ingenious concept. Such a script existed for Parse.com, I wonder if it's still around somewhere or anyone brought it over after the shutdown.

[mtrezza]

@vegidio Are you still working on this? If you need any guidance, please feel free to reach out.