WIP: Add Multithreading Support

[volfco]

Add Multithreading Support to fuser.

This implementation modifies BackgroundSession to accept a thread quantity. If this number is two or more, then n - 1 background workers will be created. There will always be a primary worker created, hence n - 1.

Session::worker is called to clone the Session, which in turn calls Channel::worker. This method accepts the Mount object, and using .session_fd to return the Session FD, fuse_fd_clone is invoked to clone the file descriptor and run the needed ioctl call to setup the session.

I can't take all the credit, as the main piece of code in channel.rs is from the Datenlord project where they are doing a native async_fuse library.

The code in question is from here: https://github.com/datenlord/datenlord/blob/d90fd43732373451207e56e9b9cd7eef9e7b53e1/src/async_fuse/fuse/session.rs#L73

Additional References:

• https://john-millikin.com/the-fuse-protocol#multi-threading
• https://fuse-devel.narkive.com/CodQceXu/how-could-fuse-support-parallel-opreation

TODO

• Clean up feature gate
• Add more function documentation
• Add changelog entry
• Bump version

[colinmarc]

Sorry for the random drive-by comment. In my opinion, it would be better to require Send + Sync instead of Clone, and change the trait methods to take non-mutable self. Then it's up to the user how they handle mutability, if they need it. Cloning + mutability is a massive footgun.

[volfco]

Thanks for the feedback!

I need to figure out exactly what you're referring to and understand it, but that does make sense and does seem better. Something fun to do this weekend!

[theunkn0wn1]

if a type is Send, it can be safely moved between threads.

if a type is Sync, it can be safely shared between threads.
see https://doc.rust-lang.org/nomicon/send-and-sync.html?highlight=Send#send-and-sync

[colinmarc]

For example, Arc<AtomicU64> is a totally reasonable way to handle mutability, here - but this example could have easily used a normal u64 before these changes and still derived Clone, leading to coinciding file handles.

[rarensu]

Do mind explaining why you picked the multithreading programming model rather than async/await programming model?

[volfco]

Do mind explaining why you picked the multithreading programming model rather than async/await programming model?

Because that would be a major refactor of the code base, and to do it properly everything would need to be rewritten

[rarensu]

@volfco I created a similar multithreading solution but instead of making a whole new session object for each worker, I just created a new channel object for each worker, and one session holds an array of channels. The entire session goes into an arc, rather than each of its mutable fields. Then the run function becomes an associated function that takes Arc rather than a traditional method that takes Self (Of course, just putting the Filesystem in an Arc would have accomplished the same thing. So maybe I over-engineered it? Oh well). This worked for me because at the same time, I was also changing the method signatures of Filesystem to take &self rather than &mut self, which is very nearly a requirement of async. This combination puts less pressure on Filesystem to behave a certain way, because there is no longer any need to clone Filesystem. It just needs to be Send + Sync. If you're still working on this, we can compare notes. For example, my strategy has far fewer unsafe blocks than yours (really, just the fuse worker ioctl function). I'm not really sure why yours has so many.

[volfco]

@rarensu I'm not currently working in this space and on this project. At this point, I'm waiting for more robust Rust support for the kernel and then I'll reimplement my idea as a kernel module.

Even with multi-threading, the overhead was still quite high. Maybe I'll revisit this eventually