Performance issues when combining useAbortAll() with a large (thousands of items) queue

[mwdiaz]

First, thanks for this awesome library! It's allowed us (SmugMug) to fairly rapidly replace our older upload code with a much simpler and easy to maintain implementation.

We currently have a custom hook that wraps a variety of the Uploady hooks in order to consistently manage and consume BatchItem states. The object returned by this hook is then passed down as a prop for other components/hooks to leverage. This is pretty clean and generally works very well but we're running into a performance issue when aborting a large queue (hundreds to thousands of items); for example when closing an upload modal.

More specifically, a useItemAbortListener() within this custom hook does some work that modifies the output of the hook itself which ultimately cascades into a variety of updates and re-renders all the way down the component tree. The problem manifests when combining useAbortAll() with a very large queue since Uploady loops over items and aborts/finalizes them individually when aborting them all, leading to useItemAbortListener() being called over and over again, which then causes a ton of otherwise unnecessary work and re-renders that bog the browser down, crashing it completely in the worst case.

Ultimately, we have room for improvement in our hook and preventing unnecessary re-renders, but I'm curious if there are any general recommendations/suggestions around how we might deal with the item abort listeners being called in a tight loop for the abortAll case when we may actually only care about handling the ALL_ABORT event.

As an arbitrary example, even assuming improvements in our render behavior, we may not want to trigger re-renders or side-effects (such as analytics) for each individual item that is aborted via abortAll. We do need to re-render and fire side-effects when items are aborted individually. There isn’t currently any way to differentiate (in useItemAbortListener(), at least) between the two types of aborts.

Thanks again!

[yoavniran]

Hey @mwdiaz thanks!

This is indeed an interesting topic and something Id need to ponder on for a bit.
The abort-all mechanism is indeed naive in its implementation and will work well enough for most cases but in such volumes as you describe can definitely become an issue.

I will put some thoughts into writing soon and share it so we can discuss whether it answers your needs without breaking other implementations.

[mwdiaz]

Thanks for the response!

I look forward to hearing your thoughts and discussing possible options for handling this natively in Uploady, since so far the workarounds I've tried have led to other issues or haven't proven viable for one reason or another. In the meantime, we can focus on making sure we improve our rendering since that's a worthwhile endeavor regardless 😄

[yoavniran]

definitely. :)
BTW, Is there a way for me to try your new uploader? Or is it still in dev and private?

[AJLemos]

@yoavniran Yeah, we can enable it for you. Create a trial SmugMug account and email me your NickName or email address used for the account at aj@smugmug.com.