Use of np.isin to accelerate dust

[manellopez13]

Hi there,

For some reason, I try to use dust on my array and it just does nothing. I've tried using runs and erase, just as in the definition of dust, but I can't make it work.
So I decided to try the slow way and use the loop in line 1140

But I think the same output can be obtained considerably faster with the following variation:
img *= ~np.isin(cc_labels, to_mask)
so it avoids the loop.

I wonder if this option is faster than the runs and erase alternative, but I cannot test it properly.

[william-silversmith]

Hi!

Would you be willing to share a data sample? It's pretty weird that dust wasn't working on it.

Will

[manellopez13]

Sorry about the late reply, I've found some weird behaviour. For some reason, when I reshape my data to 2D and reshape it back to 3D, the function dust works fine.

I've tried to replicate with a smaller toy case, but in this situation the function dust worked fine even without the reshaping trick.

So I guess it has to do with the fact that my data is too large? It's about 2 GB if stored in a CSV file.
Or maybe it comes from the fact that I'm using PyTorch and my data lives in a PT file? I'm very confused.

Thanks for your help. Please let me know if you need something more.

dust.zip

[william-silversmith]

I'm not sure, but I like your solution. It seems less brittle and special cased than what I wrote. I pushed a version of that to master and hope to release it soon.