Why is lofty so slow?

[bmillham]

I'm learning Rust and my first attempt is to read tags from my 100K+ files music library.

In Python I can read the tags in about 4 minutes. With Rust and Lofty it takes 45 minutes to do the same!!!

I've verified that I can just print every filename and it takes about 4 seconds. Add in grabbing the tags and it's 45 minutes.

I have tried building (thinking it's debugging info) but I get the same results.

[Serial-ATA]

Hello!

That doesn't sound right. Can you share the code you're using?

[bmillham]

Sorry for the delay in responding, but I wanted to do a little more testing. I got to thinking that the files I'm scanning are on a remote NFS share so I wanted to rule that out as the cause. I've installed the code on the system that is hosting the files and get the same results.

What is interesting is that if I run the scan a second time then it runs at the speed I'd expect. I'd have to guess that it's file caching that is speeding it up. But the similar Python code always runs in around 4-7 minutes. The rust code initially takes 45+ minutes to run.

Here is a link to the code on github: https://github.com/bmillham/tag_test.git

I am still learning Rust, so it may just be something I'm doing wrong. Go ahead a tear my code apart and tell me how terrible it is ☺️ That will help me learn Rust better!

[Serial-ATA]

Thanks for sharing the code. Are you running in release mode?

I have a library of 3200 files:

Estimating files to scan
"flac": 1014
"jpg": 964
"m4a": 166
"mp3": 378
"opus": 1701
"png": 58
"ytdl": 1
Valid 3259, Other: 1023 Dirs: 1570
Scanning files for tags
"flac": 1014
"jpg": 964
"m4a": 166
"mp3": 378
"opus": 1701
"png": 58
"ytdl": 1
Valid 3259, Other: 1023, Error: 0, Dirs: 1570

So significantly smaller than yours, but it takes <1sec to scan all my files in release mode:

________________________________________________________
Executed in  923.17 millis    fish           external
   usr time  338.50 millis    0.00 micros  338.50 millis
   sys time  583.45 millis  344.00 micros  583.11 millis

In debug mode though, it takes ~18sec:

________________________________________________________
Executed in   18.25 secs    fish           external
   usr time   15.66 secs    0.00 micros   15.66 secs
   sys time    0.71 secs  282.00 micros    0.71 secs

I imagine the combination of debug mode, 100k+ files, and them being transferred over the network could easily add up to many minutes.