GoPro videos stuck on futex wait

[ztmr]

GoPro videos stuck on futex wait

Context:

All is working fine with DJI drone, but when I try to import stuff from GoPro Hero 7 Black, all photos are done, yet the process gets stuck on videos.
I am pulling this from the SD card removed from my device. I even tried to rsync the SD card contents to my local drive, but the same exact thing is happening.

• Camera: GoPro Hero 7 Black
• Firmware Version: HD7.01.01.00.00
• Operating System: Linux
• Commit ID: v1.8

Expected Behavior:

All files should be imported.

Current Behavior:

Only photos are imported, videos are waiting forever, in a possible deadlock.

Logs / Screenshots:

🎥 [HERO7 Black]:
        📹 FW: HD7.01.01.00.00 SN: [redacted]
        💾 1.7 TB/1.9 TB (90.32%)
Looking at 100GOPRO
GH010281.MP4: 0.00 b / 2.76 MB [----------------------------------------------------------] 0s
GOPR0282.JPG: 4.27 MB / 4.27 MB [==========================================================]  ✔️
GH010290.MP4: 0.00 b / 8.54 MB [----------------------------------------------------------] 0s
GOPR0283.JPG: 4.24 MB / 4.24 MB [==========================================================]  ✔️
GOPR0285.JPG: 4.54 MB / 4.54 MB [==========================================================]  ✔️
GH010297.MP4: 0.00 b / 27.69 MB [----------------------------------------------------------] 0s
GOPR0284.JPG: 4.93 MB / 4.93 MB [==========================================================]  ✔️
GOPR0286.JPG: 4.78 MB / 4.78 MB [==========================================================]  ✔️
GH010299.MP4: 0.00 b / 35.67 MB [----------------------------------------------------------] 0s
GOPR0287.JPG: 4.46 MB / 4.46 MB [==========================================================]  ✔️
GH010300.MP4: 0.00 b / 75.28 MB [----------------------------------------------------------] 0s
GH010301.MP4: 0.00 b / 41.74 MB [----------------------------------------------------------] 0s
GH010302.MP4: 0.00 b / 64.02 MB [----------------------------------------------------------] 0s
GH010303.MP4: 0.00 b / 114.67 MB [----------------------------------------------------------] 0s
GH010305.MP4: 0.00 b / 87.09 MB [----------------------------------------------------------] 0s
GH010306.MP4: 0.00 b / 3.35 GB [----------------------------------------------------------] 0s
GH010307.MP4: 0.00 b / 98.16 MB [----------------------------------------------------------] 0s
GH010308.MP4: 0.00 b / 361.73 MB [----------------------------------------------------------] 0s
GH010309.MP4: 0.00 b / 4.00 GB [----------------------------------------------------------] 0s
GH010310.MP4: 0.00 b / 1.90 GB [----------------------------------------------------------] 0s
GH010311.MP4: 0.00 b / 1.26 GB [----------------------------------------------------------] 0s
GH010312.MP4: 0.00 b / 4.00 GB [----------------------------------------------------------] 0s
GH010313.MP4: 0.00 b / 13.92 MB [----------------------------------------------------------] 0s
GH010314.MP4: 0.00 b / 4.00 GB [----------------------------------------------------------] 0s
GH020309.MP4: 0.00 b / 653.80 MB [----------------------------------------------------------] 0s
GH020312.MP4: 0.00 b / 852.90 MB [----------------------------------------------------------] 0s
GH020314.MP4: 0.00 b / 1.31 GB [----------------------------------------------------------] 0s
GOPR0288.JPG: 4.01 MB / 4.01 MB [==========================================================]  ✔️
GOPR0289.JPG: 5.03 MB / 5.03 MB [==========================================================]  ✔️
GOPR0291.JPG: 4.98 MB / 4.98 MB [==========================================================]  ✔️
GOPR0292.JPG: 3.85 MB / 3.85 MB [==========================================================]  ✔️
GOPR0293.JPG: 4.22 MB / 4.22 MB [==========================================================]  ✔️
GOPR0294.JPG: 3.97 MB / 3.97 MB [==========================================================]  ✔️
GOPR0295.JPG: 4.02 MB / 4.02 MB [==========================================================]  ✔️
GOPR0296.JPG: 3.95 MB / 3.95 MB [==========================================================]  ✔️
GOPR0298.JPG: 5.21 MB / 5.21 MB [==========================================================]  ✔️
GOPR0304.JPG: 3.91 MB / 3.91 MB [==========================================================]  ✔️

Steps to Reproduce:

$ mmt import -i /sdcard/mount/point --camera-name GoProHero7 -d "yyyymmdd" -n TestProject -o /my/target/path -c gopro -v

I tried running with GODEBUG="asyncpreemptoff=1", which didn't help.

Here is an extempt from the strace log:

$ strace --follow-forks --failed-only -p $(pgrep mmt)
strace: Process 57483 attached with 19 threads
[pid 57484] restart_syscall(<... resuming interrupted read ...>) = -1 ETIMEDOUT (Connection timed out)
[pid 57539] futex(0xd2f668, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=100000}) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57539] futex(0xd2f690, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=100000}) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57539] futex(0xd2f690, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=100000}) = -1 ETIMEDOUT (Connection timed out)
[pid 57539] futex(0xd2f598, FUTEX_WAIT_PRIVATE, 2, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57539] futex(0xd2f690, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=100000}) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57484] futex(0xd2f678, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=169228917}) = -1 ETIMEDOUT (Connection timed out)
[pid 57486] futex(0xc00007c948, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57486] futex(0xc00007c948, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57484] futex(0xd2f678, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=169293148}) = -1 ETIMEDOUT (Connection timed out)
[pid 57486] futex(0xc00007c948, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57485] futex(0xc00007c548, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57484] futex(0xd2f678, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=169633355}) = -1 ETIMEDOUT (Connection timed out)
[pid 57487] futex(0xc000100148, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57495] futex(0xc000680148, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57484] futex(0xd2f678, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=177927910}) = -1 ETIMEDOUT (Connection timed out)
[pid 57487] futex(0xc000100148, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57539] futex(0xc000101148, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57487] futex(0xc000100148, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57484] futex(0xd2f678, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=178290418}) = -1 ETIMEDOUT (Connection timed out)
[pid 57484] futex(0xd2f598, FUTEX_WAIT_PRIVATE, 2, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 59589] futex(0xd2f690, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=100000}) = -1 ETIMEDOUT (Connection timed out)
[pid 57493] futex(0xd2f690, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=100000}) = -1 EAGAIN (Resource temporarily unavailable)
[pid 57493] futex(0xd2f690, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=100000}) = -1 EAGAIN (Resource temporarily unavailable)
[...]

What is strange is that even the file size is not detected. All permissions are OK, the process can read the file/directory structures. It would be great if there was any way to run mmt in a serial mode with threading disabled entirely, as mentioned in #116.

[ztmr]

It turns out that the ffprobe was not in the path. The error has not been properly propagated from the getRfpsFolder function, so the wait group is waiting for these entries although they get skipped. Would the right fix be to call wg.Done() prior returning godirwalk.SkipThis? And showing some form of warning/error message too...