[Feature]: avi support for DANDI

[bendichter]

We don't currently have a way of viewing .avi files in browser. See dandi/dandi-archive#1626 (comment)

Your readme claims you can do this.

I would like to have the ability to play .avi files stored on DANDI, e.g.

by launching neurosift as an external service. Would it be possible to provide a routing, for this e.g.

https://flatironinstitute.github.io/neurosift/#/avi?url=https://dandiarchive.s3.amazonaws.com/blobs/a04/856/a0485689-709f-44c7-b412-73aaa2f03184

We would then be able to add this as an external service with a PR analogous to dandi/dandi-archive#1647

[magland]

Right now in neurosift, Python is needed to extract frames from the .avi file. So this works when accessing data that is being hosted in an rtcshare directory with a running rtcshare service. I don't think there's a way to stream avi to the browser, unfortunately.

Maybe we can think of a way to get this to work, though.

[bendichter]

ok thanks, good to know

[magland]

One possibility is to set up a serverless function as a service that uses Python to stream the video - you'd request a collection of frames at a time. It seems that opencv has the ability to stream from a remote file. However the following test suggests that this would be too slow:

import time
import cv2
url = 'https://dandiarchive.s3.amazonaws.com/blobs/a04/856/a0485689-709f-44c7-b412-73aaa2f03184'
timer = time.time()
cap = cv2.VideoCapture(url)
print(f'Elapsed time for opening the video (sec): ', time.time() - timer)
num_frames = 20
timer = time.time()
for frame_index in range(num_frames):
    cap.set(cv2.CAP_PROP_POS_FRAMES, frame_index)
    ret, frame = cap.read()
    print(frame.shape)
print(f'Elapsed time for capturing {num_frames} frames (sec): ', time.time() - timer)

Elapsed time for opening the video (sec):  0.8549864292144775
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
(480, 480, 3)
Elapsed time for capturing 20 frames (sec):  15.460177659988403

Even if the file was in .mp4 format (so it could be played in the browser), there's not a good way to load individual frames to the browser so that you could draw annotations over top. Neurosift with rtcshare overcomes this by using the Python service.

We'll need to think about this some more.

[magland]

Submitted a PR to DANDI about this. Closing.