screen recording

[saltlas]

Hi! I'm thinking of trying to implement a method that uses Page.startScreencast and Page.stopScreencast to generate a video of the browser that can be saved (.mp4, .avi, etc, whatever) and then creating a pull for it if I succeed. If I were to try to implement this are there any suggestions to keep in mind? Thinking of using ffmpeg to feed the screencast frames to, but if there's a better/preferred tool I should know about please let me know.

[saltlas]

just an update on this - i've gotten this sort-of-almost working (it's creating .mp4 videos from jpegs but they're like 4 frames, but a success is a success) but i have some stuff to work out before i can call it actually working, would appreciate some guidance if you have any insight:

1. as far as i can see the definitions for handling events (on(event)) are primarily in page.rb and there are none in browser.rb. i might be missing something that's leading me to misunderstand the code, though. anyway, screencasting would probably be per-browser, i assume, not per-page, unless pages stay alive for as long as the browser does. how would i implement the event handling for the browser? i'm currently defining it on the page.
2. the whole screencasting shebang needs to run in the background (i.e. sending back frame acknowledgements, adding new frames to the video file, etc.) while the browser is doing its thing, but it needs to run somewhat consistently and not whenever it feels like it. i've got a little bit of experience with dealing with threads/concurrency/asynchronous methods, but not a lot of it. what's the best way to get this to run in the background? it's also especially important that all of the frames finish being added before the call to close the ffmpeg subprocess goes through - is there a good way to check that if the adding of the frames is handled asynchronously? i know there's Concurrent::Async#wait but i'm not calling the frame adding method from the method that stops the screencasting... anyway i'm probably spouting confused nonsense since i'm not that experienced with that stuff.
3. i'm using the open3 gem to run the ffmpeg process. is that alright/desirable or is there a subprocess gem that is preferred?
4. on that same note, i'm feeding the images directly into ffmpeg without saving them to disk to prevent a bunch of useless disk space clutter, but if it would be better to save them to disk rather than have the open3 subprocess running in the background (to keep the ffmpeg pipe open as i feed it more frames) do advise

just a note: all good if this functionality is not actually something you want implemented in this gem for whatever reason! guidance would still be appreciated just to help me implement it locally for myself :). as far as i can tell, selenium doesn't have screen recording, https://github.com/kapoorlakshya/screen-recorder only has headless window screen recording for linux systems (as it relies on xvfb for it), and there's an additional library for puppeteer that someone made that you can add that only works in headful mode. point is, a cross-platform headless screen recording feature doesn't really exist, and it'd be useful.

[singhprd]

Hey - I'm not a maintainer here, or have any existing knowledge of how to do this, but I wondered if you could take some inspiration from Puppeteer - I see this library does what you're looking for:
https://github.com/prasanaworld/puppeteer-screen-recorder

So that might give a start for the CDP commands needed to do this:
https://github.com/prasanaworld/puppeteer-screen-recorder/blob/main/src/lib/pageVideoStreamCollector.ts#L55

Good luck, this would be a very nice feature especially for looking at flakey tests! 🚀

PS It would be great if you could share your code!

[saltlas]

that's a good idea @singhprd , i dismissed that library out of hand bc it only works in headful mode but youre right that it uses the same cdp tool so i should have a look. thanks!

in terms of sharing code, anything i do gets pushed to https://github.com/saltlas/ferrum/tree/screencast , if you want to look, but be warned that at the moment it is Very Bad, i'm just trying to get a very basic working product before i make it pretty so it's fugly. at the moment it generates very short videos of a few frames but only when run without calling the stop_screencast method, which is... bad? but anyway, you're free to have a look and keep an eye on it. i've stopped working on it for now because i got a bit roadblocked with the concurrency issues and all the stuff i asked about here, but looking at the puppeteer library might help

[sandstrom]

I'm doing some issue gardening 🌱🌿 🌷 and came upon this issue. Since it's quite old I just wanted to ask if this is still relevant? If it isn't, maybe we can close this issue?

By closing some old issues we reduce the list of open issues to a more manageable set.

[saltlas]

yeah i'm not working on this anymore or for the foreseeable future, life got in the way - you can close it unless you reckon it's something the maintainers would be interested in implementing themselves

[sandstrom]

@saltlas It's only you and maintainers that can close it, I cannot.

[route]

If it were a PR somebody could have continue working on it, but it's an feature request