Tauri implementation example please?

[cliqer]

Is it possible to use with Tauri and if so, can you please give an example?
Many thanks

[k0nserv]

I'm not super familiar with Tauri, but isn't Tauri based on a full browser window and as such couldn't the browser's implementation of WebRTC be used?

[cliqer]

@k0nserv it is indeed and works on Windows.
The problem is with MacOS where Tauri uses WKWebView which does not support getDisplayMedia.
As such, there is no other way to capture the screen through the web interface on macOS and we need a rust implementation for this.
Many of us need to port our electron webrtc applications to Tauri and WKWebView's minimal support is the main stopper thus looking for a solution on the Webrtc-rs library.
Would really appreciate any suggestions/ideas on solving this

[k0nserv]

I don't know enough about Tauri to make a super useful comment, but given that:

1. You can run regular old Rust
2. Can bind ports(UDP/TCP) etc
3. Can run Tokio

It ought to be possible to use webrtc-rs for this, I'm not sure how you'd actually render decoded video in the appropriate context.

Note: webrtc-rs doesn't currently concern itself with decoding,encoding, or acquiring media from the local machine at all, this needs to come from somewhere else.

Lastly, I know there are a fair amount of folks interested in the Tauri use case, please join the Discord and coordinate with others if you wish to explore this further.

[morajabi]

I'm working on a project involving Tauri for the UI and webrtc-rs. A bird-eye overview of the steps to get it working are as follows:

1. You should get started by writing a WebRTC client in the Rust layer. See this for a more complete example: https://github.com/webrtc-rs/examples/pull/15/files
2. Create a struct to manage your call state and initilize it in your Tauri app's src/main.rs, then give it to app.manage for Tauri to make it available for you to tauri command functions as a param so you can call methods on it. (example)
3. Utilize Tauri commands (invoke) from your front-end to manage your call state (e.g. to get media, create peer, remove peer, end call)
4. Get audio using cpal crate (best audio lib). ( Optionally you can use https://github.com/tonarino/webrtc-audio-processing to do echo cancallation)
5. Get camera video using something like nokhwa (actively developed)
6. Get screen capture from the scrap subcrate in this project which is an updated fork: https://github.com/rustdesk/rustdesk/tree/master/libs/scrap
7. Encode/decode audio using opus and video using libvp9 or another encoder. I haven't got to this part yet.
8. Render decoded video in an egui window in Tauri: https://github.com/tauri-apps/tauri-egui (because you can't access video frames from the webview, you need to use another kind of window rendering engine called egui which is managed in Rust. It's not that hard to get working.

Hope it helps.