MSC3077: Support for multi-stream VoIP #3077

SimonBrandner · 2021-03-29T18:02:08Z

Fixes https://github.com/matrix-org/matrix-doc/issues/560
Requires #2746
Rendered

js-sdk implementation
react-sdk implementation

My first MSC, please try not to kill me 😅

FCP tickyboxes

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

proposals/3077-multi-stream-voip.md

Co-authored-by: Tulir Asokan <tulir@maunium.net>

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

…atrix-doc into msc/sdp-metadata

ShadowJonathan

screenshare when 👀

ShadowJonathan

Follow-up: I would suggest adding some more explanation to what m.usermedia and m.screenshare actually means in this (new?) context, to understand what this adds exactly, but otherwise, this looks good.

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

dbkr

otherwise this looks great - congrats on your first MSC :)

proposals/3077-multi-stream-voip.md

I would be tempted to skip the capability advertisement for this and just say that the absence of stream metadata means clients just take the first or whatever MSC2746 says. I can't think of anything the capability advertisement caters for specifically? - Dave Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

proposals/3077-multi-stream-voip.md

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

dbkr · 2021-04-07T09:15:30Z

proposals/3077-multi-stream-voip.md

+
+ `purpose` - a string indicating the purpose of the stream. For compatibility
+  between clients values `m.usermedia` and `m.screenshare` are defined.
+  `m.usermedia` is the stream that contains the webcam and microphone tracks.


continuing thread on whether we should identify front/rear facing cameras with this, for purposes like when the user rotates the camera and the same stream changes to a different camera (eg. FaceTime displays an animation on the remote side to show that the camera has changed). I feel like this is probably something for separate MSC, unless we want two separate purposes for front/rear camera.

Yeah, unless we want a separate purpose for that probably a separate MSC. If we wanted a separate purpose, would we have m.usermedia, m.usermedia.front and m.usermedia.rear or something else?

eg. FaceTime displays an animation on the remote side to show that the camera has changed

This bit seems more like a stream replacement problem rather than a purpose problem. But... maybe we could allow sending both and set a main stream which would tell the client on the other side to do the animation

My suggestion would be adding something like camere_type to further distinguish the cameras - this way not all clients need to support this and we avoid having two purposes that are almost the same

proposals/3077-multi-stream-voip.md

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

mscbot · 2023-06-20T11:33:55Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

* Draft of multi-stream MSC Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Remove unnecessary article and don't use CamelCase Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Fix naming and MSC number Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Make it prefixy Co-authored-by: Tulir Asokan <tulir@maunium.net> * Be more descriptive about keys Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Add more info about usermedia and screenshare Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Simplifie backwards compatibility I would be tempted to skip the capability advertisement for this and just say that the absence of stream metadata means clients just take the first or whatever MSC2746 says. I can't think of anything the capability advertisement caters for specifically? - Dave Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Reword parts of backwards compatibility section Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Improve explanation of backwards compatibility Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Add missing spaces to unstable perifix table Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Remove support for stream-replacement Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Apply suggestions from code review Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix concerns Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Link to specific spec version Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Be more readable Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Clarify Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Don't ref non-existing thing Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Remove confusing words --------- Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> Co-authored-by: Tulir Asokan <tulir@maunium.net> Co-authored-by: David Baker <dbkr@users.noreply.github.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Specs matrix-org/matrix-spec-proposals#3077

turt2live · 2023-08-01T05:09:28Z

Spec PR: matrix-org/matrix-spec#1602

turt2live · 2024-03-13T17:12:18Z

Spec PR: matrix-org/matrix-spec#1735

turt2live · 2024-03-19T18:58:34Z

Merged 🎉

SimonBrandner added 2 commits March 29, 2021 12:09

Draft of multi-stream MSC

6b11473

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

Remove unnecessary article and don't use CamelCase

f8a38dd

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

SimonBrandner changed the title ~~MSC: Support for multi-stream VoIP~~ MSC3077: Support for multi-stream VoIP Mar 29, 2021

Fix naming and MSC number

98053bb

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

SimonBrandner marked this pull request as ready for review March 29, 2021 18:03

turt2live added kind:feature MSC for not-core and not-maintenance stuff proposal A matrix spec change proposal proposal-in-review labels Mar 29, 2021

tulir reviewed Mar 29, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

SimonBrandner and others added 3 commits March 29, 2021 20:57

Make it prefixy

e5a95f4

Co-authored-by: Tulir Asokan <tulir@maunium.net>

Be more descriptive about keys

5939d19

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

Merge branch 'msc/sdp-metadata' of https://github.com/SimonBrandner/m…

14b37ba

…atrix-doc into msc/sdp-metadata

ShadowJonathan approved these changes Mar 30, 2021

View reviewed changes

ShadowJonathan reviewed Mar 30, 2021

View reviewed changes

Add more info about usermedia and screenshare

721eec5

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

dbkr reviewed Apr 1, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

SimonBrandner added 2 commits April 1, 2021 20:42

Reword parts of backwards compatibility section

668ef57

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

kevincox reviewed Apr 2, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

proposals/3077-multi-stream-voip.md Show resolved Hide resolved

SimonBrandner added 2 commits April 2, 2021 16:34

Improve explanation of backwards compatibility

9f372fa

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

Add missing spaces to unstable perifix table

96a878b

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

dbkr reviewed Apr 7, 2021

View reviewed changes

SimonBrandner commented Apr 10, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

SimonBrandner commented Apr 10, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

This was referenced Apr 14, 2021

once supported by m.call.*, add screensharing mid-call element-hq/element-web#239

Closed

Make screen share option more obvious (and toggle from normal call) element-hq/element-web#12652

Closed

SimonBrandner mentioned this pull request May 7, 2021

Support for screen-sharing using multi-stream VoIP (MSC3077) matrix-org/matrix-js-sdk#1685

Merged

turt2live added the needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. label Jun 8, 2021

This was referenced Jul 22, 2021

MSC3291: Muting in VoIP calls #3291

Merged

Readonly support for MSC3077 element-hq/element-android#3734

Open

mscbot added final-comment-period This MSC has entered a final comment period in interest to approval, postpone, or delete in 5 days. and removed proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. labels Jun 15, 2023

richvdh reviewed Jun 15, 2023

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

SimonBrandner and others added 5 commits June 17, 2023 18:28

Link to specific spec version

904e4c9

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Be more readable

875edb9

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Clarify

744fa24

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Don't ref non-existing thing

3d22c03

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Remove confusing words

8d43930

mscbot added finished-final-comment-period and removed disposition-merge final-comment-period This MSC has entered a final comment period in interest to approval, postpone, or delete in 5 days. labels Jun 20, 2023

turt2live merged commit ddad150 into matrix-org:old_master Jun 20, 2023

turt2live added spec-pr-missing Proposal has been implemented and is being used in the wild but hasn't yet been added to the spec and removed finished-final-comment-period labels Jun 20, 2023

SimonBrandner deleted the msc/sdp-metadata branch June 20, 2023 16:22

dbkr added a commit to matrix-org/matrix-spec that referenced this pull request Jul 18, 2023

Spec for MSC3077

47db8f2

Specs matrix-org/matrix-spec-proposals#3077

dbkr mentioned this pull request Jul 18, 2023

Spec for MSC3077 matrix-org/matrix-spec#1602

Closed

turt2live added spec-pr-in-review A proposal which has been PR'd against the spec and is in review and removed spec-pr-missing Proposal has been implemented and is being used in the wild but hasn't yet been added to the spec labels Aug 1, 2023

zecakeh mentioned this pull request Feb 28, 2024

Add support for multi-stream VoIP matrix-org/matrix-spec#1735

Merged

turt2live added merged A proposal whose PR has merged into the spec! and removed spec-pr-in-review A proposal which has been PR'd against the spec and is in review labels Mar 19, 2024

zecakeh mentioned this pull request Mar 22, 2024

Implement Matrix 1.10 changes ruma/ruma#1735

Closed

28 tasks

clokep mentioned this pull request Mar 22, 2024

Declare support for Matrix v1.10 element-hq/synapse#17021

Closed

8 tasks

zecakeh mentioned this pull request Mar 22, 2024

events: Add support for multi-stream VoIP ruma/ruma#1752

Merged

MSC3077: Support for multi-stream VoIP #3077

MSC3077: Support for multi-stream VoIP #3077

Uh oh!

Conversation

SimonBrandner commented Mar 29, 2021 • edited by richvdh Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ShadowJonathan left a comment

Choose a reason for hiding this comment

Uh oh!

ShadowJonathan left a comment

Choose a reason for hiding this comment

Uh oh!

dbkr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dbkr Apr 7, 2021

Choose a reason for hiding this comment

Uh oh!

SimonBrandner Apr 7, 2021

Choose a reason for hiding this comment

Uh oh!

SimonBrandner Apr 7, 2021

Choose a reason for hiding this comment

Uh oh!

SimonBrandner Jul 7, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mscbot commented Jun 20, 2023

Uh oh!

turt2live commented Aug 1, 2023

Uh oh!

turt2live commented Mar 13, 2024

Uh oh!

turt2live commented Mar 19, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

SimonBrandner commented Mar 29, 2021 •

edited by richvdh

Loading