Skip to content

In-app audio capture format converter #598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Feb 21, 2025
Merged

In-app audio capture format converter #598

merged 6 commits into from
Feb 21, 2025

Conversation

hiroshihorie
Copy link
Member

Runtime audio format converting instead of hard-coding the format.
ReplayKit uses different format from macOS screen share app audio.

@ladvoc
Copy link
Contributor

ladvoc commented Feb 20, 2025

I just tested this with audio samples from ReplayKit; no issues.

@hiroshihorie
Copy link
Member Author

hiroshihorie commented Feb 20, 2025 •
edited
Loading

@ladvoc Thanks for testing.
In macOS for some reason I hear crackling sounds which is bugging me for a few hours now...
Is it smooth in ReplayKit ?

@ladvoc
Copy link
Contributor

ladvoc commented Feb 20, 2025

Actually, I also hear crackling sounds now. Does it sound like this for you too?

sample.mp4

@hiroshihorie
Copy link
Member Author

@ladvoc Yes sounds pretty terrible, not sure what's going on. The convert part seems to be ok. Maybe something related to scheduling.

@ladvoc
Copy link
Contributor

ladvoc commented Feb 20, 2025

@hiroshihorie, I checked the output status of converter.convert(to:error:), and it is returning inputRanDry, which according to the docs indicates "the method doesn’t have enough input available to satisfy the request." Maybe buffering app audio samples would fix the issue?

@hiroshihorie
Copy link
Member Author

@ladvoc I think it always returns .inputRanDry when setting outStatus.pointee = .noDataNow, I think it should be ok. Test seems to be ok 🤔

@hiroshihorie
Copy link
Member Author

@ladvoc I think it's fixed now

@hiroshihorie hiroshihorie marked this pull request as ready for review February 21, 2025 14:21
@hiroshihorie hiroshihorie changed the title Dynamic In-app audio capturing In-app audio capture format converter Feb 21, 2025
@ladvoc
Copy link
Contributor

ladvoc commented Feb 21, 2025

Yes, it works perfectly now!

@ladvoc ladvoc mentioned this pull request Feb 21, 2025
Merged
@hiroshihorie
Copy link
Member Author

hiroshihorie commented Feb 21, 2025 •
edited
Loading

@ladvoc Nice! I tried the iOS version also works great. 😁

@hiroshihorie hiroshihorie merged commit 61e5bd1 into main Feb 21, 2025
12 of 13 checks passed
@hiroshihorie hiroshihorie deleted the hiroshi/dynamic-app-audio-format branch February 21, 2025 18:55
hiroshihorie added a commit that referenced this pull request Feb 24, 2025
@ladvoc @hiroshihorie
iOS screen share audio (#576)
4f2f5c2 
This PR adds support for capturing application audio during screen share
when using a broadcast extension.

Note: PR #598 should be merged first.

---------

Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hiroshihorie @ladvoc