Skip to content

WIP: Audio recording and playback #163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 50 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
cf08a1d
codal_port/modaudio: Extend AudioFrame constructor to take opt size.
dpgeorge Nov 14, 2023
2119cd9
codal_port/modaudio: Rework audio playing to play a long AudioFrame.
dpgeorge Nov 14, 2023
1dc4c12
codal_port/modaudio: Remove buffer expansion.
dpgeorge Nov 14, 2023
07190d5
codal_app: Add microphone recording interface.
dpgeorge Nov 14, 2023
edfa023
codal_port/microbit_microphone: Add methods to record.
dpgeorge Nov 14, 2023
7e3b102
codal_port/modaudio: Add audio.sound_level() method.
dpgeorge Nov 14, 2023
c487a38
src: Add test recording program.
dpgeorge Nov 14, 2023
30ef534
codal_port/modaudio: Rename "size" field to "alloc_size".
dpgeorge Jan 15, 2024
affdb0a
codal_port/modaudio: Make AudioFrame constructor public.
dpgeorge Jan 15, 2024
8ae75da
codal_app/microbithal_microphone: Store the current recording length.
dpgeorge Jan 15, 2024
1e4ae5b
codal_port/modaudio: Add a "used_size" field to AudioFrame.
dpgeorge Jan 15, 2024
0cf7f04
codal_port/modaudio: Add rate to AudioFrame.
dpgeorge Jan 15, 2024
ab6ea69
codal_port/microbit_microphone: Use rate from AudioFrame, or update it.
dpgeorge Jan 15, 2024
bd7905d
codal_port/modaudio: Use AudioFrame rate as rate when playing it.
dpgeorge Jan 15, 2024
2756929
src: Update test_record.py.
dpgeorge Jan 15, 2024
19dbe4b
codal_app/main: Set microphone gain to 0.2.
dpgeorge Jan 18, 2024
f1b03ee
codal_port/modaudio: Take the sqrt of sound_level.
dpgeorge Jan 18, 2024
4e3ac1d
codal_port/modaudio: Mostly use "alloc_size" instead of "used_size".
dpgeorge Jan 18, 2024
40979ac
codal_port/modaudio: Round up AudioFrame size to nearest 32.
dpgeorge Jan 18, 2024
4878157
codal_port/modaudio: Increas "used_size" when data is written.
dpgeorge Jan 18, 2024
5fe6307
src: Update test_record.py.
dpgeorge Jan 18, 2024
a528b1e
codal_port/modaudio: Update to build with latest micropython.
dpgeorge Jan 18, 2024
71d9430
codal_app/microbithal_microphone: Make input streaming use pullInto.
dpgeorge Feb 26, 2024
8528e89
codal_port/microbit_microphone: Implement wait argument to record_into.
dpgeorge Mar 25, 2024
5d1b3b5
codal_app/microbithal_microphone: Remove CODAL workaround.
dpgeorge Mar 25, 2024
7ba6c03
codal_app/microbithal_microphone: Add func to set the sensitivity.
dpgeorge Apr 22, 2024
b19fbde
codal_port/microbit_microphone: Add set_sensitivity method and consts.
dpgeorge Apr 22, 2024
8041d86
codal_port/modaudio: Ensure AudioFrame size is non-zero.
dpgeorge Apr 22, 2024
80cfac7
codal_port/microbit_microphone: Validate duration and rate args.
dpgeorge Apr 22, 2024
61aaf2a
codal_port/modaudio: Allow AudioFrame to be arbitrary length.
dpgeorge Apr 29, 2024
e8a5d61
codal_port/modaudio: Make None the default to AudioFrame().
dpgeorge Apr 29, 2024
0441aa2
codal_port/microbit_microphone: Require rate to be positive.
dpgeorge Apr 29, 2024
c7f64c1
codal_port/modaudio: Fix typecode of AudioFrame buffer to be unsigned.
dpgeorge Apr 30, 2024
c741736
codal_port/modaudio: Use mp_sched_schedule_node for audio fetcher.
dpgeorge May 20, 2024
26d077a
codal_port/modaudio: Separate default AudioFrame and output buffer size.
dpgeorge May 20, 2024
a4fb09b
codal_port/modaudio: Allow output buffer to be larger than 32 bytes.
dpgeorge May 20, 2024
a280e25
codal_port/modaudio: Stop streaming audio when data is exhausted.
dpgeorge May 27, 2024
9819ff8
codal_port: Implement AudioTrack and AudioRecording.
dpgeorge Aug 1, 2024
3fba8c9
src: Update test_record.py.
dpgeorge Aug 2, 2024
96a24ff
codal_port/modaudio: Remove used_size entry from AudioFrame type.
dpgeorge Aug 5, 2024
02e5289
codal_port/modaudio: Make AudioFrame add and mult helpers public.
dpgeorge Aug 21, 2024
30251c7
codal_port/microbit_audiotrack: Implement +,+=,-,-=,*,*= on AudioTrack.
dpgeorge Aug 21, 2024
ceb13c8
src: Update test_record.py to use mult for volume.
dpgeorge Aug 21, 2024
e337b94
src: In test_record.py, make sure my_track is defined.
dpgeorge Aug 22, 2024
d4c7b67
codal_port/microbit_audiorecording: Make track() args positional.
dpgeorge Aug 26, 2024
5cc0cb0
src: Use was_pressed in test_record.py.
dpgeorge Sep 9, 2024
5b36b01
codal_port/modaudio: Improve feeding of audio pipeline.
dpgeorge Sep 9, 2024
311dece
codal_app/microbithal_audio: Add microbit_hal_audio_is_playing() func.
dpgeorge Sep 9, 2024
4c957c4
codal_port/modaudio: Make audio waiting wait for audio to be silent.
dpgeorge Sep 9, 2024
df4cb05
codal_port/modaudio: Make sure final partial audio data is sent out.
dpgeorge Sep 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
codal_port/modaudio: Add a "used_size" field to AudioFrame.
Signed-off-by: Damien George <damien@micropython.org>
  • Loading branch information
dpgeorge committed Sep 18, 2024
commit 1e4ae5be56120c1b5e96ecd07157db5f5bee6473
12 changes: 8 additions & 4 deletions src/codal_port/microbit_microphone.c
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ static MP_DEFINE_CONST_FUN_OBJ_1(microbit_microphone_get_events_obj, microbit_mi

static void microbit_microphone_record_helper(microbit_audio_frame_obj_t *audio_frame, int rate, bool wait) {
// Start the recording.
microbit_hal_microphone_start_recording(audio_frame->data, audio_frame->alloc_size, rate);
microbit_hal_microphone_start_recording(audio_frame->data, audio_frame->alloc_size, &audio_frame->used_size, rate);

if (wait) {
// Wait for the recording to finish.
Expand Down Expand Up @@ -206,10 +206,14 @@ static mp_obj_t microbit_microphone_record_into(mp_uint_t n_args, const mp_obj_t
mp_arg_val_t args[MP_ARRAY_SIZE(allowed_args)];
mp_arg_parse_all(n_args - 1, pos_args + 1, kw_args, MP_ARRAY_SIZE(allowed_args), allowed_args, args);

mp_buffer_info_t bufinfo;
mp_get_buffer_raise(args[ARG_buffer].u_obj, &bufinfo, MP_BUFFER_READ);
// Check that the buffer is an AudioFrame instance.
if (!mp_obj_is_type(args[ARG_buffer].u_obj, &microbit_audio_frame_type)) {
mp_raise_TypeError(MP_ERROR_TEXT("expecting an AudioFrame"));
}
microbit_audio_frame_obj_t *audio_frame = MP_OBJ_TO_PTR(args[ARG_buffer].u_obj);

microbit_hal_microphone_start_recording(bufinfo.buf, bufinfo.len, args[ARG_rate].u_int);
// Start the recording.
microbit_hal_microphone_start_recording(audio_frame->data, audio_frame->alloc_size, &audio_frame->used_size, args[ARG_rate].u_int);

return mp_const_none;
}
Expand Down
16 changes: 9 additions & 7 deletions src/codal_port/modaudio.c
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ static void audio_data_fetcher(void) {

if (audio_source_frame != NULL) {
// An existing AudioFrame is being played, see if there's any data left.
if (audio_raw_offset >= audio_source_frame->alloc_size) {
if (audio_raw_offset >= audio_source_frame->used_size) {
// AudioFrame is exhausted.
audio_source_frame = NULL;
}
Expand Down Expand Up @@ -318,7 +318,7 @@ static mp_obj_t microbit_audio_frame_new(const mp_obj_type_t *type_in, mp_uint_t
static mp_obj_t audio_frame_subscr(mp_obj_t self_in, mp_obj_t index_in, mp_obj_t value_in) {
microbit_audio_frame_obj_t *self = (microbit_audio_frame_obj_t *)self_in;
mp_int_t index = mp_obj_get_int(index_in);
if (index < 0 || index >= self->alloc_size) {
if (index < 0 || index >= self->used_size) {
mp_raise_ValueError(MP_ERROR_TEXT("index out of bounds"));
}
if (value_in == MP_OBJ_NULL) {
Expand All @@ -341,7 +341,7 @@ static mp_obj_t audio_frame_unary_op(mp_unary_op_t op, mp_obj_t self_in) {
microbit_audio_frame_obj_t *self = (microbit_audio_frame_obj_t *)self_in;
switch (op) {
case MP_UNARY_OP_LEN:
return MP_OBJ_NEW_SMALL_INT(self->alloc_size);
return MP_OBJ_NEW_SMALL_INT(self->used_size);
default:
return MP_OBJ_NULL; // op not supported
}
Expand All @@ -351,14 +351,14 @@ static mp_int_t audio_frame_get_buffer(mp_obj_t self_in, mp_buffer_info_t *bufin
(void)flags;
microbit_audio_frame_obj_t *self = (microbit_audio_frame_obj_t *)self_in;
bufinfo->buf = self->data;
bufinfo->len = self->alloc_size;
bufinfo->len = self->used_size;
bufinfo->typecode = 'b';
return 0;
}

static void add_into(microbit_audio_frame_obj_t *self, microbit_audio_frame_obj_t *other, bool add) {
int mult = add ? 1 : -1;
size_t size = MIN(self->alloc_size, other->alloc_size);
size_t size = MIN(self->used_size, other->used_size);
for (int i = 0; i < size; i++) {
unsigned val = (int)self->data[i] + mult*(other->data[i]-128);
// Clamp to 0-255
Expand All @@ -371,6 +371,7 @@ static void add_into(microbit_audio_frame_obj_t *self, microbit_audio_frame_obj_

static microbit_audio_frame_obj_t *copy(microbit_audio_frame_obj_t *self) {
microbit_audio_frame_obj_t *result = microbit_audio_frame_make_new(self->alloc_size);
result->used_size = self->used_size;
for (int i = 0; i < self->alloc_size; i++) {
result->data[i] = self->data[i];
}
Expand All @@ -381,7 +382,7 @@ mp_obj_t copyfrom(mp_obj_t self_in, mp_obj_t other) {
microbit_audio_frame_obj_t *self = (microbit_audio_frame_obj_t *)self_in;
mp_buffer_info_t bufinfo;
mp_get_buffer_raise(other, &bufinfo, MP_BUFFER_READ);
uint32_t len = MIN(bufinfo.len, self->alloc_size);
uint32_t len = MIN(bufinfo.len, self->used_size);
for (uint32_t i = 0; i < len; i++) {
self->data[i] = ((uint8_t *)bufinfo.buf)[i];
}
Expand Down Expand Up @@ -418,7 +419,7 @@ int32_t float_to_fixed(float f, uint32_t scale) {

static void mult(microbit_audio_frame_obj_t *self, float f) {
int scaled = float_to_fixed(f, 15);
for (int i = 0; i < self->alloc_size; i++) {
for (int i = 0; i < self->used_size; i++) {
unsigned val = ((((int)self->data[i]-128) * scaled) >> 15)+128;
if (val > 255) {
val = (1-(val>>31))*255;
Expand Down Expand Up @@ -474,6 +475,7 @@ microbit_audio_frame_obj_t *microbit_audio_frame_make_new(size_t size) {
microbit_audio_frame_obj_t *res = m_new_obj_var(microbit_audio_frame_obj_t, uint8_t, size);
res->base.type = &microbit_audio_frame_type;
res->alloc_size = size;
res->used_size = 0;
memset(res->data, 128, size);
return res;
}
Expand Down
1 change: 1 addition & 0 deletions src/codal_port/modaudio.h
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
typedef struct _microbit_audio_frame_obj_t {
mp_obj_base_t base;
size_t alloc_size;
size_t used_size;
uint8_t data[];
} microbit_audio_frame_obj_t;

Expand Down