SODA Dataset for Training #35
Conversation
farzadab
force-pushed
the
farzad-soda-train
branch
2 times, most recently
from
6248f64
to
5ba015c
Compare
|
The PR is ready. |
farzadab
force-pushed
the
farzad-soda-train
branch
from
5ba015c
to
e4ec48a
Compare
ultravox/data/datasets.py
Outdated
| roles = ["user", "assistant"] if len(turns) % 2 == 0 else ["assistant", "user"] | ||
|
|
||
| num_prompts = min(self._args.num_prompts, len(self.SYS_PROMPTS)) | ||
| sys_prompt = self.SYS_PROMPTS[idx % num_prompts] |
There was a problem hiding this comment.
did we end up using a RNG for this sort of thing rather than the index?
There was a problem hiding this comment.
(I forget where but we discussed adding a private RNG to datasets to allow them to simply pull a value from the RNG rather than using the index counter and various moduli)
There was a problem hiding this comment.
Yes, the idea was that we do that in the next PR.
There was a problem hiding this comment.
Might as well just do it now I guess since I have the code.
| for column_name in self.BASE_AUDIO_COLUMNS: | ||
| dataset = dataset.cast_column( | ||
| column_name, datasets.Audio(sampling_rate=SAMPLE_RATE) | ||
| ) |
There was a problem hiding this comment.
Bugfix for datasets that have audio column that is not named audio.
There was a problem hiding this comment.
This was not an issue for SODA since it was constructed with 16K Hz, but it was sloppy of me.
No description provided.