Unexpected audioOffset Value for bookmarkReached Event in SSML with Break Times

[aamoronatti]

• Package Name: microsoft-cognitiveservices-speech-sdk
• Package Version: 1.29.0
• Operating system: Any
• nodejs
  • version: 18

Describe the bug

Bookmarks that are preceded by break times in SSML don't seem to work as expected when using the speakSsmlAsync method of the SpeechSynthesizer object. The audioOffset property in the bookmarkReached event seems to be misaligned.

To Reproduce

Steps to reproduce the behavior:

1. Create a SpeechSynthesizer object.
2. Subscribe to the bookmarkReached event.
3. Use speakSsmlAsync to synthesize an SSML string that includes bookmarks preceded by break times.
4. Observe that the audioOffset property of the bookmarkReached event data does not reflect the correct position.

Code

const sdk = require("microsoft-cognitiveservices-speech-sdk");

const subscriptionKey = "*";
const serviceRegion = "northeurope";
const config = sdk.SpeechConfig.fromSubscription(subscriptionKey, serviceRegion);

config.setProperty(sdk.PropertyId.SpeechServiceResponse_RequestSentenceBoundary, 'true');

const audioConfig = sdk.AudioConfig.fromDefaultSpeakerOutput();

const synthesizer = new sdk.SpeechSynthesizer(config, audioConfig);

synthesizer.bookmarkReached = function (s, e) {
    var str = `BookmarkReached event: \
            \r\n\tAudioOffset: ${(e.audioOffset + 5000) / 10000}ms \
            \r\n\tText: \"${e.text}\".`;
    console.log(str);
};

const speech_synthesis_voice_name = 'es-ES-AlvaroNeural';

const ssml = `<speak version='1.0' xml:lang='en-US' xmlns='http://www.w3.org/2001/10/synthesis' xmlns:mstts='http://www.w3.org/2001/mstts'>
    <voice name='${speech_synthesis_voice_name}'>
        Hola! <break time="5s"></break> <bookmark mark='1'/>Posicion de este mark.<bookmark mark='2'/>.
    </voice>
</speak>`;

console.log("SSML to synthesize: \r\n" + ssml);

synthesizer.speakSsmlAsync(
    ssml,
    result => {
        if (result) {
            console.log(JSON.stringify(result));
        }
        synthesizer.close();
    },
    error => {
        console.log(error);
        synthesizer.close();
    });

Expected behavior

BookmarkReached event:             
	AudioOffset: 5575.5ms             
	Text: "1".
BookmarkReached event:             
	AudioOffset: 7725.5ms             
	Text: "2".

Actual behavior

BookmarkReached event:             
	AudioOffset: 575.5ms             
	Text: "1".
BookmarkReached event:             
	AudioOffset: 7725.5ms             
	Text: "2".

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @robch @oscholz.

[aamoronatti]

Any update?