Speech API not working for some files

[BalaAudio]

Though speech_rest API worked correctly for the given resources, it returned null JSON everytime whenever I passed other audio files in raw format (16-bit, little endian, 16KHz)?

I've attached screenshot of the error. I've also attached that audio file as ZIP (Though the conversation is in another language, it has a few English words like "Hello" which I want to capture.).

[theacodes]

@puneith or @jerjou can you take a look?

[edesaintchamas]

Same issue:

The provided .raw files are working:
$ python speech_rest.py resources/audio2.raw
{"results": [{"alternatives": [{"confidence": 0.97258663, "transcript": "the rain in Spain stays mainly on the plain"}]}]}

But the provided FLAC does not
$ python speech_rest.py resources/audio.flac
{}

I also tried with an mp3 voice sample I found online (https://archive.org/details/VoiceSample) which also returns {}, no results or error are provided.

[puneith]

Can you please edit the speech_rest.py file (in line 63) and change the encoding to "FLAC". I will open a ticket to make this configurable.

[BalaAudio]

Hey,

I did NOT pass a FLAC filr.

I passed in a "raw" format file. It worked for the raw files in the
resources directory. When I passed our own 16-bit little endian 16 KHz
sampled raw audio file, it printed null set.

PFA the raw file. Thanks.

Balaraman Lakshmanan | Research Engineer | +91 9900531463
Website http://exotel.in/ | We’re Hiring http://exotel.in/about/careers/ |
Twitter https://twitter.com/balaramanl | LinkedIn
https://in.linkedin.com/in/balaramanl

On Thu, Jul 21, 2016 at 9:48 PM, puneith notifications@github.com wrote:

Can you please edit the speech_rest.py file (in line 63) and change the
encoding to "FLAC". I will open a ticket to make this configurable.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#424 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/APKwsF3CiVN1Oym6A4XYGdaSraZSY7-fks5qX5vGgaJpZM4JOawa
.

[jerjou]

To be fair, I didn't understand the audio file either :-)

I think at least one of the main issues here is that it wasn't obvious that the api looks for english (US) by default if you don't specify the language. I'm updating the sample to explicitly state the language, and add a link to the config docs so you can look up other languages you can specify.

I realize you mentioned the audio wasn't in English, but there were isolated english words within the audio. That's a little bit tougher for the API to pick out. If you think of how humans process audio, oftentimes you'll recognize a word not just based on its own sound, but on the words and context that surround it. For example, "year" and "ear" sound pretty similar, and you mostly distinguish them by context. Machines also still aren't as good as humans at recognizing speech, so they rely more heavily on surrounding words than humans do.

Re: flac / mp3. The speech_rest.py (and all the files in this repo, really) is meant more as code to be read and learned from, than utilities in and of themselves. If you take a look in the file, you'll notice that the script hard-codes a specific audio encoding of the audio file (ie uncompressed 16-bit signed little-endian). You can change that to specify FLAC, to process the FLAC files, but the API does not support mp3 at this time. If you'd like to process mp3 files, you'll have to decode them to a supported encoding first, although recognition accuracy may be worse than if you had an audio file that hadn't been through lossy compression.

[edesaintchamas]

Hi @puneith @jerjou

Thanks for your answers. Got it working perfectly on my side!

[BalaAudio]

Thanks for the detailed explanation. :)

I get your point on differentiating between "year" & "ear" based on
context. My interest is just to find specific English keywords inside these multi-lingual
recordings. Can that be done?

I will check with flac files and update you whether it works. Thanks again.

Balaraman Lakshmanan | Research Engineer | +91 9900531463
Website http://exotel.in/ | We’re Hiring http://exotel.in/about/careers/ |
Twitter https://twitter.com/balaramanl | LinkedIn
https://in.linkedin.com/in/balaramanl

On Mon, Jul 25, 2016 at 11:10 PM, Jerjou notifications@github.com wrote:

To be fair, I didn't understand the audio file either :-)

I think at least one of the main issues here is that it wasn't obvious
that the api looks for english (US) by default if you don't specify the
language. I'm updating the sample to explicitly state the language, and add
a link to the config docs so you can look up other languages you can
specify.

I realize you mentioned the audio wasn't in English, but there were
isolated english words within the audio. That's a little bit tougher for
the API to pick out. If you think of how humans process audio, oftentimes
you'll recognize a word not just based on its own sound, but on the words
and context that surround it. For example, "year" and "ear" sound pretty
similar, and you mostly distinguish them by context. Machines also still
aren't as good as humans at recognizing speech, so they rely more heavily
on surrounding words than humans do.

Re: flac / mp3. The speech_rest.py (and all the files in this repo,
really) is meant more as code to be read and learned from, than utilities
in and of themselves. If you take a look in the file
https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/speech/api/speech_rest.py#L63,
you'll notice that the script hard-codes a specific audio encoding of the
audio file (ie uncompressed 16-bit signed little-endian
https://cloud.google.com/speech/reference/rest/Shared.Types/AudioEncoding).
You can change that to specify FLAC, to process the FLAC files, but the API does
not support
https://cloud.google.com/speech/reference/rest/Shared.Types/AudioEncoding
mp3 at this time. If you'd like to process mp3 files, you'll have to decode
them to a supported encoding first, although recognition accuracy may be
worse https://cloud.google.com/speech/docs/best-practices than if you
had an audio file that hadn't been through lossy compression.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#424 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/APKwsAyeEkWBvwexZXzEgSyHc62tjB5Pks5qZPUMgaJpZM4JOawa
.

[puneith]

@BalaExotel Thanks for your comment. I checked the audio and see that its in Hindi. Adding further, we were able to get a non-null transcription when the language hindi was specified ( not perfect though because I hear a lot of noise in the audio). But, the challenge is that when we set the language to hi-IN (see here https://cloud.google.com/speech/docs/best-practices#language_support) we will not get the mix of hindi and english as pointed earlier by @jerjou . We will push the change soon in the sample code and then you will be able to specify the language to see the results yourself.

[jerjou]

@BalaExotel you might try using some higher-quality audio (eg less background noise) to see what results you can get, but I wouldn't rely on it at this time.