microphone input stream

[staminna]

Hi

Why isn't there streaming support?
I am looking at https://github.com/gillesdemey/node-record-lpcm16

Thanks

[Hgergely]

You can do it for yourself. You can start a sub-process in node with
"arecord" and pipe it instead of file input
I have done it. it worked.

On 21 July 2016 at 17:43, Jorge notifications@github.com wrote:

Hi

Why isn't there streaming support?
I am looking at https://github.com/gillesdemey/node-record-lpcm16

Thanks

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#157,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFyOcZEfv4rKSdUujKVNmZAvHd5q1Hm2ks5qX6GugaJpZM4JR_j9
.

[jmdobry]

@staminna Which sample are you talking about?

[Hgergely]

Iam sure he is talking about streaming.js

On Thu, 21 Jul 2016, 17:51 Jason Dobry, notifications@github.com wrote:

@staminna https://github.com/staminna Which sample are you talking
about?

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#157 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFyOcXO4d-AD7jYsRJB84kyAPdrcIrNuks5qX6OPgaJpZM4JR_j9
.

[jmdobry]

Then I don't understand the question. speech/recognize_streaming.js is an example of using the Speech API's bi-directional streaming gRPC method. streamingRecognize() returns a duplex stream object.

[Hgergely]

This guy wants an example of how it can be used with a hardware audio input
( microphone) instead of audio file. I understand the question but I think
he must understand : Firstly that this topic is not about all ways how the
api can be used. Secondly this nodejs example is environment independent
and cannot provide a environment dependent solution like microphone usage.
Microphone is used totally different ways depending on the op system under
node.

On Thu, 21 Jul 2016, 18:29 Jason Dobry, notifications@github.com wrote:

Then I don't understand the question. speech/recognize_streaming.js is an
example of using the Speech API's bi-directional streaming gRPC method.
streamingRecognize() returns a duplex stream object.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#157 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFyOcSN7NokBw5tyXnD4JlTqQ6rE4dAHks5qX6yBgaJpZM4JR_j9
.

[jmdobry]

Makes sense. I think the one of the biggest takeaways from the sample is that streamingRecognize() returns a duplex stream object, which means that one can do the following (borrowing from the sample):

var grpcStream = speechService.streamingRecognize();
grpcStream.write(streamingRecognizeRequest);

readableStream
  .pipe(toRecognizeRequest)
  .pipe(grpcStream);

where readableStream is any Node.js readable stream, e.g. file stream, http stream, recording stream from node-record-lpcm16, or whatever.

[Hgergely]

Yeas thats what I did. Simply replaced the readablestream with a audio
input stream provided by linux arecord data. It works like charm but node
record lmpcm16 can be an even better choice. Thanks Jason.

On Thu, 21 Jul 2016, 19:53 Jason Dobry, notifications@github.com wrote:

Makes sense. I think the one of the biggest takeaways from the sample that
we have is that streamingRecognize() returns a duplex stream object,
which means that one can do the following (borrowing from the sample):

var grpcStream = speechService.streamingRecognize();grpcStream.write(streamingRecognizeRequest);

readableStream
.pipe(toRecognizeRequest)
.pipe(grpcStream);

where readableStream is any Node.js readable stream, e.g. file stream,
http stream, recording stream from node-record-lpcm16, or whatever.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#157 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFyOcU1bNA02ye2ifBD3YvqbzI9q2Nblks5qX8A3gaJpZM4JR_j9
.

[staminna]

@jmdobry Where should this go?
I keep getting grpcStream.write(streamingRecognizeRequest);
ReferenceError: streamingRecognizeRequest is not defined

[jmdobry]

Can you show all of your code?

[staminna]

http://pastebin.com/puXvAr5r

[jmdobry]

@staminna Here's a more concise example you can try. Save it to a file and run it. You don't have to pass any arguments to the file.

// Copyright 2016, Google, Inc.
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//    http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

'use strict';

var async = require('async');
var fs = require('fs');
var path = require('path');
var grpc = require('grpc');
var googleProtoFiles = require('google-proto-files');
var googleAuth = require('google-auto-auth');
var Transform = require('stream').Transform;
var record = require('node-record-lpcm16');

var PROTO_ROOT_DIR = googleProtoFiles('..');
var protoDescriptor = grpc.load({
  root: PROTO_ROOT_DIR,
  file: path.relative(PROTO_ROOT_DIR, googleProtoFiles.speech.v1beta1)
}, 'proto', {
  binaryAsBase64: true,
  convertFieldsToCamelCase: true
});
var speechProto = protoDescriptor.google.cloud.speech.v1beta1;

function getSpeechService (callback) {
  var googleAuthClient = googleAuth({
    scopes: [
      'https://www.googleapis.com/auth/cloud-platform'
    ]
  });

  googleAuthClient.getAuthClient(function (err, authClient) {
    if (err) {
      return callback(err);
    }

    var credentials = grpc.credentials.combineChannelCredentials(
      grpc.credentials.createSsl(),
      grpc.credentials.createFromGoogleCredential(authClient)
    );

    console.log('Loading speech service...');
    var stub = new speechProto.Speech('speech.googleapis.com', credentials);
    return callback(null, stub);
  });
}

async.waterfall([
  function (cb) {
    getSpeechService(cb);
  },
  function sendRequest (speechService, cb) {
    console.log('Analyzing speech...');
    var call = speechService.streamingRecognize()
      .on('error', cb)
      .on('data', function (recognizeResponse) {
        console.log(JSON.stringify(recognizeResponse, null, 2));
      })
      .on('end', cb);

    // Write the initial recognize reqeust
    call.write({
      streamingConfig: {
        config: {
          encoding: 'LINEAR16',
          sampleRate: 16000
        },
        interimResults: true,
        singleUtterance: false
      }
    });

    var toRecognizeRequest = new Transform({ objectMode: true });
    toRecognizeRequest._transform = function (chunk, encoding, done) {
      done(null, {
        audioContent: chunk
      });
    };

    // Start recording
    var readableMicrophoneStream = record.start({
      sampleRate : 16000,
      verbose : true
    });

    // Stream the microphone audio to the Speech API
    readableMicrophoneStream
      .pipe(toRecognizeRequest)
      .pipe(call);
  }
], function (err) {
  if (err) {
    return console.error(err);
  }
  console.log('DONE');
});

[staminna]

I can't seem to make this loop once JSON returns isFinal true. Can someone help me with this? I should be able to continue from there.

[jmdobry]

I'm not sure I understand. Perhaps use recursion to restart your stream?

[staminna]

I had all sorts of problems with recursive calls exceeding memory on using node-async-loop. This example only listens once, and exits node when it's done ( only once). I would like it to keep listening at all times and not exit once is 'DONE'.

[jmdobry]

I added a microphone streaming sample: https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/master/speech/recognize.js#L91

[linhdoha]

@jmdobry can you explain me what problems did I met?

C:\Users\Linhdoha\nodejs-docs-samples\speech>node recognize listen
Listening, press Ctrl+C to stop.
events.js:160
throw er; // Unhandled 'error' event
^

Error: spawn rec ENOENT
at exports._errnoException (util.js:1026:11)
at Process.ChildProcess._handle.onexit (internal/child_process.js:193:32)
at onErrorNT (internal/child_process.js:359:16)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickCallback (internal/process/next_tick.js:98:9)
at Module.runMain (module.js:606:11)
at run (bootstrap_node.js:394:7)
at startup (bootstrap_node.js:149:9)
at bootstrap_node.js:509:3

C:\Users\Linhdoha\nodejs-docs-samples\speech>

node recognize sync worked fine
node recognize async worked fine

[jmdobry]

It seems that the node-record-lpcm16 struggles on Windows. There is an outstanding issue: gillesdemey/node-record-lpcm16#8

[freund17]

I think I found a fix for it:
See gillesdemey/node-record-lpcm16#8

[endoplasmic]

Just curious if something has changed on this. I'm new to the google cloud API stuff, but I am looking for a speech-to-text solution using a microphone (or even just passing audio samples from another source).

I can't seem to get the mic sample to output interim results (if I set interimResults or interim_results to true, I don't get any response at all, just hangs). If I run the sample as-is I see the events for START_OF_SPEECH, END_OF_SPEECH and ENDPOINTER_EVENT_UNSPECIFIED (which contains the entire string)

Node version is 6.9.1 and the speech library is 0.8.0. Running on macOS 10.12.4

Thanks for any help in advance, looking forward to getting this up and running and trying something other than Houndify.

edit:

Alright, I got it working, the error was that the config object doesn't hold all of the config and that interimResults needs to be a sibling to config in the request object, like so:

const request = {
  config: {
    encoding: encoding,
    sampleRate: sampleRate
  },
  interimResults: true,
};

[Nathan4KImpact]

Hello everybody,

When I try the example of @jmdobry above,

I too, get no error, but no text as response, just that

Loading speech service... edit.js:189 Analyzing speech... E:\NathanTestsWS\simple-protocol-recorder\src\node_modules\node-record-lpcm16\index.js:73 Recording with sample rate 16000... E:\NathanTestsWS\simple-protocol-recorder\src\node_modules\node-record-lpcm16\index.js:77 Recording 5548 bytes E:\NathanTestsWS\simple-protocol-recorder\src\node_modules\node-record-lpcm16\index.js:81 End Recording: 289.128ms edit.js:193 { "error": null, "results": [], "resultIndex": 0, "endpointerType": "END_OF_AUDIO" } edit.js:236 DONE

Thanks for any help !