[Speech] speech_streaming.py code does not worl

[heroddaji]

Hello,

I am testing the new google cloud speech api on raspberry-pi2, I have access to the beta and have the json key ready.

I can run the file speech_rest.py and get the result, like this:

{"responses": [{"results": [{"alternatives": [{"confidence": 0.98304343, "transcript": "the rain in Spain stays mainly on the plain"}], "isFinal": true}]}]}

But for the streaming version (we need this for our product), I encountered error.
I follow all the requirement by install package gcloud, grpcio, PyAudio, grpc-google-cloud-speech. All of the packages are installed successfully. But when I run the example:

$python speech_streaming.py

I got this error:

Traceback (most recent call last):
  File "speech_streaming.py", line 7, in <module>
    from google.cloud.speech.v1.cloud_speech_pb2 import *  # noqa
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/speech/v1/cloud_speech_pb2.py", line 490, in <module>
    from grpc.beta import implementations as beta_implementations
  File "/usr/local/lib/python2.7/dist-packages/grpc/beta/implementations.py", line 38, in <module>
    from grpc._adapter import _intermediary_low
  File "/usr/local/lib/python2.7/dist-packages/grpc/_adapter/_intermediary_low.py", line 57, in <module>
    from grpc._adapter import _low
  File "/usr/local/lib/python2.7/dist-packages/grpc/_adapter/_low.py", line 33, in <module>
    from grpc._cython import cygrpc
ImportError: /usr/local/lib/python2.7/dist-packages/grpc/_cython/cygrpc.so: undefined symbol: x25519_NEON

I don't know how to solve this, I would appreciate some help here.

[theacodes]

@jerjou can you take a look here?

[jerjou]

Hm... looks like a grpc on ARM thing. A quick search turns up this bug, which seems to indicate that grpc has a fix. Try replacing grpcio==0.14.0rc1 in requirements-speech_streaming.txt with git+https://github.com/grpc/grpc.git#egg=grpcio and do the pip install -r requirements-speech_streaming.txt again (you might have to pip uninstall grpcio first).

Let me know how that goes.

[heroddaji]

thanks for your help,
i try it, now i got this error

In file included from ./src/core/ext/lb_policy/grpclb/load_balancer_api.h:40:0,

                 from src/core/ext/lb_policy/grpclb/load_balancer_api.c:34:

./src/core/ext/lb_policy/grpclb/proto/grpc/lb/v1/load_balancer.pb.h:38:35: fatal error: third_party/nanopb/pb.h: No such file or directory

quick google search shows me this link: grpc/grpc#6083
seem like another problem in grpc.

Can you give me more advices?

[heroddaji]

This is my new requirement file

gcloud
git+https://github.com/grpc/grpc.git#egg=grpcio
PyAudio
grpc-google-cloud-speech

Then I go to the grpcio src folder, issue command:
#git submodule update --init

After that, I issue command:
#pip install -r requirements-speech_streaming.txt

And this is the error:

In file included from third_party/boringssl/crypto/asn1/tasn_dec.c:66:0:
third_party/boringssl/crypto/asn1/../internal.h:121:22: fatal error: stdalign.h: No such file or directory
compilation terminated.

Any advices?

[heroddaji]

anyone?

[httpdss]

@heroddaji found the solution: make sure you are using gcc 4.7

to make gcc 4.7 the default compiler con raspbian follow this link:

http://www.rpiblog.com/2014/07/installing-gcc-on-raspberry-pi.html

hope this helps!

[heroddaji]

Thank you, i will try that and let you know the result soon

[heroddaji]

Finally can compile all the dependencies for the speech_streaming.py, thank you @httpdss
The next issue is... I have errors when running the speech_streaming.py

So what I did is plugin the camera with mic build-in, and talk after I execute the program.
I ran it on my raspberry pi3, and this is the error. ( i believe I have all the alsa driver install)

Pls help me again, thank you all

root@Dpi3:~/dai/python-docs-samples/speech/api$ python speech_streaming.py
I0530 14:28:08.343444245 3145 socket_utils_common_posix.c:161] Disabling AF_INET6 sockets because socket() failed.
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.front.0:CARD=0'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM front
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround40.0:CARD=0'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM surround40
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM surround41
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM surround50
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM surround51
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround71.0:CARD=0'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM surround71
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM iec958
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4241:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4720:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_dmix.c:957:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
Expression 'SetApproximateSampleRate( pcm, hwParams, sr )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1968
Expression 'PaAlsaStreamComponent_InitialConfigure( &self->capture, inParams, self->primeBuffers, hwParamsCapture, &realSr )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2643
Expression 'PaAlsaStream_Configure( stream, inputParameters, outputParameters, sampleRate, framesPerBuffer, &inputLatency, &outputLatency, &hostBufferSizeMode )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2767
Traceback (most recent call last):
File "speech_streaming.py", line 145, in
main()
File "speech_streaming.py", line 137, in main
service.Recognize(request_stream(stop_audio), DEADLINE_SECS))
File "speech_streaming.py", line 114, in listen_print_loop
for resp in recognize_stream:
File "/usr/local/lib/python2.7/dist-packages/grpc/framework/crust/_control.py", line 415, in next
raise self._termination.abortion_error
grpc.framework.interfaces.face.face.LocalError: LocalError(code=None, details="None")
E0530 14:28:08.480472131 3135 handshake.c:215] Read failed.

[heroddaji]

Also, when I run the code on MacOS, I got this error message (it hangs for a while before showing the error message)

2016-05-30 14:34:30.918 Python[50584:1012967] 14:34:30.918 WARNING: 140: This application, or a library it uses, is using the deprecated Carbon Component Manager for hosting Audio Units. Support for this will be removed in a future release. Also, this makes the host incompatible with version 3 audio units. Please transition to the API's in AudioComponent.h.

Traceback (most recent call last):
File "speech_streaming.py", line 145, in
main()
File "speech_streaming.py", line 137, in main
service.Recognize(request_stream(stop_audio), DEADLINE_SECS))
File "speech_streaming.py", line 114, in listen_print_loop
for resp in recognize_stream:
File "/Library/Python/2.7/site-packages/grpc/framework/crust/_control.py", line 425, in next
self._condition.wait()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 340, in wait
waiter.acquire()

[heroddaji]

After fixing the audio driver, I got a new error, the special part is google error. Any help?

Traceback (most recent call last):
File "speech_streaming.py", line 145, in
main()
File "speech_streaming.py", line 137, in main
service.Recognize(request_stream(stop_audio), DEADLINE_SECS))
File "speech_streaming.py", line 114, in listen_print_loop
for resp in recognize_stream:
File "/usr/local/lib/python2.7/dist-packages/grpc/framework/crust/_control.py", line 415, in next
raise self._termination.abortion_error
grpc.framework.interfaces.face.face.LocalError: LocalError(code=None, details="None")
E0530 20:59:11.030871346 3377 tcp_client_posix.c:191] failed to connect to 'ipv4:216.58.197.138:443': timeout occurred

[httpdss]

@heroddaji im suffering the same issue. battled with it all weekend, but i didnt get to the solution. if you have any info, please let me know. thx

[heroddaji]

Any Google employee can help?
because my speech api trial is expired, that why I have the problem of connecting to google server?

grpc.framework.interfaces.face.face.LocalError: LocalError(code=None, details="None")
E0530 20:59:11.030871346 3377 tcp_client_posix.c:191] failed to connect to 'ipv4:216.58.197.138:443': timeout occurred

[heroddaji]

So at this moment, all the test file work, my audio device also working fine. Nothing wrong.
But when I tried the "speech_streaming.py", I always receive this error:

Pls give some feedback

Traceback (most recent call last):
File "speech_streaming.py", line 145, in
main()
File "speech_streaming.py", line 137, in main
service.Recognize(request_stream(stop_audio), DEADLINE_SECS))
File "speech_streaming.py", line 114, in listen_print_loop
for resp in recognize_stream:
File "/usr/local/lib/python2.7/dist-packages/grpc/framework/crust/_control.py", line 415, in next
raise self._termination.abortion_error
grpc.framework.interfaces.face.face.LocalError: LocalError(code=None, details="None")

[jerjou]

Sorry - I've been at a conference these past couple of days..

Yeah - the LocalError is thrown when there's some error within one of the threads that the grpc client library spins up for the streaming process. For some reason the grpc library swallows the original exception, so it's not shown in the stack trace - and instead you get the generic LocalError.

I've been getting around it using brute force, adding debug statements variously around my code to try to isolate where the exception is thrown, and what the problem is. I would suggest running the different components of the code in isolation, to try to figure out what is going wrong, and where.

[heroddaji]

well, this is not my code, this is the google sample code...
anyway, i will try to add logging everywhere

[jerjou]

Right - but it's still helpful to know where exactly the problem is occurring. For example, perhaps there's a permissions problem accessing the audio interface, or perhaps the pyAudio library doesn't have support for some device or platform, or perhaps there's some assumption I made when writing the sample that isn't valid on your setup.

Whatever it is, we need to figure out where the problem is occurring and why, in order to start to try to fix it.

[httpdss]

@heroddaji got it working with some exception handling and modifying RATE and CHUNKS. check if it works for you too:

https://gist.github.com/httpdss/c116b4b3a056e350d9f4ed4ae6325270

[heroddaji]

Thank you @httpdss .
Can you show me the output result?

I follow your code and test on my rpi2 again.
I get some error like this during the rec
[Errno -9981] Input overflowed
within this area (i added the exception handling)

while not stop_audio.is_set():
            print i;
            try:
                data = audio_stream.read(chunk)
                if not data:
                    raise StopIteration()
                # Subsequent requests can all just have the content
                audio_request = cloud_speech.AudioRequest(content=data)
                yield cloud_speech.RecognizeRequest(
                    audio_request=audio_request)
                i += 1
            except Exception as inst:
                print type(inst)
                print type(inst.args)
                print inst

[heroddaji]

ok, so I got the code to work on my mac machine (still has input overflowed randomly)
so the code is not really stable.

on the rpi, i still have the input overflowed, it might the rpi limitation, i'm not sure.
I read on stackoverflow, that to avoid the input overflow error, use a callback when reading audio chunk from pyaudio, i will try that.

If you have code using callback, pls let me know as well

[httpdss]

@heroddaji dont have any callback example. one thing i forgot to mention is that playing arround with rate, channels and chunks was also part of the solution cuz i was getting a message that i wasnt streaming the data in realtime or something like that. check which is your mic rate using arecord and change the params based on that info.

[heroddaji]

Thx, any way some time it works, some time it shows error, and on rpi it seems the combination of
(audio driver problem on pi) + (unstable google sample result) = nothing work for me.
My boss got fed up and ask me to switch to use vision api for another project.

So to recap:
this problem somewhat has been solved, you can close it.

For other:
I think the most frustrating for me is to install the components of streaming requirement.
So thx to @httpdss , make sure you install gcc-4.7

Thx for all the help