Speech recognition doesn't work under electron

[efunneko]

With a fresh checkout running under electron (npm start), annyang is not longer working. I am using a USB logitech webcam microphone for the input.

If I run the exact same set of files under chromium, the microphone and speech recognition work fine. Was there any configuration necessary under electron, alsa or asound to get electron to work?

[efunneko]

After playing around with it a bit more, it seems that if I run 'sudo npm start' instead of just 'npm start', the microphone will work.

[rsandstroem]

Thank you for that hint. I had the same problem with a Creative USB webcam, annyang reported unknown input once per second. Starting with sudo makes it work as intended.

[evancohen]

I don't think this is an issue with the mirror, it's more likely an issue with Electron. Having to run with sudo is kind of a bummer. I'm using a Logitech Webcam and it's working fine for me... I'll take a look at this and let you know if I figure something out.

[ghost]

I am unable to get it to speech to work. How do I set this up? I am not getting any prompts to choose a device on launch. When I record with my usb mic via command line, I get sound back.

[efunneko]

I found getting the mic to work with electron quite finicky. As mentioned above, I never got it to work without running with sudo. I also had issues starting electron in rc.local - it would start (running as root), but the mic still didn't work.

In the end, I start it as the 'pi' user from a '@reboot' in the 'pi' user's crontab file:

@reboot sudo /home/pi/smart-mirror/start &

with the start script being:

#!/bin/bash
export DISPLAY=:0
export XAUTHORITY=/home/pi/.Xauthority
cd /home/pi/smart-mirror && npm start

I also played around with my ~/.asoundrc file. I am not sure if it was necessary at all. Here is what is in my file:

pcm.!default {
        type hw
        card 0
}

ctl.!default {
        type hw
        card 0
}
pcm.default.card 1 

I did post questions to both stackoverflow and atom.io hoping for some info on this, but I have not heard anything back.

[evancohen]

I wasn't able to come to any conclusions when I looked at it over the weekend - although I have a sneaking suspicion that for some reason electron isn't picking up your microphone is the default (despite your configuration). It's possible to select which media device to use inside Electron, could you try using http://cl.ly/code/303S1R1o332n and seeing if your device gets picked up properly?

[efunneko]

Thanks - I will try it when I get home. If this can work, we could add a config to config.js that allows a hard-coded selection of the microphone.

[ghost]

If I can't get it to work on the default browser. How do you npm start it with chromium/ chrome? I have Chromium installed but, have not seen a way to get npm to start with that instead of default browser.

[ghost]

I did what efunneko did with:

pcm.!default {
        type hw
        card 0
}

ctl.!default {
        type hw
        card 0
}
pcm.default.card 1

But, I also had to go to /usr/share/alsa/alsa.conf and edit that file. I changed the defaults of the following to 1.

defaults.ctl.card 1
defaults.pcm.card 1

Only issue I saw so far (about 10 minutes of testing before bed), I was unable to do an image search. I was able to do maps.

[evancohen]

Seems like that's the solution then :)

If you still wanted to test device selection for getUserMedia use https://github.com/atom/electron-quick-start as a starting point and then drop in the contents of http://cl.ly/code/303S1R1o332n in index.html, that should work.

[ojrivera381]

Some help hear would be appreciated. Linux is definitely not my strongest attribute.
I have everything up and running, finally! With 1 exception my mic. I'm using an hp KQ245aa
Tested working using sudo arecord -D plughw:1,0 test.wav ran the file with aplay test.wav and confirmed audio.

I can't seem to get smart-mirror to listen for commands.

Since I am Linux stupid.

can some one give me a step by step as to what BrianDawson did or anything I can follow to get this up and running proper

Much appreciated and thank you in advance.

[rsandstroem]

I got the voice recognition to work without sudo finally.

I had already made the changes to alsa.conf and the .asoundrc discussed above, but in addition I also had swapped the ordering of the USB webcam and ALSA, so for me 0->1 and 1->0 compared to the solution above.

The problem was that I did not have "pcm.default.card 0" in my .asoundrc file. With that added, it works like charm!

[evancohen]

Nice job figuring this out folks! I've added a link to this issue in the readme for anyone else who has the same issue.

[evancohen]

Please follow the instructions above to sort this out. There are some variations to this solution (as it depends on which mic you are using) so read through the thread and see what works for you.

[evancohen]

@Keopss for new questions please file a new issue or ask on gitter, I want to keep this thread uncluttered so people who have this problem don't have to sift through a bunch of information to get what they need :)

[nandor690]

anyone else have a problem with losing sound out of the audio jack when mic is working. and when I edit the files the to get the sound to work I lose the mic again?

[nojkepop]

For those that are having audio issues. I'm not sure if this is a universal fix but this is what worked for me.

I ended up editing the /usr/share/alsa/alsa.conf file.
Basically run arecord --list-devices and find the card #/device #. For me I simply needed the card#.

In the /usr/share/alsa/alsa.conf file I then changed these lines
defaults.ctl.card #
defaults.cm.card #
to my corresponding card number of my input device.
I also deleted the # from this line (load card-specific configuration files (on request)).

My .asounrc file is now blank.

I was trying to also add Airplay functionality to my mirror and was unable to use the onboard or secondary usb output after following the steps in #20 and this resolved that issue for me.

I found most of this info online and on these forums:
http://superuser.com/questions/989385/alsa-base-conf-missing-in-new-raspberry-pi-raspbian-jesse
http://askubuntu.com/questions/429945/what-is-the-hardware-address-of-my-webcam-mic

[evancohen]

Awesome information @nojkepop thanks a ton!

[beto2909]

Does your mic still work?

[nojkepop]

Yes I have a USB microphone and USB speakers connected, these are two separate devices. I can Airplay audio to my Raspberry Pi using shairport-sync and still use voice commands on the Mirror.

[beto2909]

But the AirPlay is not running within the mirror? Is it just another application running in the R-Pi? I have my regular audio working and the mic works if I run "sudo npm start" , however if I try to play a regular audio file with JavaScript it won't play it. So I think is something that has to do with nodejs. I found this info about audio and npm, not sure if it will work.will test it when I get home. https://www.npmjs.com/package/speaker

[nojkepop]

You might be right, what you're trying to do is very different than my setup. I basically wanted the smart mirror to have access to the microphone while still being able to use my usb connected speakers. This setup wasn't possible with the .asroundrc setup.

I can try you're setup and see what happens but I feel like I read somewhere that the USB and headphone port are on the same bus(?) which prevents them both from being used at the same time as an audio source. The headphone jack also doesn't have that great of sound quality which is why there isn't much documentation on it. I think a lot of people are using audio add-on cards like the hifiberry or usb devices.

I will look some more into it and see what I can come up with. But you are simply wanting to be able to have voice commands and audio playback via the smart-mirror correct?

[beto2909]

yea the main reason i want to do it is because i am using a tv and i already have speakers in the tv and i dont want the speakers just to sit there, but it just doesn't make sense for me because the audio is going out in the hdmi but I might be wrong.

[nojkepop]

Have you tried anything like this which forces the audio out form HDMI to the 1.8inch jack?
https://www.raspberrypi.org/forums/viewtopic.php?f=91&t=40872

Maybe hop into https://gitter.im/evancohen/smart-mirror so we don't turn this into a giant thread. I can try to help you some more on there.

[evancohen]

This is the definitive answer to all of your problems (hopefully). There is no need to mess around with ALSA conf files. All you have to do is run:

sudo aptitude install pulseaudio

Then, optionally, if you want to force sound out of the headphone jack:

sudo modprobe snd_bcm2835
sudo amixer cset numid=3 1

Or, to get sound over HDMI:

sudo modprobe snd_bcm2835
sudo amixer cset numid=3 0

A reboot may be required after this.

[secdude]

Seems like you dug yourself through tons of *nix support forums in the past days...your effort is appreciated! Thanks evan. Will do a fresh jessie install on the weekend and maybe test this! If this proves stable it should enable merge of the search branch to dev-->master?

[3y3c0ugh]

There shouldn't be a specific time I install pulse is there? Like after node and the mirror are already installed? I did a fresh run of jessie, used those commands for HDMI, installed node, installed mirror, installed npm, then did a reboot. When i launch mirror, I still dont get sound out of HDMI, but it did pick up my mic without changing any options and without a sudo. I even tried forcing sound to HDMI from raspi-config and changing boot/confit.txt for hdmi_drive=2.

I'm going to try this the opposite way, installing everything first then doing pulse last.

[evancohen]

@secdude I'll merge the search branch in next week
@3y3c0ugh I installed PulseAudio last, but it technically shouldn't matter... The sound should come out of HDMI by default, so maybe running that command actually messed up something in your configuration?

I'll run through all of this with a fresh install again early next week and make sure everything works properly.

[beto2909]

Check that you .asoundrc is empty.

[Sachin1968]

I finally just reinstalled pulseaudio per evan's definitive answer...Voila!