suggestion: NodeJS module #4
Comments
|
I think it will be a good idea if we create examples of using Snowboy for different programing languages. I'll add some C++ examples first. |
|
@amir-s any progress on the NodeJS module? Do you need additional explanations regarding the C++ example? Happy to explain more. |
|
@chenguoguo Perfect. Yes, I'm working on it, but I'm quite slow ... The rest is just some modification to the C++ example you provided. Here is an article about gyp. It is much easier to deal with different operating systems and conditions ... |
|
Hey @amir-s, any progress on this? I'd be happy to take a look, but I'd like to avoid doing duplicate work. |
|
@amir-s Hey Amir, any progress on this? Shall we get help from @evancohen? It will be great if snowboy runs on node at some point :-) |
|
Looks like @amir-s is busy with other stuff recently. @evancohen if you could help us create the wrapper and send a pull request to our repo, that will be great too! I'm quite excited about the NodeJS wrapper. |
|
@chenguoguo @evancohen Sorry guys, for some reasons i didn't get notified about this thread... |
|
No worries @amir-s, I can try to pick this up :) Were you able to make any progress with @chenguoguo the trick, as I see it, is to get that bindings file to generate something that maps to what already exists in the example Makefile and demo.mk, from there everything else should be really straightforward. I'll take a crack at this either tomorrow or Wednesday and report back 🚀 |
|
:gets popcorn: |
|
Is this the proper approach for the module?: https://nodejs.org/api/addons.html#addons_function_factory Thank you folks for the hard work. |
|
I've made some progress on this here: evancohen@56f59a0 One major drawback that I found with using @illperipherals it really depends on how you want to use the it and where you want the heavy lifting to be. In this case I'm using as much C++ code as possible, and writing a dev friendly layer in JavaScript. |
|
Hi, I'm maker of SARAH (http://sarah.encausse.net) using NodeJS Server and Microsoft Speech Engine client. SARAH is powerd by 4500 geek looking for a RPi client for SARAH. So I made A NodeJS experimental version on top of SoX: https://github.com/JpEncausse/SARAH-Client-NodeJS
I sucks in C :-) is there a way to send the audio buffer to SnowBoy throught precompiled CLI ? Thanks ! |
|
Regarding "send the audio buffer to SnowBoy through precompiled CLI ", this might be possible to modify the demo.cc file. Instead of capturing the audio with PortAudio, you modify the demo.cc file to load the data from stdin, and you can use some bash command like Regarding NodeJS, @evancohen might have something that you can play with? |
|
@evancohen, not sure how far you got, but I got curious. $ cd snowboy
$ cd node
$ npm installAnd it complained about pi@alphared-agent-virtual:~/Tinkering/snowboy/node$ npm install
npm WARN package.json [email protected] No README data
> [email protected] install /home/pi/Tinkering/snowboy/node
> node-gyp rebuild
make: Entering directory '/home/pi/Tinkering/snowboy/node/build'
CXX(target) Release/obj.target/snowboy/snowboy.o
../snowboy.cc:8:27: fatal error: pa_ringbuffer.h: No such file or directory
#include <pa_ringbuffer.h>
^
compilation terminated.
snowboy.target.mk:88: recipe for target 'Release/obj.target/snowboy/snowboy.o' failed
make: *** [Release/obj.target/snowboy/snowboy.o] Error 1
make: Leaving directory '/home/pi/Tinkering/snowboy/node/build'Version info: I didn't see an issues item on your repo, so I'm posting this here, let me know if I should put this somewhere else. |
|
Hey @nomadic-squirrel you have to run the Sorry for taking so long on this, I've been extra slammed with work. Hopefully I can finish it this weekend 😄 |
|
Ok, closer. I think. I am running on a Debian vm in the hopes that I can do some development while I travel.. yea, not so sure that is a great idea, but I'm making some progress here and there. which I"m pretty sure is this one: https://github.com/fivdi/onoff/wiki/Node.js-v0.10.29-and-native-addons-on-the-Raspberry-Pi So I might need to install node from source. I'll give that a shot if I have some time next week. |
|
Depending on the arch of your Pi you should be able to follow the instructions here to upgrade node: If you are using a first gen Pi (revision number starts with |
|
Ok, progress. Again, this is a Debian VM: $ uname -a
Linux alphared-agent-virtual 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2 (2016-04-08) x86_64 GNU/LinuxThat I'm playing with before I get to the pi, so the pi will likely be different. But after running the install instructions for nodesource I was able to successfully run Just poking around, I fired up node to see what it would do. What's this call to $ ~/Tinkering/snowboy/node$ node
>
> var snowboy = require( './snowboy' );
TypeError: snowboy.pow is not a function
at Object.<anonymous> (/home/pi/Tinkering/snowboy/node/snowboy.js:79:21)
at Module._compile (module.js:541:32)
at Object.Module._extensions..js (module.js:550:10)
at Module.load (module.js:458:32)
at tryModuleLoad (module.js:417:12)
at Function.Module._load (module.js:409:3)
at Module.require (module.js:468:17)
at require (internal/module.js:20:19)
at repl:1:15
at REPLServer.defaultEval (repl.js:272:27)
> The last line is this: console.log(snowboy.pow(4, 2)); // Prints '16' Otherwise, it looks pretty straightforward. I'll need to get a mic, but I'll post back if/when I make further progress. Great work!! |
|
Like I said in my previous post, not everything is wired up yet. |
|
Hey everyone, it's been a while! I finally got my act together and got the Nan module working. You can find the code and instructions here: https://github.com/evancohen/snowboy/tree/devel/swig/Node A few caveats:
I'll be sending a PR once I iron out a few more kinks and finalize the wrapper interface. 🚀 |
|
Awesome! @evancohen |
|
Awesome, I will test this once it works also on Linux / RPi3 |
|
Still plugging along! I hit some unexpected hurdles when trying to publish to npm, mainly using node-pre-gyp to build precompiled native node binaries for people who don't want to build the plugin from scratch. This should make things significantly easier for everyone using the node library. While I sort out the rest of the build issues, if you are on an Mac, you can test out a prebuilt binary by running the following in your existing node project: Note: this still requires A prebuilt node binary will be installed, if for some reason the binary can't be found then it will fallback to building the project (which will fail because it's not quite configured correctly). If it does download the binary, you can do something like: var snowboy = require('snowboy');
snowboy.addModel("path/to/resources/snowboy.umdl");
// Start detection and pass in a function as our callback for when we detect the keyword
snowboy.detect(function(data){
console.log("Example Detected: ", data);
});
// After 2 seconds check if snowboy is listening
setTimeout(function() {
console.log("Listening: ", snowboy.isListening());
}, 2000);
// After 3 seconds of running, terminate snowboy
setTimeout(function() {
snowboy.stop()
console.log("Listening: ", snowboy.isListening());
}, 3000);It's a bit chatty right now and logs a bit too much (something I plan on parameterizing). But it does work:
tl;dr I've made a lot of progress Todo:
If there there are any specific features that people would like, or if you have a preference about how the library is called, I'm totally open to feedback 😃 |
|
Hi there, I just made a 1:1 nodejs wrapper and it also provide a WritableStream interface. You can find the code here : https://github.com/heXeo/snowboy/tree/node Also an npm module is publish : Exampleconst record = require('node-record-lpcm16');
const SnowboyDetect = require('snowboy-detect');
const d = new SnowboyDetect({
resource: "node_modules/snowboy-detect/resources/common.res",
model: "node_modules/snowboy-detect/resources/snowboy.umdl",
sensitivity: "0.5",
audioGain: 2.0
});
d.on('silence', function () {
console.log('silence');
});
d.on('noise', function () {
console.log('noise');
});
d.on('error', function () {
console.log('error');
});
d.on('hotword', function (index) {
console.log('hotword', index);
});
const r = record.start({
threshold: 0,
verbose: true
});
r.pipe(d); |
|
@nekuz0r just looking over the code now, awesome work! I'd be interested in combining what we've done and sending a PR, if that's alright with you. I'll take a stab at it tonight/over the weekend and get it building for other platforms. |
|
nice work! I will also take a look into it. |
|
Hi @evancohen and @nekuz0r, Also I have some questions, so the difference between your both modules seems to be that @nekuz0r module needs an audio stream and @evancohen uses portaudio to get the audio. I personally prefer to have the audio stream in my hands as it is not possible to get it from snowboy ( using portaudio ), or am I wrong? @nekuz0r your Node Interface does not support multiple Hotwords for one detection, it would be great to have this opportunity. |
|
@Mexxxo What do you mean by "multiple hotwords for one detection". It maps exactly the C++ API, you can specify multiples models and it will returns the index of the detected keywords. Or am i missing something ? |
|
@Mexxxo Indeed since the implementation is just a wrapper around the C++ API, you need to feed it with audio data. I only added writable stream to it for convenience. I greatly prefer to control the audio source, so you are able the feed it with whatever audio source you want. Can be a file, audio data from an external software, from a web stream, from WebAudio API, ... it will simply works. |
|
@Mexxxo, As @nekuz0r mentioned it's a direct mapping for the wrapper, so you can add multiple models by doing something like: const d = new SnowboyDetect({
resource: "somecommon.res",
model: "firstmodel.pmdl,secondmodel.pmdl",
sensitivity: "0.5,0.4",
audioGain: 2.0
});Totally agreed that using a stream is superior to portaudio, it makes the interface much more flexible (although it does add a small layer of complexity for the end user, maybe @nekuz0r can provide some sample code for the examples mentioned above?). One of the modifications to the interface that I'm thinking about are some functions for adding and removing models. I'm also debating about weather or not it would be beneficial to optionally pass in a callback along with the model for when that specific keyword is spotted (in addition to the event emitter) to make it simpler to specify actions when a given keyword is spotted. For example: function optionalCallback(){
// Call me when firstmodel.pmdl this is spotted.
}
d.addModel("firstmodel.pmdl", "0.5", optionalCallback); |
|
@evancohen your Maybe it would be nice to have a const models = new HotwordModels();
models.add({ file: 'firstmodel.umdl', sensitivity: '0.5', count: 2);
models.add({ file: 'secondmodel.pmdl', sensitivity: '0.5');
const detector = new SnowboyDetect({
resource: "common.res",
models: models,
gain: 1.0
});The We could go event further and give the ability to specify hotwords mapping for each model, so we could provide in the // For universal model
models.add({
file: 'firstmodel.umdl',
sensitivity: '0.5',
hotwords: [
'hello', 'world'
]
});
// For personal model
models.add({
file: 'secondmodel.pmdl',
sensitivity: '0.5',
hotwords: 'foo'
});
Let me know what are your thoughts on it :) |
|
Yeah, that sounds awesome! I took a crack at that in my branch and updated the example to reflect the updated structure. I haven't updated the npm package yet. Take a look and let me know what you think: evancohen@9409cd7
I'm a bit confused by this, do you mean that there can be multiple words in a "single" hotword? @chenguoguo can you confirm that universal models can have multiple hotwords? I checked I still haven't looked at removing models (it might be nice to do this dynamically - automatically detecting the removal/addition of a model on the fly and restarting the |
|
On Mon, Sep 12, 2016 at 10:37 PM, Evan Cohen [email protected]
Yes universal model can have multiple hotwords, although the provided
|
|
@evancohen Hi, this is what i had in mind https://github.com/heXeo/snowboy/blob/typescript/lib/node/index.ts The implementation handles UMDL with multiple hotwords. I rewrote it in typescript, i still have to set up npm |
|
@nekuz0r Awesome job! 😄 I was thinking of rewriting in TypeScript this evening, you beat me to it! Let me know once you get After that I'll send a PR to @chenguoguo there are a few build configuration things you'll have to deal with on your end. I'll include that info in the PR. In the meantime, there's an issue with versioning that we can address: The
Consistent version numbering might not even be something that you care about, but if you do we could bump up both the npm package and the git repo to 1.1.0 (I think adding a new supported language justifies a minor version bump, yes?). Also, I'm assuming that you'll want at least some documentation to go along with this? |
|
Awesome! Yes sure we can bump the version up to 1.1.0, we can certainly consider it as "substantial new functionality" :-) And yes we definitely would like to see some documentation for it, perhaps in README? |
|
The Also:
I have merged It needs some documentation, and then i think it will be ready for PR. Also, i will unpublish my |
|
Great! I'll finish up the documentation I was writing and send the PR. I'm going to hold off on publishing v1.1.0 until the PR is actually accepted in case we need to make any changes. |
|
@evancohen @nekuz0r great work! |
|
Hi guys, really great work on this, but still can't get it working, as I'm getting this error (and yes im using strict in my example.js :) class SnowboyDetect extends stream.Writable { SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode |
|
for reference, adding "use strict" to the top of index.js made everything work like a charm though :) |
|
@WarlaxZ Which version did you use ? |
|
I edited my previous answer, because |
|
Closing this after #52 |
Hello and thanks for the awesome job.
I was trying to create an NPM module for snowboy, but I kind of lost.
So, it might be easier to work together. If anyone could write simple C++ file to use snowboy, I can help with wrapping it as a javascript module and putting it in npm.
Or if anyone wants to go all the way, here are the directions to create c++ addons: https://nodejs.org/api/addons.html
Thanks
The text was updated successfully, but these errors were encountered: