Skip to content

Commit

Permalink
Trigger (#20) fixes #18
Browse files Browse the repository at this point in the history
* add trigger event so users can mantually start streaming detection

* simple markdown file for documentation. website coming soon(ish) 😄

* fix "unused" eslint error

* 0.1.2

* make index and keyword optional paramaters to trigger

* using int for hotword trigger

remove console.log

* 0.1.3
  • Loading branch information
evancohen authored Dec 27, 2016
1 parent d119eb6 commit 79587fa
Show file tree
Hide file tree
Showing 4 changed files with 180 additions and 3 deletions.
112 changes: 112 additions & 0 deletions docs/API.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
## API Methods

Paramaters marked in **bold** are required

### Require sonus and a cloud speech recognizer in your project:
``` javascript
const Sonus = require('sonus')
const speech = require('@google-cloud/speech')({
projectId: 'streaming-speech-sample',
keyFilename: './keyfile.json'
})
```
For more information about Google Cloud Speech see: https://cloud.google.com/speech/
Note: don't forget to enable billing!

### Custom hotwords
You can train and download custom hotwords for sonus from https://snowboy.kitt.ai
In order to initialize Sonus you need to pass in 1 or more hotwords.
Each hotword supports the following proporties:
**`file`** - The path to your hotword model (either pmdl or umdl)
**`hotword`** - The string that represents your hotword (ex: "sonus")
`sensitivity` - (default `'0.5'`) If you are getting a lot of false positives or are having trouble detecting your hotword adjusting this value shoud help

**Example:** (to be passed into the sonus constructor)
``` javascript
const hotwords = [
{file: '/mymodel.pmdl', hotword: 'sonus'},
{file: 'snowboy.umdl', hotword: 'snowboy'}]
```

### Languages
Sonus lets you customize the lenguage for streaming speech recognition. For details on supported lenguages see the docs for your streaming speech recognizer

**Example:** (to be passed into the sonus constructor)
``` javascript
const lenguage = "en-US"
```

### Initialize Sonus
Sonus's initialization accepts two paramaters:
**`options`** - an options object that contains your hotwords, lenguage, etc
- **`hotwords`** - an array of recognizable hotwords
- `lenguage` - streaming lenguage recognition
- `dictionary` - [TODO] only supported by some streaming recognizers
**`speechRecognizer`** - the speech recognizer of your choice

**Example:**
``` javascript
const sonus = Sonus.init({ hotwords, language }, speech)
```

### Start recognition
Pass your initialized sonus object into `Sonus.start`
**Example:**
``` javascript
Sonus.start(sonus)
```

### Pause recognition
Pass your initialized sonus object into `Sonus.pause`.
Pausing recognition while streaming will not cancel the request, instead it will cause it to simulate the "end" of speech and return final results.
**Example:**
``` javascript
Sonus.pause(sonus)
```

### Resume recognition
Pass your initialized sonus object into `Sonus.resume`
**Example:**
``` javascript
Sonus.resume(sonus)
```

### Stop recognition
If you want to stop recognition enterly you can use `Sonus.stop`
**Example:**
``` javascript
Sonus.stop(sonus)
```
Note that after recognition is stopped it can not be started again without creating an enterly new sonus instance.

### Trigger keyword/hotword manually
You can manuall trigger a hotword by passing your initialized sonus object and an index into `Sonus.trigger`
The indexes of your hotwords are base 1 and are deturmined by the order in which the hotwords are passed into `Sonus.init`

**Exceptions**
- `NOT_STARTED` - will be thrown if you have not started sonus when this is called.
- `INVALID_INDEX` - will be thrown if you pass an invalid index.

**Example:**
``` javascript
Sonus.trigger(sonus, 1)
```
sonus will be triggered with a hotword index of `1`

You can also optionally specify an index of `0` and an arbitrary hotword that will be returned in the `hotword` event
**Example:**
``` javascript
sonus.trigger(sonus, 0, 'some hotword')
```
sonus will be triggered with a hotword index of `1` and a hotword of `some hotword`

Passing a hotword with a valid index will override the hotword name and trigger that hotword
**Example:**
``` javascript
sonus.trigger(sonus, 1, 'override')
```
## Events
hotword
partial-result
final-result
error
42 changes: 42 additions & 0 deletions examples/trigger-example.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
'use strict'

const ROOT_DIR = __dirname + '/../'
const Sonus = require(ROOT_DIR + 'index.js')
const speech = require('@google-cloud/speech')({
projectId: 'streaming-speech-sample',
keyFilename: ROOT_DIR + 'keyfile.json'
})

const hotwords = [{ file: ROOT_DIR + 'resources/sonus.pmdl', hotword: 'sonus' }]
const language = "en-US"
const sonus = Sonus.init({ hotwords, language }, speech)

try{
Sonus.trigger(sonus, 1)
} catch (e) {
console.log('Triggering Sonus before starting it will throw the following exception:', e)
}

Sonus.start(sonus)

sonus.on('hotword', (index, keyword) => console.log("!" + keyword))

sonus.on('partial-result', result => console.log("Partial", result))

sonus.on('error', (error) => console.log(error))

sonus.on('final-result', result => {
console.log("Final", result)
if (result.includes("stop")) {
Sonus.stop()
}
})

try{
Sonus.trigger(sonus, 2)
} catch (e) {
console.log('Triggering Sonus with an invalid index will throw the following error:', e)
}

//Will use index 0 with a hotword of "triggered" and start streaming immedietly
Sonus.trigger(sonus, 0, "some hotword")
27 changes: 25 additions & 2 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ const record = require('node-record-lpcm16')
const stream = require('stream')
const {Detector, Models} = require('snowboy')

const ERROR = {
NOT_STARTED : "NOT_STARTED",
INVALID_INDEX : "INVALID_INDEX"
}

const CloudSpeechRecognizer = {}
CloudSpeechRecognizer.init = recognizer => {
const csr = new stream.Writable()
Expand Down Expand Up @@ -56,6 +61,7 @@ Sonus.init = (options, recognizer) => {
sonus = new stream.Writable(),
csr = CloudSpeechRecognizer.init(recognizer)
sonus.mic = {}
sonus.started = false

// If we don't have any hotwords passed in, add the default global model
opts.hotwords = opts.hotwords || [1]
Expand All @@ -80,8 +86,7 @@ Sonus.init = (options, recognizer) => {

// When a hotword is detected pipe the audio stream to speech detection
detector.on('hotword', (index, hotword) => {
sonus.emit('hotword', index, hotword)
CloudSpeechRecognizer.startStreaming(opts, sonus.mic, csr)
sonus.trigger(index, hotword)
})

csr.on('error', error => sonus.emit('error', { streamingError: error }))
Expand All @@ -97,6 +102,21 @@ Sonus.init = (options, recognizer) => {
}
}
})

sonus.trigger = (index, hotword) => {
if(sonus.started){
try{
let triggerHotword = (index == 0)? hotword : models.lookup(index)
sonus.emit('hotword', index, triggerHotword)
CloudSpeechRecognizer.startStreaming(opts, sonus.mic, csr)
} catch (e) {
throw ERROR.INVALID_INDEX
}
} else {
throw ERROR.NOT_STARTED
}
}

return sonus
}

Expand All @@ -107,8 +127,11 @@ Sonus.start = sonus => {
})

sonus.mic.pipe(sonus.detector)
sonus.started = true
}

Sonus.trigger = (sonus, index, hotword) => sonus.trigger(index, hotword)

Sonus.pause = sonus => sonus.mic.pause()

Sonus.resume = sonus => sonus.mic.resume()
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "sonus",
"version": "0.1.2",
"version": "0.1.3",
"description": "Open source cross platform decentralized always-on speech recognition framework",
"main": "index.js",
"scripts": {
Expand Down

0 comments on commit 79587fa

Please sign in to comment.