-
Notifications
You must be signed in to change notification settings - Fork 14
5. Actions
Here are some actions that Nala can currently do along with the query intents needed to activate them. If you have any other ideas, let us know on the github issues tab (as an enhancement)!
| Action | Description | Example query intent |
|---|---|---|
| ⏰ alarm.py | Plays an alarm sound (e.g. to wake up in morning) based on the time of day you specified during user registration. | “set alarm”, “stop alarm” |
| 🎵 chillout.py | Plays music in the background to help you calm down. | “chill out” |
| 🏟️ espn.py | Scrapes ESPN website to find any events going on later tonight. This is a work-in-progress. | “get sports” |
| 🎴 events.py | Scrape meetup.com for events in your local area. | “find events” |
| 📜 generate_poem.py | Generate a poem based on the generate poetry script we wrote a while back. | “make a poem” |
| 📝 grateful.py | Helps you keep a gratitude journal by recording a sample of what you’re grateful for today. | “be grateful” |
| Plays back a joke from the database. | “make a joke” | |
| 🌅 meditation.py | Guides you through a simple 60 second meditation. | “meditate” |
| 🎧 music.py | YouTube music links - based on a genre. | “play music” |
| 📰 news.py | searches some basic news sites related to computer science and machine learning (e.g. Hacker News). | “grab the news” |
| 🍐 nutrition.py | Searches for some healthy food nearby | “be healthy” |
| Schedules a trip in terms of AirBnB suggestions and flights | “plan trip” | |
| ↪️ reboot.py | Restarts the computer. | “restart” |
| 🔍 search.py | search bing with a query. | First query: “search”, Second query: [search term] |
| ⌛ shutdown.py | Shuts down the computer. | “shut down” |
| 😴 sleep | Puts the computer to sleep for a designated time period. | "go to sleep" |
| 👥 social.py | Alternative script to give you some suggestions based on your budget to go out later. | "be social" |
| 🌡️ weather.py | Searches weather.com for the current weather at your location. | "get the weather" |
| ☕🍦🎉🍺 yelp.py | Based on the query, searches yelp for coffee, restaurants, food, nightlife, ice cream, or bars. | "get me coffee", "get me some food", "nightlife", "get ice cream", "grab beer" |
| 📂open program | GitHub, Facebook, LinkedIn, Twitter, Spotify, Sublime, Atom | "open github", "open facebook", "open linkedin", "open twitter", "open spotify", "open sublime", "open atom" |
| 📴close program | Close a Spotify session. | "close spotify", "stop spotify" |
| 🎙️Record audio | record a stream of audio for a designated time period and save to desktop | "record audio" |
| 📹Record video | Open up zoom meeting link to record a video session. | "record video" |
It's quite easy to make new actions. Here's a general guide for you as you code them up.
You can easily modify Nala's event loop to recognize a new action (nala.py). Specifically, you can look for keywords like 'joke' and then execute a query command like this as part of the chain of queries.
...
elif query_transcript[i] in ['joke']:
command='python3 makeajoke.py %s'%(hostdir)
os.system(command)
query={
'date':get_date(),
'audio': unique_sample,
'transcript type': transcript_type,
'query transcript': query_transcript[i],
'transcript': transcript,
'response': command,
'meta': list(),
}
query_count=query_count+1
queries.append(query)
session.append(query)
action_count=action_count+1
query_request=True
if and_num == 0:
break
else:
and_num=and_num-1
...
If you are using any words outside of the standard Nala transcription corpus (see below), you'll have to train a new language model with Pocketsphinx. You can do that by reading the SETTINGS section of this wiki here.
play music
get the weather
get social
get coffee
get the news
get sports
get food
get ice cream
get beers
get beer
get social
get food
get nightlife
find a bar
plan trip
set alarm
stop alarm
make a poem
make a joke
record audio
record video
open atom
open sublime
open spotify
open twitter
open linkedin
open facebook
open github
chill out
exercise
I love you
search
be grateful
meditate
shut down
restart
log out
sleep
Let's go through this makeajoke.py example:
import random, os, sys, json, datetime
##############################################################################
## HELPER FUNCTIONS ##
##############################################################################
def get_date():
return str(datetime.datetime.now())
def speaktext(hostdir,text):
# speak to user from a text sample (tts system)
curdir=os.getcwd()
os.chdir(hostdir+'/actions')
os.system("python3 speak.py '%s'"%(str(text)))
os.chdir(curdir)
##############################################################################
## MAIN SCRIPT ##
##############################################################################
hostdir=sys.argv[1]
os.chdir(hostdir)
jokes=['What do you call a speeding french motorist? A speeding Monseiur.',
'What do chemists’ dogs do with their bones? They barium!',
'What did the cat say when the mouse got away? You’ve got to be kitten me!',
'What did the ocean\xa0say to the sailboat? Nothing, it just waved.',
'What do you call a snowman in July? A puddle.',
'What lies at the bottom of the ocean and twitches? A nervous wreck.',
'What does a dolphin say when he’s confused? Can you please be more Pacific?',
'What is a Queens favorite kind of precipitation? Reign!',
'What is the Mexican weather report? Chili today and hot tamale.',
'What did the evaporating raindrop say? I’m going to pieces.',
'What did the hail storm say to the roof? Hang onto your shingles, this will be no ordinary sprinkles.',
'What do you call a wet bear? A drizzly bear',
'What do you call two straight days of rain in Seattle? A weekend.',
'What goes up when the rain comes down? An Umbrella.',
'What does it do before it rains candy? It sprinkles!',
'What did one raindrop say to the other? Two’s company, three’s a cloud',
'What’s the difference between a horse and the weather? One is reined up and the other rains down.',
'What is a king’s favorite kind of precipitation? Hail!',
'What kind of music are balloons afraid of? Pop Music',
'What is the musical part of a snake? The scales.',
'What did Beethoven say to Johann Sebastian when he was helping him parallel park? “Bach it up.”',
'What’s an avocado’s favorite music? Guac ‘n’ roll.',
'What do you get when you drop a piano down a mineshaft? A-flat minor.',
'What do you call a cow that can play a musical instrument? A moo-sician.',
'What do you call a musician with problems? A trebled man.',
'What was Beethoven’s favorite fruit? BA-NA-NA-NAAAAAA.',
'What did Jay-Z call his wife before they got married? Feyonce.',
'What’s a golf clubs favorite type of music? Swing.']
# tell a random joke
randomint=random.randint(0,len(jokes)-1)
joke = jokes[randomint]
speaktext(hostdir, joke)
# update database
database=json.load(open('actions.json'))
action_log=database['action log']
action={
'action':'makeajoke.py',
'date': get_date(),
'meta': [joke],
}
action_log.append(action)
jsonfile=open('actions.json','w')
json.dump(database,jsonfile)
jsonfile.close()
There are a few key things here as general pointers. You almost always want to have these import statements and two functions to start off the actions, as you'll often need to speak to users and add the date when you get a new query:
import random, os, sys, json, datetime
def get_date():
return str(datetime.datetime.now())
def speaktext(hostdir,text):
# speak to user from a text sample (tts system)
curdir=os.getcwd()
os.chdir(hostdir+'/actions')
os.system("python3 speak.py '%s'"%(str(text)))
os.chdir(curdir)
Also, most actions make use of the sys.argv[] function to access variables from the command line. Specifically, when you look at the above section of code in Nala, you'll find that the makeajoke.py action is called with this command='python3 makeajoke.py %s'%(hostdir). In this way, we can pass the host directory to the action (which is the default directory where the nala script is executed), so you can reference where to store the data (in the actions.json file). Therefore, this snippet of code is extremely useful for you to put into your code base.
hostdir=sys.argv[1]
os.chdir(hostdir)
Then, after the main action is executed, you can terminate the action by updating the database. In general, this always has the 'action', 'date', and 'meta' fields. The action is the name of the script that is executed, the date is the current date/time, and the meta is anything useful that you'd like to pass along in the database in the form of a list - in this case, it's the joke itself that was played back to the user. In this way, you can cache actions and repeat them back so you don't have re-execute actions all the time.
# update database
database=json.load(open('actions.json'))
action_log=database['action log']
action={
'action':'makeajoke.py',
'date': get_date(),
'meta': [joke],
}
action_log.append(action)
jsonfile=open('actions.json','w')
json.dump(database,jsonfile)
jsonfile.close()
If you keep all these pointers in mind, it should greatly streamline and simplify the process of making new actions! If you have any questions, feel free to reach out by submitting a new github issue.
Once you have coded up the action and updated the Nala code base, all you need to do is put it into the ./actions/ directory along with all the other actions. Right now, here are all the current actions and supporting data in the folder:
['airport_data.json', 'alarm.py', 'breathein.mp3', 'breatheout.mp3', 'chillout.py', 'data', 'espn.py', 'events.py', 'exercise.py', 'generate_poem.py', 'grateful.mp3', 'grateful.py', 'grateful5.mp3', 'imagecrawl.py', 'makeajoke.py', 'meditation.py', 'mostgrateful.mp3', 'music.py', 'news.py', 'nutrition.py', 'plan_trip-speak.py', 'plan_trip.py', 'poetry.txt', 'reboot.py', 'search.py', 'shutdown.py', 'social.py', 'speak.py', 'startmeditation.mp3', 'turnoff.py', 'wakeup.wav', 'weather.py', 'yelp.py']