Build a chat bot to answer questions.
This repository contains code for the QnABot, described in the AWS AI blog post “Creating a Question and Answer Bot with Amazon Lex and Amazon Alexa”.
See the "Getting Started" to launch your own QnABot
New features in 4.2.0 Beta Kendra FAQ Support, Bug fixes, Multiple document chaining, Repeat question, Elastic Search 7.7 upgrade
New features in 4.0.0 Update to Elasticsearch 7.4, improved question matching accuracy, fuzzy matching, new multi-language support debug setting, SSML for Amazon Connect, improved Kendra integration, full upgrade support
New features in 3.0.3 New content tuning Readme, Enhanced CFN Lex Resource to work with manually created Bot versions
New features in 3.0.2 New Elicit Response Bots, Lambda Functions backing conditional chaining, Lex Bot versioning use
New features in 3.0.0 ElicitResponse, Conditional Chaining, new Launch regions
During an upgrade, we recommend that existing QnABot content first be exported and downloaded from the Content Designer prior to the upgrade. In this release we expect upgrade to be smooth but just in case you should always have your QnABot content preserved.
- Run Linux. (tested on Amazon Linux)
- Install npm >6.13.1 and node >10.16.3. (instructions)
- Clone this repo.
- Set up an AWS account. (instructions)
- Configure AWS CLI and a local credentials file. (instructions)
Two approaches can be used to get started. Deploy from pre-created repositories or clone the repo and build a version yourself.
Click a button to launch QnABot CloudFormation stack in the desired region
| Region | Launch |
|---|---|
| Northern Virginia |  |
| Oregon | |
| Ireland | |
| Sydney | |
First, install all prerequisites:
npm install Next, set up your configuration file:
npm run confignow edit config.json with you information.
| param | description |
|---|---|
| region | the AWS region to launch stacks in |
| profile | the AWS credential profile to use |
| namespace | a logical name space to run your templates in such as dev, test and/or prod |
| devEmail(required) | the email to use when creating admin users in automated stack launches |
Next, use the following command to launch a CloudFormation template to create the S3 bucket to be used for lambda code and CloudFormation templates. Wait for this template to complete (you can watch progress from the command line or AWS CloudFormation console)
npm run bootstrapFinally, use the following command to launch template to deploy the QnA bot in your AWS account. When the stack has completed you will be able to log into the Designer UI (The URL is an output of the template). A temporary password to the email in your config.json:
npm run upIf you have an existing stack you can run the following to update your stack:
npm run updateThe CloudFormation test templates are in the templates/test folder. The current templates are:
- Master: the template contains all the resources for QnABot.
- Public: this is a version of the Master template with less parameters, less outputs, and the bootstrap bucket hardcoded to the publicBucket in config.json
- various templates in /templates/dev: needed for local testing of the lambda functions.
Run a template test with:
npm run stack test/{template-name}For example, if you want to test the domain template run:
npm run stack test/domainTo understand the command more run:
npm run stack -hYou also can check a template's syntax with:
npm run check {template-name}ex.
npm run check domainTo understand the command more run:
npm check stack -hLambda functions are found in the /lambda directory. Refer to the README.md file in each directory for instructions on setting up a dev environment and testing. Fulfillment CFN Lex-Build Import
The Designer UI and client UI code is in the /website directory.
To Test the web ui, Launch a development master stack:
npm run stack dev/master upwhen that stack has finished run:
cd ./website ; make devthis will launch a running webpack process that will watch for changes to files and upload the changes to your running dev/master stack.
Currently the only browsers supported are:
- Chrome
- FireFox
We are currently working on adding Microsoft Edge support.
See the LICENSE.md file for details
- New Kendra FAQ support (Beta version) using the setting KENDRA_FAQ_INDEX. New menu item in Designer UI to export Questions as a Kendra FAQ. See revised Blog Post for details.
- New GetSessionAttribute Handlebars helper to obtain session attribute. Works similar to lodash get(). Will not through exception and will return a default value.
- Enhanced handlebars to support string concatenation including handlevar 'variables' like Session Attributes and UserInfo, etc. Use case, e.g. to build a url containing a users email, eg a google calendar URL. Example of syntax now supported - in this case to dynamically build a personalized URL based on user info. {{setSessionAttr 'link' 'https://calendar.google.com/calendar/embed?src=' UserInfo.Email '&ctz=America%2FNew_York'}}
- Moved 'previous' and 'navigation' session attributes under a new 'qnabotcontext' session attribute so that Connect (and other) clients have fewer session attributes to preserve.
- Allows Chaining rule Lambda function to return a modified session object in addition to the string for chaining.
- Allows Chaining of up to 10 documents. Each document's Lambda hooks will also be invoked in sequence if defined.
- Added a new Repeat QID in the QNAUtility example package. Allows QnABot to easily repeat the last answer.
- Allow the chaining rule to specify a specific QID rather than an answer. A QID can be specified in the chaining rule by using string such as QID:: e.g. QID::Admin.001. Note, the new QID:: syntax can also be used from the webUI, say as button values if/when you prefer to target a specific QID (exact query) rather than rely on question matching.
- Fixed a defect to allow conditional chaining to be invoked after an elicit response bot failure.
- Upgrades to and installs ElasticSearch 7.7.
- Install / Upgrade now supports the option to configure S3 Buckets and Elastic Search cluster using encryption at rest
- Install / Upgrade now supports the option to require Cognito based user authorization to access the built-in full screen web UI (Public/Private parameter in template) - Public is the default
- Added two settings parameters to enforce user identity verification check, so that bot can be secured for use by authenticated users only
- ENFORCE_VERIFIED_IDENTITY. Default is false. Set to true to make QnABot require verified identity from client
- NO_VERIFIED_IDENTITY_QUESTION. The default is "no_verified_identity". If user identity cannot be verified, replace question string with this. If not verified, the system will respond to user's question with the result of searching for NO_VERIFIED_IDENTITY_QUESTION. This allows a customizable message which informs the user that they must log in. A default question with qid "no_verified_identity" is included in QNAUtility example package.
- Enhanced Kendra fallback integration to use a specific answer if there is a best answer available and bold face highlighted words from Kendra response
- Added Comprehend sentiment analysis to all utterances and text captured by the QNAFreeText elicit response bot
- Enhanced Kibana dashboard to identify Lex client channels - Connect, Web, SMS
- Improved internal use of Booleans from settings configuration
- Enhanced Connect integration
- Added session attribute named "qnabot_qid" that holds the matching question id found in elastic search
- Added session attribute "qnabot_gotanswer" that holds boolean true/fale if an answer was fround
- Encapsulating all Kendra and Elicit Response Bot session attributes into a single "qnabotcontext" attribute making it easier to store and reset in Connect contact flow
- Added new QNAYesNoExit elicit response bot which allows a user to exit the YesNoExit question using "exit", "bye", "quit", "admin", "rep","representative","stop", "help", "bye", "goodbye" which sets the Yes_No_Exit slot value / session attribute to "Exit".
- Update to 0.17.0 of embedded lex-web-ui
- Resolved additional dependabot identified security issues with dependent packages
- Fixed lambda/fulfillment unit tests
- Fixed defect where response bot was not triggered on next question when using lambda function for conditional chaining
- Update to Elasticsearch 7.4
- Improved question matching accuracy and tuning
- Tolerance for typos and minor spelling errors with fuzzy matching setting
- Easier troubleshooting when using voice or multi-language support with new debug setting
- SSML support when using Amazon Connect
- Improvements to Amazon Kendra integration
- Full upgrade support without data loss when upgrading from previous versions.
Content Tuning an and Accuracy Guide now available as a Markdown Readme. README
QnABot can be configured to redact information written to CloudWatch logs, S3 metrics, and Kibana metrics logs. This feature is disabled by default. Use the Designer UI Settings form to enable this feature. One can configure the RegEx applied to strings as they are logged. If RegEx matches are found, the match is replaced with the string 'XXXXXX'.
The initial RegEx is
\b\d{4}\b(?![-])|\b\d{9}\b|\b\d{3}-\d{2}-\d{4}\b
This replaces 4 digit numbers not followed by a hyphen, a 9 digit number (SSN without hyphens), and a typical SSN using nnn-nn-nnnn syntax with hyphens.
New example demonstrating how QnABot can be asked by a user for a live agent based phone callback. The implementation provides a new LambdaHook example as well as four sample questions that ask a user for their name and phone number prior to handing off to an Amazon Connect instance to initiate the callback.
Two configuration updates are required to use this example with Amazon Connect.
The IAM Role/Policy used by the ConnectCallback Lambda must include a new policy that allows the action "connect:StartOutboundVoiceContact" to be used with the resource "arn:aws:connect:::instance//*". The following is an example of this policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "connect:StartOutboundVoiceContact",
"Resource": "arn:aws:connect:*:*:instance/<YourConnectInstanceId>/*"
}
]
}
- Find the Lambda ConnectCallback Function in the AWS Lambda Console
- Open the AWS Console and select the Lambda Service
- In the Console's filter enter 'ConnectCallback' and press enter
- The displayed function will start with -ExamplePYTHONLambdaConne... If you have multiple QnABot stacks installed you'll see multiple functions listed
- Open the Lambda function by clicking on the function
- Select the Permissions tab
- Click on the Role name to open this Role in a new tab
- Click on + Add inline policy
- Select the JSON tab
- Copy the sample text above, paste as JSON, and change to the Instance ID identified in the Connect Console.
- Click on Review policy
- Enter a name for the policy and click Create policy
You've now enabled Lambda functions using this role to start outbound calls via the connect instance
Lambda Hook Arguments need to be updated. Before being used, the item with qid CONNECT_TO_AGENT.04 should have its Arguments field adjusted to reflect identifiers from the Connect instance:
"AWS_connect_instance_id": "<your-connect-instance-id >",
"AWS_connect_contact_flow_id": "<your-connect-contact-flow-id>",
"AWS_connect_queue_id": "<your-connect-queue-id>",
Once these configuration changes are in place, QnABot can be successfully use Amazon Connect to place outbound calls.
You can specify that a Conditional Chaining rule runs a Lambda Function using the "Lambda::FunctionName" syntax. The Lambda function name must start with "QNA". The Lambda will receive arguments the same as Lambda Hooks. The function must return a String that is used to resolve the next item just the same as other conditional chaining rules.
This release of QnABot utilizes Lex Bot aliases and Lex versioning. The QnA Bot and all ElicitResponse Bots are now installed using the alias 'live'. 'live' points at the latest numbered Bot version. All Lex resources are now versioned starting with '1' after initial installation.
This release adds several elicit response bots for QNAWage - 7 digit wage value - returns slot named "Wage" QNASocialSecurity - SSN Numbers in the nnn-nn-nnnn format - returns slot named "SSN" QNAPin - 4 digit pin value - Returns slot named "Pin"
QnABot was designed to answer questions, but now it can also ask questions and process the user’s answers. Use this feature for data collection and validation; or implement surveys, quizzes, personalized recommendations; or to build a triage chatbot application. See the blog post for details about this new feature. This feature is know as ElicitResponse from a configuration perspective.
QnABot can now chain the user from one answer to another. This can be used with ElicitResponse to ask multiple questions from a user. Conditional chaining allows branching based on the state of session variables. Please see the blog post for a description on how to use this feature.
QnABot version 2.6.0 optionally supports integration with Amazon Kendra as a fallback mechanism if a question/answer can not be found in QnABot.
Important note. Use of Kendra as a fallback mechanism will incur additional charges for your AWS Account. Please review the Kendra pricing structure. The fallback mechanism for QnABot can be useful when deploying Kendra as an Enterprise search solution.
To enable this support for your Kendra indexes, use the Settings UI in the Designer and add your index to the ALT_SEARCH_KENDRA_INDEXS parameter. This parameter takes an array of strings and uses the form below.
["a672e3a2-nnnn-nnnn-nnnn-7b3abc81c313"]
Don't forget to use your Kendra Index ID rather than the one in the sample
Next use the QnABot Designer UI to import a Sample/Extension named KendraFallback.
This loads a new question with a qid of "KendraFallback". Edit this question in the Designer UI and change its question from "no_hits_alternative" to "no_hits" and save the changes.
If you have previously loaded the QnAUtility.json from Examples/Extensions you need to either remove the question with the ID "CustomNoMatches" or change the question for this ID from "no_hits" to "no_hits_original"
Once the new question, "KendraFallback" is configured as the response for "no_hits", the Kendra index will be searched for an answer whenever a curated answer can not be found. This feature provides a fallback mechanism prior to telling the user an answer could not be found.
A workshop is available in github that will walk you through setting up this feature.
Important note. Use of Kendra as a fallback mechanism will incur additional charges for your AWS Account. Please review the Kendra pricing structure. The fallback mechanism for QnABot can be useful when deploying Kendra as an Enterprise search solution.
QnABot version 2.6.0 supports use of multiple languages with these limitations:
- MultiLanguage support for voice is limited to use with Alexa skills only. (Amazon Lex currently supports voice recognition in English only)
- MutiLanguage support for text is available via all text clients (e.g. Lex Web UI, SMS, etc.)
####Lex (text mode only):
If the user enters a question in a language other than english, QnABot will attempt to return an answer in the other language. It does this by using Amazon Comprehend to identify the language typed. If Comprehend can identify the language based on a configured minimum confidence, QnABot will serve up content based on that locale.
Users can also set a preferred language whereby QnABot will always attempt to respond with content in the chosen locale. If the user sets the preferred language to be Spanish, QnABot will always try and serve up content using Spanish when possible.
####Alexa (voice)
You will need to add each language you want to use to your QnABot skill using the Alexa Developer console. The intent schema for each language will be identical except for the skill invocation name. Give the skill a unique invokation name for each language that you add.
QnABot will use the language setting provided by Alexa, and will attempt to respond in that language.
####How it works
QnABot converts the question posed by the user to English, using Amazon Translate, and performs a lookup of the answer in Elastic Search just as it normally does, using the English translation of the question. ElasticSearch searches are done in English only since QnABot documents are indexed using the English text analyser (stemming, stop words, etc.)
To ensure good matching of translated questions, you can use the Amazon Translate console to see the English tranlation of your local language question. Use QnABot content designer to ensure your QnA item has the right sample questions to ensure a match.
Once it finds the question, QnABot will serve up the configured answer.
You can use Handlebar blocks to define explicit answers in each different language you want to support. OR, if you do not explicitly define an answer in the user's language, QnABot will automatically use Amazon Translate to convert the default enghlish answer to the target language.
####Configuration
By default this feature is disabled. Use the following three steps to enable and configure this feature. Step 1 enables the feature. Step 2 loads in two questions from this extension that allow the user to select a preferred language. The defaults supplied in this question are English, Spanish, French, German, and Italian. You can extend this list to support other languages.
Step 1) Enable multi language support
a) QnABot uses a property named ENABLE_MULTI_LANGUAGE_SUPPORT, default value of "false". You can change this setting using the Content Designer Settings page. Set it to "true" to enable multi language support.
Step 2) Use the Designer UI to import the Sample/Extension named Language / Multiple Language Support.
This will add two questions to the system: Language.000 and Language.001. When using Lex text clients, these questions will allow you to set your preferred language.
The preferred language, if set, will take precendence over
the auto detected language.
When using Alexa, the language is automatically set by the skill. You will not be able to override the preferred language when using Alexa.
Language.000 provides a question that allows the user to set the current sessions preferred output saying a simple word such as French, German, or Spanish, or Italian.
Language.001 resets the preferred language. This can be performed by saying or typing 'reset language' or 'detect language'. You can also input using your language of choice assuming AWS Translate can translate the input back to English.
Once you've imported this extension question try typing the question 'Spanish'. You should see a Spanish response.
Next enter 'English' and you will have switched your preference back to English.
Next enter 'reset language' and your preference will be reset and language auto detection will occur again.
The answer for Language.000 uses the following handlebar syntax
{{#setLang 'fr' false}} D'accord. J'ai défini votre langue préférée sur l'anglais. {{/setLang}}
{{#setLang 'es' false}} Okay. He configurado tu idioma preferido al inglés. {{/setLang}}
{{#setLang 'de' false}} In Ordnung. Ich habe Ihre bevorzugte Sprache auf Englisch eingestellt. {{/setLang}}
{{#setLang 'it' false}} Ok. Ho impostato la tua lingua preferita sull'inglese.{{/setLang}}
{{#setLang 'en' true}} Ok. I've set your preferred language to English. {{/setLang}}
The helper function setLang performs the necessary processing to set the preferred language/locale for teh session. To add support for other languages just extend the answer in Language.000 with additional locales.
Step 3) In order to serve up content that is locale specific you can
- allow QnABot to automatically translate your english answers to the session language using Amazon Translate.
- OR provide explicitly curated answers in QnA items, in multiple languages, using handlebars, as shown below.
Lets modify the question sun.1. The following would be an example where the handlebar function ifLang is used to specify a response for spanish.
Use the handlebar template defaultLang to specify the response QnABot should provide when the language is unknown. By default this is typically in English but could be in any language as needed.
{{#defaultLant}}{{/defaultLang}} must be the last element in the answer block.
{{#ifLang 'es'}}
Nuestro sol tiene 4.600 millones de años. Se considera una enana amarilla con un diámetro de 1,392,684 kilómetros y una circunferencia de 4,370,005 kilómetros. Tiene una masa que es igual a 333,060 tierras y una temperatura superficial de 5,500 grados centígrados. ¡Muy caliente!
{{/ifLang}}
{{#defaultLang}}
Our sun is 4.6 billion years old. Its considered a yellow dwarf with a diameter of 1,392,684 kilometers and a circumference of 4,370,005 kilometers. It has a mass that is equal to 333,060 earths and a surface temperature of 5,500 degrees celsius. Really Hot!
{{/defaultLang}}
The handlebar function ifLang takes locale as a quoted parameter. This tells QnABot which locale to associate with the subsequent text.
A workshop is available in github that will walk you through setting up this feature.