AWS Contact Center

Analyze Amazon Connect Chatbot performance using Contact Trace Record, Amazon Lex logs, Amazon Athena and Amazon QuickSight – Part 4

Note: This is the fourth blog in the Amazon Connect reporting blog series. If you haven’t read “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight–Part 1”, we strongly recommend you do before proceeding further.

Introduction

Many organizations want the ability to generate chatbot performance reports for their digital customer experience offering. Amazon Lex is the natural language understanding (NLP) engine for chatbots used by Amazon Connect. Relating Amazon Lex logs to the Contact Trace Record (CTR) and conversation logs, these organizations can identify chatbot issues, tune the NLP engine, understand customer sentiments, and improve the chatbot performance.
While CTR captures transactional metrics, such as hold time, wait time, and agent interaction time, and more, Amazon Lex conversation logs capture caller’s utterance, NLP confidence score, sentiment score, intent name, and more.
In this blog, you will learn how to enable Amazon Lex conversation logs, relate it to the CTR logs, and generate reports using Amazon QuickSight.

Overview

Fig 1–High level architecture diagram

In the above architecture, Amazon Kinesis Streams Amazon Connect CTRs as raw data to an Amazon Simple Storage Service (S3) bucket. The CTR captures the events associated with a contact call from your contact center. Amazon Connect delivers contact records at least once. It may deliver more CTR records for the same call, such as new information arriving after initial delivery.
Amazon Lex delivers text and audio logs for the conversation. For this blog, we will focus on the text logs with its associated metadata. Amazon Lex delivers conversational logs to S3 for each caller utterance. The conversation log is identified by a session ID, used to troubleshoot a distinct caller-chatbot session.
AWS Glue Catalog has the table definitions for the CTR and Amazon Lex output files. You can relate CTR and Amazon Lex logs by Contact Id and run queries using Amazon Athena. For visualization, Amazon QuickSight is used.
This solution uses a parameter named “Project Name” that defines the solution domain. You can deploy this solution multiple times with different project names in order to maintain isolation. Since it will be used to name fundamental components, such as an S3 bucket, the project name must be unique globally.

Pre-requisites

To follow along with the solution presented in this blog post, you must understand the following AWS services and features:

Walkthrough

There are two options to deploy based on the level of analyses:

  • Option 1—detailed analyses: This solution allows you to query Amazon Lex conversation logs with CTR entities, such as queues and agents.
  • Option 2—basic analyses: This solution allows you to analyze the Amazon Lex conversation logs only.

Option 1 deployment steps

Follow the blog “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight–Part 1” to deploy and validate the solution until Athena. For data visualization in QuickSight, follow the steps in this blog.

Note the “ProjectName” used while deploying the CloudFormation template in the first blog, as the same name must be used in this step to continue to build on top of the same deployment. You can use a project name like “acathenaquicksight” but the name must be unique or CloudFormation will return an error while creating the S3 bucket. If an error happens, you can redeploy with a different project name.

  1. Sign in to the AWS Management Console.
  2. Download the CloudFormation template from here into your local machine.
  3. In your preferred Region, create a CloudFormation stack using the template file downloaded in step 2
  4. The deployment requires the following parameters:
    1. Under “CreateANewGlueDatabaseAndS3BucketForTheProject” select the “Use existing…” option
    2. Use the same project name as given while deploying “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight-Part 1” blog.
  5. Keep the defaults on the remaining options page and select Create Stack
  6. To associate Amazon Connect Contact ID with the Amazon Lex conversation logs, add the “ContactId” (Destination key) as the session attribute in the Contact Flow for each “Get customer input” node.
  7. Provide the Amazon Lex Bot Alias in the Get customer input node.

Option 2 deployment steps

  1. Sign in to the AWS Management Console.
  2. Download the CloudFormation template from here into your local machine.
  3. In your preferred Region, create a CloudFormation stack using the template file downloaded in step 2
  4. The deployment requires the following parameters:
    1. Under “CreateANewGlueDatabaseAndS3BucketForTheProject” select the “Create a new…” option
    2. Provide a project name that will be used to create resources within AWS
  5. Keep the defaults on the remaining options page and select Create Stack

Steps Common to “Option 1” and “Option 2”

For Amazon Lex V1

  1. Navigate to bot’s setting tab and click on “Conversation logs”. Then click on the setting icon (cog wheel) for the bot Alias.
  2. Enable the Text logs and select Log group name and IAM role from the drop-down (both the names should contain the project name and the bot’s name provided in the CloudFormation Template (CFT) parameters).
  3. Now, build and publish the bot.

For Amazon Lex V2

  1. Navigate to the bot Aliases and click on the Alias name.
  2. Scroll down to the conversation logs section and click on Manage conversation logs
  3. Enable the Text logs and select the Log group name (It will contain the project name and the bot name provided in the CFT parameters)
  4. Scroll down and click on Save

Validation

Generate data by initiating a chat session and interacting with Amazon Lex chatbot. Within a few minutes, Amazon Lex logs data appear in the storage bucket named as your project name under the folder “lex-base” as a “.parquet” file.

  1. Log in to Amazon Athena, under the Tables you should see “connect_lex_<bot name>”.
  2. To query the data with Athena, click on three dots besides the ‘connect_chat’ table, then click on “Preview Table”
  3. Athena displays the query results when you run it.
  4. If you are deploying with option 1, create Amazon Athena view by executing the query under the CloudFormation output.

Visualize in Amazon QuickSight

Using Athena to query the data allows you to visualize the results directly with Amazon QuickSight. Before connecting Amazon QuickSight to Athena, you must grant Amazon QuickSight access to Athena and its associated S3 bucket. For more information, see managing Amazon QuickSight permissions to AWS resources. After granting the access permissions, you can create a new dataset in Amazon QuickSight based on the Athena table that was created.

  1. Navigate to QuickSight and select ‘Analyses’.
  2. Click on “New Dataset, then select “Athena” and give the data source a name.

In the example below, the name chosen is ‘connect-lex-ctr-visualization’. Afterwards, click on Create data source.

    1. Choose the name of the database and the source view/table based on the following deployment option, then click select:
      • If you are deploying with option 1, select view name “<ProjectName>_<BotName>_Lex_with_Ctr_AverageSentimentAndCtrDataPerContact”
      • If you are deploying with option 2, select the table name”connect_lex_<botname>”
      • Note: Create views within Amazon Athena to build enhanced visualization. For more details refer to this blog post.
    2. Finally, click ‘Visualize’ to complete the data-set creation.
    3. In the next screen, you should see the chat data like below

You can use Amazon QuickSight to add other chat records visualizations. Build enhanced visualizations, by creating views within Amazon Athena, check this blog post for more details. In addition, you can build dashboards that continuously monitor your connect instances and share those dashboards.

Clean up

In order to remove the resources created by the stack, perform the following:

  1. Delete the CloudFormation stacks created while implementing this blog.
  2. Delete the object and the S3 bucket created from the CloudFormation template. The Bucket name should contain the project name.
  3. Delete the Glue database created from the CloudFormation template.
  4. Delete the Athena view created for option-1 deployment.

Conclusion

In this blog, you learned how you can enable Amazon Lex logs, stream it to Amazon S3 using Amazon Kinesis Data Firehose. This blog also demonstrates how to relate CTR and Amazon Lex Logs, and generate reports on Amazon QuickSight. To build enhanced visualizations, by creating views within Amazon Athena, check this blog post for more details. To visualize your data in Tableau and Power BI, follow Building AWS data lake visualizations with Amazon Athena and Tableau and Using the Amazon Athena Power BI connector blog, respectively.

For more analytics capabilities on Amazon Connect data sources, refer the following blogs in the Amazon Connect reporting blog series:

Author Bio

Mehmet Demir is a Senior Solutions Architect at Amazon Web Services (AWS) based in Toronto, Canada. He helps customers in building well-architected solutions that support business innovation.
Ankur Taunk is a Senior Specialist Solutions Architect at AWS. He helps customer achieve their desired business outcomes in the Contact Center space leveraging Amazon Connect.