Build smart chat apps with Amazon machine learning APIs and the PubNub ChatEngine

This is a guest blog post by Adam Bavosa, Open Source Software Engineer at PubNub. In their own words, “PubNub is a leading provider of real-time APIs for building chat, device control and real-time mapping apps that scale globally.”

The demand for chat apps is growing wildly. From social apps to business collaboration, in-game chat to customer support, every app needs chat. Chat apps like Telegram are making mainstream news and generating widespread interest.

Historically, building a scalable, feature-rich chat app was surprisingly difficult. Not anymore. PubNub ChatEngine has integrated Amazon machine learning APIs, which makes building intelligent, scalable chat apps super easy for mobile and web.

In this blog post, you’ll learn how to spin up the PubNub ChatEngine and add Amazon Translate and Amazon Polly for cross-lingual, speech-enabled chat apps that you can embed anywhere.

Amazon’s new machine learning (ML) APIs make chat applications more powerful. And when paired with pre-built PubNub integrations, the Amazon ML APIs can be easily added to any ChatEngine-powered chat app. You can use this blog post as a quickstart for using the new Amazon ML APIs and PubNub ChatEngine to build and deploy any kind of smart chat app quickly, whether for 1:1 chat or chatrooms that can each support over 1 million people, with enterprise-grade security and reliability.

The PubNub BLOCKS Catalog contains three blocks for Amazon ML APIs:

• Amazon Translate: Natural and fluent language translation
• Amazon Polly: Turn text into lifelike speech using deep learning
• Amazon Comprehend: Discover insights and relationships in text

A fourth block Amazon Lex (for adding conversational interfaces to your applications) will be added soon.

The following walk-through shows you how to get ChatEngine running, and demonstrates how to enable Amazon Translate and Amazon Polly within your chat app. The pattern is similar for the other Amazon APIs like Amazon Comprehend and coming soon (at the time of this blog post)… Amazon Lex, with those integrations (i.e., blocks) also available in the BLOCKS catalog.

Components covered

You will be building an app using the new Amazon ML APIs using the following components:

1. Browser tab with CodePen with built-in Chat
2. PubNub API Keys
3. Amazon Translate and Amazon Polly API Key

Getting Started with PubNub ChatEngine

PubNub ChatEngine makes building enterprise-ready chat apps easy. The PubNub ChatEngine allows for any UI to be integrated, so the UI example in this blog post is just that… a simple example.

Step 1: Create a free PubNub account

The best way to start is to create a free PubNub account. When you’re done, you should see your PubNub dashboard. Leave the portal open in a browser tab. If you already have a PubNub account, simply log in and leave the tab open.

Step 2: Create a ChatEngine instance in your PubNub account

Navigate to the PubNub ChatEngine Quickstart Tutorial page. In section 1 of the tutorial called “PubNub Keys,” you should see a   button. Press it. It will create a new PubNub keyset and configure it for use with ChatEngine. After a few seconds, you should see a success message followed by some code containing your new keyset:

Step 3: Create a copy of the ChatEngine Client with CodePen

Click here to launch a new CodePen window with ChatEngine pre-installed. Once CodePen is launched, you have to do two things:

• In CodePen, replace JavaScript lines 3-6 with your PubNub publish and subscribe keys. (That’s the code that appears after you pressed the Setup button earlier).

• Save your CodePen URL so it doesn’t get lost!
  

Congratulations! So far so good. Next you will implement Amazon APIs into your CodePen.

You now have a functioning ChatEngine app that you can begin tweaking. Open a new browser instance in Incognito mode to the same URL as your CodePen. Now you can chat with yourself, naturally. 

Later on (outside of this tutorial) you may want to dig into what you can do with ChatEngine, including Android and iOS support, complete the quickstart guide and also check out the ChatEngine docs.

Adding Amazon Translate to your PubNub ChatEngine instance

To demonstrate how simple it is to add Amazon APIs to PubNub ChatEngine, let’s start with Amazon Translate. There are only a few new concepts and details you need to understand before continuing the code tutorial:

• PubNub Functions allow you to intercept chat messages and add inline functionality. We’ll use PubNub Functions to call the Amazon Translate API and replace the source language with the target language.
• Note that your free PubNub key only allows 3 Functions to run simultaneously. Since ChatEngine already uses two Functions on its own, you can only experiment with one Amazon API at a time in your free account. Simply contact PubNub to get your account configured for more.

Step 1a: Amazon API Keys

You’ll also need your Amazon access and secret keys. If you have not already done so, create an AWS account and get an Amazon Translate and Amazon Polly access key and secret key (find out how here). Once you have your keys you will be able to add them into the AWS secure vault in your function.

Step 1b: Clone the Amazon Translate Block into Your PubNub account

1. First, ensure you’re still logged into PubNub. Then, open a new tab and go to the Amazon Translate Block. Scroll down the page, and click the “Try It Now” blue button.
   Important: on the next page, make sure you select “ChatEngine App” in the App dropdown:
2. Now click “Get Started”. This will take you to the PubNub Functions page, allowing you to edit your Functions code and deploy it.
   

Step 2: Configure your Amazon Translate PubNub Function 

Here, you will add your Amazon keys into the PubNub Functions Vault so the PubNub Function can successfully call the Amazon APIs. Then, you will also select the Source and Target languages for your Function.

• Click the “My Secrets” button on the left side of the PubNub Functions page. Add two new secrets: your Amazon access key and your Amazon secret key. The key names must be “AWS_access_key” and “AWS_secret_key”. When complete, your secrets screens should look like this:
  
  Important: do not delete the “secretKey” already in the vault. This is needed for ChatEngine and is not related to the Amazon APIs.
  

Step 3: Launch Your PubNub Function and Translate to Your Heart’s Content

There’s one final step.  You need to bind your Function to all PubNub “channels” to ensure that chat messages coming through any channel are translated. On the left side of the Functions page (right above the “My Secrets” button, find the Channel setting and change the value to a wildcard: *

Now, deploy your PubNub Function by clicking the red “Save” button, and then click the “Start module” button. After a few seconds, the PubNub Function will be deployed globally!

Now, go back to your existing ChatEngine app in CodePen and type in the source language. You should see the translations happening instantly.

Note: You can change your Source and Target language for Amazon Translate by changing the language codes on lines 9 and 10 of the JavaScript in CodePen.

Turn Chat messages into lifelike speech with Amazon Polly 

Amazon Polly creates audio, spoken language for any chat message sent through PubNub ChatEngine. This next section will show you how to deploy the Amazon Polly block, and also modify your ChatEngine client code to support audio files and spoken word.

Note: Your free PubNub account may require you to stop your Amazon Translate PubNub Function prior to deploying Amazon Polly. So remember to click the Stop Module button on your Amazon Translate function prior to starting this section of the guide. 

Step 1: Create your Amazon Polly PubNub Function

This tutorial has made some tweaks to the existing Amazon Polly Function in the BLOCKS Catalog. So, this time, instead of just cloning the existing block, we’re going to create our own PubNub Function from scratch. While the previous Amazon Translate Function used a “Before Publish” Event to intercept the chat message in-flight, this time we’ll use a PubNub REST endpoint Function, called “On Request” in the PubNub Portal.

1. Go to the PubNub Admin Dashboard, click on your ChatEngine App, then the FUNCTIONS tab on the left. You should see the ChatEngine module, click it and then click create to instantiate a new event handler.
   
2. Now, create the new module called “Polly”, and inside the module create a function also called Polly. (For clarity, a Module is a collection of Functions).
   
3. Click “Create” to add a new Function, and be sure to select the “On-Request” function type. Specify “polly-endpoint” for the URI Path:
   
4. Next we’ll copy the edited source code for the Amazon Polly integration, found in polly-pfunc.js, from the GitHub repository. Paste the code into the editor.The reason you are creating a Function Endpoint this time is the Chat App is setup to invoke a Voice MP3 Render request out-of-band from the original message. This is a great reason to use a Function Endpoint rather than an On-Before or On-After event handler.

Step 2: Configure your Amazon Polly PubNub Function

You should have already added your AWS access key and AWS secret key into the Vault in Step 2 in the Amazon Translate section. If you haven’t already done so, go ahead and add those two secret keys into the vault now.

Important: As before, do not delete the “secretKey” already in the vault. This is needed for ChatEngine and is not related to the Amazon APIs.

Step 3: Configure your ChatEngine client in CodePen 

Now, Start the Module by clicking the button at the top right. This button deploys the new code globally in just a few seconds. Our ChatEngine frontend is ready to go.

Before closing the Functions editor, click the COPY URL button. This copies the URL for the REST endpoint for the service.

In your CodePen file, paste this URL in the JavaScript to set the pollyFunctionURI (line 7).

var pollyFunctionURI = 'https://pubsub.pubnub.com/v1/blocks/sub-key/__SUB_KEY__/__URI__';

That’s it!  In CodePen, type a chat message. Turn up the volume, and click the Amazon Polly icon next to the chat message. You will hear the chat message spoken out loud!

Note for users who started this tutorial in the middle: If you haven’t started this tutorial from the top you may not have entered your API Keys into the CodePen. To get your API Keys and paste them into the CodePen JavaScript, visit the Key Info page in the PubNub portal for your PubNub ChatEngine app. Copy the Publish and Subscribe keys from the portal and paste them near the top of your CodePen.

Amazon Polly UI audio

The Amazon Polly audio data is played in the browser by using base64 encoded mp3 data URIs and the HTML audio element. The audio data is retrieved from the API only once when an Amazon Polly button is pressed. The data URI is stored on the dataset attribute of the chat log button element. There is one audio element that has its play feature triggered by the button click. In JavaScript, the URI is either fetched via AJAX, or subsequently it is moved from the button dataset to the audio source parameter.

ChatEngine features and plugins

You may have looked further at the CodePen application code. The extended features of ChatEngine plugins is used by the CodePen. You can also implement chat log history fetch, typing indicator, and the markdown plugin. ChatEngine and all of its plugins are open source on PubNub’s GitHub and available for import through npm.

Your ChatEngine code is tied to your PubNub Publish and Subscribe keys, and the chat channel. So, every other browser that launches this UI will see the same chat log as you.

There you have it! Now you can make chat apps smarter using the power of Amazon machine learning and PubNub ChatEngine. Using API integrations between ML services on AWS and PubNub, it is now possible for any developer to easily enhance their chat apps with powerful language translation, text-to-speech, sentiment analysis, and (coming soon!) chatbot functionality.

And with GDPR and SOC 2 Type II certifications using the PubNub network and ChatEngine, the largest enterprises can quickly deploy intelligent chat and collaboration functionality that meet any scaling, security, or reliability requirements.

About the author

Adam Bavosa is an Open Source Software Engineer at PubNub with interests including JavaScript, Blockchain and API development.