AWS Robotics Blog

Build an Alexa controlled robot with AWS RoboMaker

This blog references AWS RoboMaker Integrated Development Environment (IDE), a feature that has been deprecated. To follow along with this blog post, use the AWS Cloud9 IDE. See our updated blog on how to Build and simulate robotics applications in AWS Cloud9.

Introduction

Alexa is Amazon’s cloud-based voice service and Alexa-compatible devices allow users to control their smart devices with voice commands. Today, Alexa is available on hundreds of millions of devices from Amazon echo devices to third-party devices. Alexa Skill Kit (ASK) provides a way for robot developers and manufacturers to build a natural voice interface for their robots. AWS RoboMaker is a cloud robotics solution that provides a customized development environment, a fully-managed simulation infrastructure, and built-in cloud extensions making it easy to build and test robot applications and integrate them with other AWS services. In this blog post, I will show you how to use AWS RoboMaker to develop and test an Alexa-controlled robot.

Overview of Solution

Alexa provides a set of built-in capabilities, referred to as skills. A robot controller skill receives the customer’s voice command through an Alexa enabled device like Amazon Echo and converts the command to a text message. The text message is published to an IoT topic in an Alexa skill AWS Lambda code hook. The robot receives the text command through the subscribed AWS IoT topic and executes the command. AWS RoboMaker is used to develop and simulate the voice-controlled robot.

Walkthrough

Building an Alexa-controlled robot in AWS RoboMaker involves the following steps:

  • Create an IoT thing in AWS IoT.
  • Create an Alexa skill.
  • Create a robot application in AWS RoboMaker.
  • Simulate the robot in AWS RoboMaker simulations.

Prerequisites

For this walk through, you need to have the following prerequisites:

Step 1: Create a IoT thing in AWS IoT

First, create an IoT thing in AWS IoT Core Service. The IoT thing represents a robot in AWS IoT, and the command sends from Alexa to the robot through the IoT topic. For more details, see Create a thing.

  1. Log in to AWS IoT Core Console, select Onboard, then select Get Started. Click Onboard a device.
  2. Select Linux/OSX as your platform and select Python as the AWS IoT Device SDK, then click Next.
  3. Create a name for your IoT thing. For example: VoiceControlledRobot, then click Next.
  4. Select Linux/OSX to download the IoT connection kit package. Use the IoT thing certificates in the downloaded package to create the IoT MQTT client in the following steps. Click Next Step.
  5. Review the Configure and test your device section, then click Done.
  6. Go to AWS IoT Core, select Manage and then select Things. Select Create IoT thing and select Interact. Save the Rest API Endpoint of the IoT thing. The IoT thing endpoint is used to create the IoT MQTT client in the following steps.

Step 2: Create an Alexa skill

Here we build a new robot skill that converts voice commands to text message and sends the command to the robot. For more details, see Build Skills with the Alexa Skills Kit.

  1. Log in to Alexa Developer Console, then click Create Skill.
  2. Create a skill name, and select Alexa-Hosted(Python) as the method to host your skill’s back end resources, then click Create Skill.

  1. Select default Hello World Skill as template, then select Continue with template to create a new skill.
  2. Select + Add Intent in the intents section to create a name for the new intent, for example, MoveForwardIntent.

  1. Select Create Custom Intent and create a new MoveForwardIntent.

Add sample utterances to MoveForwardIntent:

tell robot to move forward
move forward
forward

  1. Select Save Model and then select Build Model.
  2. Select the Code tab at the top of the page. Add Amazon Root CA1 and AWS IoT Certificates downloaded in Step 1 to lambda folder. These certificates are used to build the IoT MQTT connection’

8. Add IoT MQTT Client and intent handler to existing lamba_funtion.py.

This sample Python script shows how to publish the command to an IoT Topic in MoveForwardIntent Handler. For more details about MQTT, see here.

from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient

# Parameters for AWS IoT MQTT Client.
iotThingEndpoint = "<iot_host_endpoint>" #  IoT thing endpoint saved in Step 1.
iotThingPort = 8883
rootCAPath = "<root-CA_path>" # eg. "./certificates/AmazonRootCA1.pem"
privateKeyPath = "<private_key_path>" # eg. "./certificates/VoiceControlledRobot.private.key"
certificatePath = "<certificate_key_path>" # eg. "./certificates/VoiceControlledRobot.cert.pem"
iotTopic = "topic_1"

# Init AWSIoTMQTTClient
skillIoTMQTTClient = AWSIoTMQTTClient("sdk-nodejs-publisher")
skillIoTMQTTClient.configureEndpoint(iotThingEndpoint,iotThingPort)
skillIoTMQTTClient.configureCredentials(rootCAPath, privateKeyPath, certificatePath)
# AWSIoTMQTTClient connection configuration
skillIoTMQTTClient.configureAutoReconnectBackoffTime(1, 32, 20)
skillIoTMQTTClient.configureOfflinePublishQueueing(-1)  # Infinite offline Publish queueing
skillIoTMQTTClient.configureDrainingFrequency(2)  # Draining: 2 Hz
skillIoTMQTTClient.configureConnectDisconnectTimeout(10)  # 10 sec
skillIoTMQTTClient.configureMQTTOperationTimeout(5)  # 5 sec

# Connect and subscribe to AWS IoT
skillIoTMQTTClient.connect()
logger.info("mqtt connected")

class MoveForwardIntentHandler(AbstractRequestHandler):
    """Handler for Move Forward Intent."""

    def can_handle(self, handler_input):
        # type: (HandlerInput) -> bool
        return ask_utils.is_intent_name("MoveForwardIntent")(handler_input)

    def handle(self, handler_input):
        # type: (HandlerInput) -> Response
        speak_output = "Ok, move forward"
        skillIoTMQTTClient.publish(iotTopic, "forward", 1)
        return (
            handler_input.response_builder
                .speak(speak_output)
                .response
        )
  1. Repeat previous Step 2 to add a MoveBackwardIntent.

Step 3: Create a robot application with AWS RoboMaker

Create a sample Robot application in AWS RoboMaker. You can find the source code in this aws-robotics GitHub repo. For more information, see Creating a Robot Application and Sample Robot Application.

  1. Go to AWS RoboMaker console, select Development, and then select Robot applications. Click Create robot application.
  2. Enter a name for the robot application. For ROS distribution, select ROS Kinetic. Use the default VPC and select a subnet. Select Create to create the AWS Cloud9 development environment.
  3. In the AWS Cloud9 development environment, select AWS RoboMaker Resources, then select Download samples, and then select Hello World.
  4. On the left, in the Environment tab, expand HelloWorld, HelloWorld, robot_ws, src, hello_world_robot. Create certificates fold and upload Amazon Root CA1 and AWS IoT Certificates downloaded in Step 1 to this folder. These certificates are used to build the IoT MQTT connection.

  1. Add AWS IoT dependencies to ROS.

Open a new terminal in AWS RoboMaker Development environment and run following command:

pip install AWSIoTPythonSDK
sudo bash -c 'echo "yaml https://s3.us-west-2.amazonaws.com/rosdep/iot.yaml" > /etc/ros/rosdep/sources.list.d/21-aws-iot-pip.list'
cd ~/environment/HelloWorld/robot_ws
rosdep update

cd ~/environment/HelloWorld/simulation_ws
rosws update
rosdep install --from-paths src --ignore-src -r -y
colcon build
colcon bundle
  1. Create a new node to receive MQTT command from Alexa and create a mqtt_listener.launch file for new created node.

This sample MQTT listener node script shows how to subscribe and receive command from IoT Topic.

import rospy
import logging
import time
from geometry_msgs.msg import Twist
from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient

# Configure logging
logger = logging.getLogger("AWSIoTPythonSDK.core")
logger.setLevel(logging.WARN)

# Parameters for AWS IoT MQTT Client.
iotThingEndpoint = "<iot_host_endpoint>"
iotThingPort = 8883
certificatePath = os.environ["CERTIFICATES"]
rootCAPath = certificatePath + "<root-CA_filename>" # eg. "AmazonRootCA1.pem"
privateKeyPath = certificatePath + "<private_key_filename>" # eg. "VoiceControlledRobot.private.key"
certificatePath = certificatePath + "<certificate_key_filename>" # eg. "VoiceControlledRobot.cert.pem"
iotTopic = "topic_1"

# Init AWSIoTMQTTClient
skillIoTMQTTClient = AWSIoTMQTTClient("sdk-nodejs-subscriber")
skillIoTMQTTClient.configureEndpoint(iotThingEndpoint,iotThingPort)
skillIoTMQTTClient.configureCredentials(rootCAPath, privateKeyPath, certificatePath)
# AWSIoTMQTTClient connection configuration
skillIoTMQTTClient.configureAutoReconnectBackoffTime(1, 32, 20)
skillIoTMQTTClient.configureOfflinePublishQueueing(-1)  # Infinite offline Publish queueing
skillIoTMQTTClient.configureDrainingFrequency(2)  # Draining: 2 Hz
skillIoTMQTTClient.configureConnectDisconnectTimeout(10)  # 10 sec
skillIoTMQTTClient.configureMQTTOperationTimeout(5)  # 5 sec

# Connect and subscribe to AWS IoT
skillIoTMQTTClient.connect()

DONE = False

# Initialize the node
rospy.init_node('listener', anonymous=True)

cmd_pub = rospy.Publisher('/cmd_vel', Twist, queue_size=10)

# Custom MQTT message callback
def customCallback(client, userdata, message):
    global DONE
    rospy.loginfo("Received payload!")
    rospy.loginfo(message.topic)
    rospy.loginfo(message.payload)
    
    command = message.payload
    rospy.loginfo("Processing command: " + command)
    
    # Execute command
    if command == "forward":
        twist = Twist()
        twist.linear.x = 1
        cmd_pub.publish(twist)
        time.sleep(1)
        twist.linear.x = 0
        cmd_pub.publish(twist)
    elif command == "backward":
        twist = Twist()
        twist.linear.x = -1
        cmd_pub.publish(twist)
        time.sleep(1)
        twist.linear.x = 0
        cmd_pub.publish(twist)
    else:
        DONE = True

# Subscribe to topics
skillIoTMQTTClient.subscribe(iotTopic, 1, customCallback)

time.sleep(2)

while True:
    time.sleep(1)
    if DONE:
        exit(0)

skillIoTMQTTClient.unsubscribe(iotTopic)

Here is the sample launch file.

<launch>
  <arg name="use_sim_time" default="true"/>
  <param name="use_sim_time" value="$(arg use_sim_time)"/>

<!-- Start MQTT listener on launch -->
  <node name="listener" pkg="hello_world_robot" type="listener.py" output="screen">
            <env name="CERTIFICATES" value="$(find hello_world_robot)/certificates/" />
  </node>
</launch>
  1. Build and bundle the robotics application sources
cd ~/environment/HelloWorld
colcon build
colcon bundle

Open a new terminal, copy the robot application source bundle to your Amazon S3 bucket. 

aws s3 mb s3://<your-bucket-name>
aws s3 cp ~/environment/HelloWorld/robot_ws/bundle/output.tar s3://<your-bucket-name>/robot.tar
aws s3 cp ~/environment/HelloWorld/simulation_ws/bundle/output.tar s3://<your-bucket-name>/simulation.tar
  1. Create a robot application.
aws robomaker create-robot-application --name MyRobotApplication --sources s3Bucket=<your-bucket-name>,s3Key=robot.tar,architecture=X86_64 --robot-software-suite name=ROS,version=Kinetic

This command returns information about the newly created robot application.

  1. Create a simulation application.
aws robomaker create-simulation-application --name MySimulationApplication --sources s3Bucket=<your-bucket-name>,s3Key=simulation.tar,architecture=X86_64 --robot-software-suite name=ROS,version=Kinetic --simulation-software-suite name=Gazebo,version=7 --rendering-engine name=OGRE,version=1.x

The command returns information about the newly created simulation application.

Step 4: Simulate the Robot with AWS RoboMaker Simulations

  1. Create a simulation job role by following our AWS document here.
  2. Go to AWS RoboMaker console, select Simulation Jobs, and click Create simulation job.
  3. For ROS distribution, select ROS Kinetic. For IAM role, select the role created in step 1.
  4. For Networking, select default VPC, default security groups, and two subnets. Select Next.
  5. For Robot application, select MyRobotApplication. Usehello_world_robot as launch package name and use mqtt_listener.launch as launch file name. Click Next.
  6. For Simulation application, use hello_world_simulation as launch package name and use empty_world.launch as launch file. Click Next.
  7. Review the details and click Create to create a simulate job.
  8. Select the new create simulation job and connect to Gazebo. A simulation window opens up.
  9. Go to Alexa Developer Console, select the new created skill, and select Test.
  10. Enter “Ask <invocation name> to move forward” and the robot moves forward in the simulation window.

Conclusion

In this blog, we demonstrated and walked you through the process on how to build a voice-controlled robot with an Alexa skill kit and AWS RoboMaker. To learn more about AWS IoT, visit https://thinkwithwp.com/iot/. To learn more about Alexa Skill Kit, visit https://developer.amazon.com/en-US/alexa/alexa-skills-kit. To learn more about AWS RoboMaker, visit https://thinkwithwp.com/robomaker/.

Marty Jiang

Marty Jiang

Marty Jiang is a Conversational AI Consultant with AWS Professional Services. Outside of work, he loves spending time outdoors with his family and exploring new technologies.