How AWS uses AI to power interactive artwork at new Smithsonian exhibit

What do you want your future to look like?

This fall, artist Suchi Reddy and Amazon Web Services (AWS), in collaboration with the Smithsonian FUTURES Exhibition, debuted me+you in Washington, DC, which embodies the collective answers to that question. me+you is an interactive work of art powered by artificial intelligence (AI) and machine learning (ML) and is the centerpiece of the Smithsonian FUTURES exhibition.

The artwork me+you creates a unique visual experience for all participants who express emotions about their futures, showcasing a plurality of ideas about the future, shaped by individuals and the power of the collective. The two-story sculpture invites visitors to speak their vision of the future into its base by saying, “My future looks…” and choosing a word to describe this vision. The sculpture then translates their words into a unique mandala of color and light, which travels up a central totem, where these patterns of light and color will grow and evolve over time as more visitors share their vision, as a representation of all our futures.

Prior to the fall exhibition, Suchi Reddy and AWS launched the me+you interactive web app. The app lets anyone, anywhere in the world, type a word about their future, and see art come to life. They can also see how others are feeling and what’s trending, creating an instant connection with the broader community.

Both the sculpture and web app me+you are powered by AI and AWS Cloud services. These technologies translate the participant’s spoken or typed word for the future into a unique mandala, both as a unique visual light pattern for an individual at eye-level on the sculpture’s base, as well as a collective visualization through the central totem that displays their personal future among all others. The exhibit and the web app use the same AI APIs to enable the experience.

What are ML and AI, and how do they power this art?

Machine learning (ML) powers AI, which allows computers to learn by finding patterns in data. The more a machine detects a pattern—or in the case of the me+you sculpture, hears a word—the more accurate it becomes at recognizing similar patterns. Beyond industrial applications, artists can use these tools and patterns to support unique emotional experiences.

Art and AI come in many forms, and me+you uses multiple AWS services to make the sculpture and web app come to life. Amazon Transcribe converts spoken word to text. From that converted text, Amazon Comprehend uses natural language processing (NLP) to find patterns in the converted text and does sentiment analysis of that text. Amazon SageMaker then eases collaboration, building, training, deploying, and managing the AI models that support the exhibit.

Beyond AWS ML Services, the sculpture runs on compute power from AWS Lambda. AWS Lambda combines the sentiment analysis from Amazon Comprehend with an ML technique called “word embedding” in Amazon SageMaker to form a feature vector per visitor. This vector helps the artist use AI to craft the individual experience — the mandala — for each visitor or web app user. These individual feature vectors are then transformed via a Docker based Lambda into an ‘embedding’ containing all interactions that appears in the sculpture’s central totem. This all happens quickly; visitors to the website or sculpture each get a real-time and interactive experience that then lives on in the collective totem. In this way, individual users are contributing and building the artwork throughout the duration of the installation, like a living, evolving organism. To secure communication between the exhibit, the website, and technical backend, the sculpture uses Amazon API Gateway, Site to Site VPN, and Amazon Route 53.

What else can ML and AI do? The future is bright. Learn more about how these technologies help transform agriculture to feed more people, find missing children, discover a cure for cancer, improve access to credit and financial services in developing countries, and more.

Supporting artists and dreamers with technology

AWS is committed to supporting the arts and builders, dreamers, and artists of all kinds, who invent, communicate, and delight by bringing their ideas to life through AWS technology – builders like Leila Nouri, the AWS employee who first imagined the artwork during an annual AWS “Think Big” process, and arts curator and current deputy director of the New Museum in New York, Isolde Brielmaier, who was instrumental in bringing the vision to life. The AWS team also consisted of talented professional service leaders and engineers, who made the vision a reality. By working with visionary curators like Isolde, artists like Suchi Reddy, and talented engineers – who all work at the intersection of culture and technology – and providing technical, logistical and financial support, AWS wants to help artists to bring to life new possibilities and encourage us all to envision a more dynamic, innovative, and connected future.

Learn more about me+you here and listen to Suchi Reddy describe the experience of creating the sculpture on the AWS Fix This podcast.

Get involved in the artwork now through the interactive web app.

Special thanks to the following for helping to bring this project to life: Nikki Bonjorni, engagement manager; Victor Wang, cloud infrastructure architect; Ashmeet Singh Pahwa, operations and monitoring; and Vaibhav Chaddha, cloud infrastructure architect.

Subscribe to the AWS Public Sector Blog newsletter to get the latest in AWS tools, solutions, and innovations from the public sector delivered to your inbox, or contact us.

Please take a few minutes to share insights regarding your experience with the AWS Public Sector Blog in this survey, and we’ll use feedback from the survey to create more content aligned with the preferences of our readers.