Perplexity Builds Advanced Search Engine Using Anthropic’s Claude 3 in Amazon Bedrock

Launched in December 2022, Perplexity AI can gauge context and personalize interactions by learning a user’s interests and preferences over time. Users also gain visibility into the credibility of information because each search result is accompanied by a list of sources.

Since the inception of their public API service, Perplexity has been using Amazon SageMaker, a fully managed service that brings together a broad set of tools for high-performance, low-cost machine learning (ML) for virtually any use case. After evaluating several cloud providers, Perplexity chose AWS for training and inference of its models to complement its use of Amazon Bedrock. “Using AWS, we had access to GPUs and benefited from the technical expertise of the proactive AWS team,” says Denis Yarats, chief technology officer at Perplexity. The company tested instance types from Amazon Elastic Compute Cloud (Amazon EC2)—which delivers a broad choice of compute, networking up to 3,200 Gbps, and storage services purpose-built to optimize price performance for ML projects. Specifically, Perplexity uses Amazon EC2 P4de Instances, which are powered by NVIDIA A100 GPUs and are optimized for distributed training, to fine-tune open-source FMs.

Through Amazon Bedrock, Perplexity AI users can select a model from the Claude 3 family of models from Anthropic, an AWS Partner. Claude 3 models feature expert knowledge, accuracy, and contextual understanding in addition to state-of-the-art performance. “Using a high-performing service such as Amazon Bedrock means we are tapping into Anthropic's powerful models in a way that allows our team to effectively maintain the reliability and latency of our product” says William Zhang, technical team member at Perplexity.

Because Claude provides information in concise, natural language, users can arrive at clear answers quickly. Users can also quickly upload and analyze large documents because Claude 3 models feature a context window of 200,000 tokens, the equivalent of roughly 150,000 words or more than 500 pages. “Ease of use is essential for making something part of our product,” says Zhang. “Using Claude 3 on Amazon Bedrock has been part of a great developer experience.”

Perplexity aims for every search result to be accurate and helpful by reducing hallucinations—inaccurate outputs of LLMs. Anthropic’s previous model, Claude 2.1, had already reduced their rate of hallucination by half. And Anthropic made further improvements in reducing hallucinations and increasing accuracy with the Claude 3 family, which has even further improved accuracy over Claude 2.1. As Anthropic works to drive model hallucinations to zero, Perplexity uses human annotators to further give its users accurate, safe, and trustworthy information. Additionally, Perplexity benefits from the commitment of Anthropic and AWS to responsible AI. “We appreciate that Amazon Bedrock has built-in content filters to alert us when people are trying to use our solution for unintended purposes,” says Aarash Heydari, cloud infrastructure engineer at Perplexity. As a safety and research company at its core, Anthropic is a market leader in combating “jailbreaks”—that is, attempts to generate harmful responses or misuse models.

Perplexity also continues to fine-tune other models on its AWS-powered infrastructure. In August 2023, Perplexity became an early beta tester of Amazon SageMaker HyperPod, which removes the undifferentiated heavy lifting involved in building and optimizing ML infrastructure for training FMs. Perplexity’s engineers worked alongside AWS solutions architects to create a groundbreaking scalable infrastructure that automatically splits training workloads across accelerated Amazon EC2 P4de Instances and processes them in parallel. Amazon SageMaker HyperPod is preconfigured with the distributed training libraries of Amazon SageMaker, further improving performance. “The speed of the training throughput doubled,” says Heydari. “The infrastructure was simple to manage, and the hardware-related failures dramatically reduced.”

To learn more about how Perplexity accelerates foundation model training by 40% with Amazon SageMaker HyperPod, read this case study.

After 2 months, Perplexity released a public API so that users can access its proprietary online models, Sonar Small and Medium, which are hosted on AWS and fine-tuned using Mistral 7B and Mixtral 8x7B. These online LLMs prioritize knowledge from the internet over training data to respond to time-sensitive queries. “Our infrastructure for model training and inference is all powered by Amazon SageMaker HyperPod, which was a critical factor for us in choosing AWS,” Heydari says. “Amazon SageMaker HyperPod has been instrumental in driving our AI innovation.”