Amazon Bedrock Intelligent Prompt Routing is now available in preview

Posted on: Dec 4, 2024

Amazon Bedrock Intelligent Prompt Routing routes prompts to different foundational models within a model family, helping you optimize for quality of responses and cost. Using advanced prompt matching and model understanding techniques, Intelligent Prompt Routing predicts the performance of each model for each request and dynamically routes each request to the model that it predicts is most likely to give the desired response at the lowest cost. Customers can choose from two prompt routers in preview that route requests either between Claude Sonnet 3.5 and Claude Haiku, or between Llama 3.1 8B and Llama 3.1 70B.

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models from leading AI companies via a single API. Amazon Bedrock also provides a broad set of capabilities customers need to build generative AI applications with security, privacy, and responsible AI capabilities built in. These capabilities help you build tailored applications for multiple use cases across different industries, helping organizations unlock sustained growth from generative AI while ensuring customer trust and data governance. With Intelligent Prompt Routing, Amazon Bedrock can help customers build cost-effective generative AI applications with a combination of foundation models to get better performance at lower cost than a single foundation model.

During preview, customers are charged regular on-demand pricing for the models that requests are routed to. Learn more in our documentation and blog.