Empowering the public sector with secure, governed generative AI experimentation

Across industries, generative artificial intelligence (AI) adoption is revolutionizing the way we work, from enabling new capabilities to streamlining inefficient processes and improving service delivery.

Successful adoption of generative AI, especially within the public sector, requires organizations to enable systematic experimentation and exploration, with their own data, for their workforce and constituents. Access to dedicated exploratory environments is crucial for meaningful progress. Without hands-on experience, public sector entities risk missing transformative opportunities. Through controlled trials and POCs, teams can identify and validate valuable use cases for AI assistance, recognize the technology’s current limitations, and set realistic expectations. However, government agencies, educational institutions, and nonprofits face unique challenges in adopting and deploying generative AI.

Challenges

Concerns around data privacy, regulatory compliance, and technical expertise can often hold organizations back from fully embracing this transformative technology. Here are some key considerations public sector organizations typically face when using generative AI systems:

• Data privacy and security – Public sector organizations often handle sensitive citizen and student data. Protecting the privacy and security of this information when using generative AI can be a significant challenge. Strict data governance protocols are typically required.
• Regulatory compliance – Stringent regulations in areas such as the Family Educational Rights and Privacy Act (FERPA) and the data privacy and breach laws applicable to government and nonprofit sectors may constrain the permissible use cases for generative AI.
• Skills and capability gaps – Public sector organizations often lack the in-house artificial intelligence (AI) and machine learning (ML) expertise required to effectively evaluate, deploy, and maintain generative AI systems.

But what if there was a way to create a safe, controlled environment for public sector employees to learn, experiment, and build with generative AI—while addressing those key barriers? Enter the Generative AI Sandbox on Amazon Web Services.

Solution overview

The Generative AI Sandbox on AWS, powered by Amazon Bedrock Studio, provides a secure, governed, and isolated environment for organizations to explore the power of large language models (LLMs) and other generative AI capabilities. Bedrock Studio users can test different LLMs side by side to understand which ones best suit their specific use cases: from drafting policy documents to analyzing public feedback, or creating educational content. This secure and fully managed playground allows users to create customized AI assistants (Apps) and enhance LLM interactions by incorporating their organization’s proprietary data and documentation. Additionally, the ability to share developed assets (prompts, apps, functions, knowledge bases) across departments promotes collaboration and efficiency, ensuring that successful AI applications can be replicated and scaled throughout the organization. Built on Amazon Bedrock, this turnkey solution empowers users of all skill levels to get hands-on with cutting-edge generative AI tools without the typical hurdles of data management, model governance, and infrastructure complexity.

Some of the key capabilities and benefits of the Generative AI Sandbox on AWS include:

• Secure, governed, and isolated environment – Deploy to an isolated Amazon Web Services (AWS) account and use AWS Identity and Access Management (IAM) to control who can access the sandbox and what actions they can perform. Customize security, data privacy controls, and Amazon Bedrock guardrails to align with your organization’s policies.
• Empower community learning and innovation – Give employees the freedom to prototype and share successes—all within a centralized, user-friendly workspace. No deep technical expertise required.
• Accelerate high-impact use cases – As promising Generative AI Sandbox experiments emerge, quickly jump-start them to production-ready solutions. Take advantage of the scalability and reliability of AWS services to build, deploy, and operationalize your most impactful generative AI applications.
• Cost-effective for experimenting – Pay per use, no licensing fees, and no upfront costs with access to high-performing foundation models (FMs) from leading AI companies such as: AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon using the Amazon Bedrock On-Demand Pricing model without any upfront licensing costs.

The Generative AI Sandbox is a straightforward and safe way for customers to start their generative AI journey, in which business and technical teams can collaborate and innovate, from modest chat-based assistants to complex workflows with multiple data sources and use cases using generative AI tools.

The following diagram illustrates the solution architecture.

Figure 1. Architectural diagram of the solution described in this post. The major components are an AWS account, AWS IAM Identity Center, and Amazon Bedrock.

How it works

The sandbox is designed to leverage AWS services such as AWS IAM Identity Center, AWS PrivateLink, and Amazon DataZone to offer a tailored approach to meet the diverse needs of technical and business teams within an organization, all while ensuring data privacy, access control, and consistent governance across the entire AI experimentation lifecycle.

Authentication and access

• Users access Amazon Bedrock Studio through single sign-on, using a secure URL
• Authentication is managed through an account instance of AWS IAM Identity Center
• Customer identity provider (IdP) integrates with IAM Identity Center for user management
• Users can be organized into groups (for example, GROUP-TEAM1) for easier access management

Workspace organization

• Multiple workspaces can exist within a single AWS account (for example, Workspace-Project XYZ, Workspace-Team ABC, Workspace-Innovation) and serve as the core isolation level construct for the service, separating departments, teams, and business lines from one another
• Each workspace contains multiple projects, owned by different users
• Users can access projects they own or that are shared with them
• Projects provide isolation and organization of resources within workspaces

Secure connectivity

• All communication between the customer AWS account and Amazon Bedrock occurs through AWS PrivateLink
• Traffic flows through VPC endpoints and never traverses the public internet
• The solution maintains separate VPCs in the customer account and AWS managed service account

Data management and governance

• Amazon DataZone serves as the underlying data governance layer
• Projects, apps, and various other components in Amazon Bedrock Studio each have a corresponding blueprint in Amazon DataZone
• Amazon DataZone maintains metadata and access controls for all resources
• This integration promotes consistent governance across all workspaces and projects

This architecture enables organizations to:

• Maintain strict access controls and user management
• Keep sensitive data and traffic within their AWS environment
• Organize and govern generative AI resources effectively
• Scale their generative AI initiatives across teams while maintaining security and compliance

Get started

To get started using the AWS Generative AI Sandbox, follow these steps:

1. Provision an isolated AWS account using AWS best practices for creating and managing sandbox accounts in AWS. The isolated AWS account makes sure there is no connectivity to the customer’s production and nonproduction AWS environments, and it sets up an AWS PrivateLink endpoint for the Amazon Bedrock API to make sure traffic doesn’t go through the internet.
2. Set up Account instances of IAM Identity Center with isolated users that will use the Generative AI Sandbox applications.
3. Provision the Generative AI Sandbox on AWS using Amazon Bedrock Studio, a web-based application that makes it easy to experiment with the broad set of pre-trained FMs on Amazon Bedrock. From there, users can dive into the included Chatbot Playground, build Retrieval Augmented Generation (RAG) applications, and more.

Conclusion

Whether you’re looking to enhance constituent services, improve student learning outcomes, or boost employee productivity, the Generative AI Sandbox on AWS provides a safe, scalable path to unleashing the power of generative AI across your public sector organization. Learn more and get started today.

Additional resources

Explore these additional resources to continue learning about how AWS can help power your generative AI solutions:

• Your guide to free and low-cost AWS courses that can help you use generative AI
• AWS Training and Certification Blog – Generative AI
• AWS Skill-Builder generative AI training and certification
• Amazon Bedrock Studio for generative AI app development