Generate images using Amazon Bedrock with stability diffusion model in .NET

Introduction

Amazon Bedrock is a fully-managed serverless service which provides high-performing Foundation Models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, and Stability AI. Amazon Bedrock offers developers the flexibility to select the most suitable Generative AI model based on their specific requirements. You can build generative artificial intelligence (AI) applications with security, privacy, and responsible AI using Amazon Bedrock unified APIs.

Stable Diffusion is one of the family of foundation models which is trained on vast image-text data. It enables generating high-fidelity images from descriptions. It generates images of high quality in virtually any art style and is a leading open model for photorealism.

As a .NET developer, you may want to integrate those new capabilities into your application to improve your end user experience. In this post, we will provide a walkthrough for developing and deploying a serverless solution for generating images using generative AI. This solution will leverage Amazon Bedrock and the Stability Diffusion Model to generate images based on user-provided text input prompts, utilizing the AWS SDK for .NET.

Stable Diffusion XL Model

Stable Diffusion XL is an industry leading image generation model that can generate images based on text or image input. Amazon Bedrock makes Stable Diffusion XL available through a unified API. The API supports passing various inference parameters to control and customize the text-to-image generation process using Stable Diffusion XL. The Stability.ai Diffusion 1.0 model has the following inference parameters for a text to image inference call:

{
    "text_prompts": [
        {
            "text": string,
            "weight": float
        }
    ],
    "height": int,
    "width": int,
    "cfg_scale": float,
    "clip_guidance_preset": string,
    "sampler": string,
    "samples",
    "seed": int,
    "steps": int,
    "style_preset": string,
    "extras" :JSON object
}

You can find parameter details at Stability.ai Diffusion 1.0 text to image in the Amazon Bedrock User Guide.

Following are a few examples of image generation using Amazon Bedrock with Stability diffusion model.

Figure 1: Generated image

Request payload:

{
  "prompt":"An image for girls having fun with beautiful colors, nice mountain view, with a river and some drinks",
  "seed":0,
  "stylePreset":"cinematic"
}

Figure 2: Generated image

Request payload:

{
  "prompt":"A cute baby of lion, 8k ultra realistic, trending on artstation, 4 k",
  "seed":0,
  "stylePreset":"digital-art"
}

In the preceding example, these parameters will be used by a Lambda function to prepare the request payload for diffusion model:

prompt: This parameter specifies the text description or prompt that the AI model will use to generate the image. You can specify as much as possible details about your image.
seed: It is an integer value that serves as a seed for the random number generator used by the AI model during image generation. Setting a specific seed value ensures that the same input parameters will produce the same output image every time. If you don’t explicitly set a seed value or set it to 0, then model will use a randomly generated seed value.
stylePreset: This parameter specifies the artistic style or visual aesthetic that the Stability Diffusion model should apply to the generated image. Allowed values are 3d-model, analog-film, anime, cinematic, comic-book, digital-art, enhance, fantasy-art, isometric, line-art, low-poly, modeling-compound, neon-punk, origami, photographic, pixel-art, tile-texture.

Solution Overview

The solution is a .NET serverless backend API that generates images based on the descriptive prompts provided. You can easily call it from your .NET application. It leverages Amazon Bedrock and Stable Diffusion XL model. Here is an overview of the solution components:

Components:

Amazon API Gateway – Exposes API endpoint for clients to call and pass the image prompt.
AWS Lambda – Contains image generation code using Amazon Bedrock and handles uploading image to S3.
Amazon Bedrock – Used for generating images based on different inference parameters, using Stable Diffusion model.
Amazon S3 – Stores generated images.
IAM – Provides necessary permissions to Lambda to call Bedrock API and access S3.

Architecture

Figure 3: Architecture

Figure 3: Architecture

Flow

The following outlines the process flow for generating images using the Stable Diffusion model based on architecture diagram.

Client calls the API Gateway endpoint and passes different inference parameters like the prompt, seed etc.
The API Gateway triggers a Lambda function which prepares the request model specific to stability diffusion model via Bedrock API.
Bedrock API generates the image.
Lambda function generates a pre-signed URL based on the object key and bucket name
Lambda function uploads the Base64 generated image to S3 bucket.
Lambda function returns a response to API Gateway, which includes the pre-signed URL for the image that was uploaded to S3.
The client can then access the image file directly from S3 using this pre-signed URL. The URL allows temporary access without requiring permissions or credentials.

Key benefits

As this solution utilizes serverless components, it enables automatic scaling to accommodate fluctuating workloads efficiently without needing to manage infrastructure. Furthermore, it integrates built-in access control, security measures, and distributed storage capabilities provided by AWS services. The serverless model ensures cost-effectiveness, as users only pay for the resources they consume. Also, the solution facilitates integration with other applications through the API Gateway, promoting interoperability and enhancing overall functionality.

Prerequisites

The following prerequisites are essential for the implementation of this solution.

An AWS account to create and manage the necessary AWS resources for this solution.
Get the latest version of the AWS CLI, and configure it for your account.
Get the AWS Cloud Development Kit (CDK) to define and provision AWS resources.
.NET CLI to run .NET code.

Provisioning Infrastructure

Model Access

Access to Amazon Bedrock foundation models is not granted by default. To obtain access, an authorized IAM user must submit a request through the console. The Model Access section in the Amazon Bedrock console allows users to view available models, their output modalities, access status, and End User License Agreements (EULAs). Users should review the EULA before requesting access to a model. For pricing information, refer to the Amazon Bedrock Pricing documentation.

Follow this process to enable access to the “Stable Diffusion XL” model within the Amazon Bedrock service.

Sign in to the AWS console.
Choose the region you want to work in from the top right corner.
Find the “Amazon Bedrock” service in the AWS console.
Once in the Amazon Bedrock service, select “Model access” on the left side menu.
Choose the “Manage Model Access” button.
Find “Stability AI” and check “SDXL 1.0” model. This will allow you to access and use this model.
After checking the box, choose Save changes to confirm your selection.

Creating CDK stack to provision AWS Resources

AWS Cloud Development Kit (CDK) is an infrastructure-as-code (IaC) open framework that enables you to model your infrastructure with your favorite high-level programming language like C#. You can use the following CDK code to provision required resources for this solution. If you are not familiar with AWS CDK apps, you may create your first CDK app in C# . Upon creation, the CDK application will contain a single stack. You can modify that stack with this code. This CDK stack will provision AWS Lambda, Amazon API Gateway, S3 bucket to store the image, IAM permissions for Lambda function to invoke the model The following code snippet demonstrates the creation of a AWS Lambda construct utilizing the AWS CDK. The complete code can be accessed on the GitHub repository.

        ...
                        
            // Create Lambda function which invoke Amazon Bedrock          
            var stableDiffusionXLG1Handler = new Function(this, "StableDiffusionXLG1Handler", new FunctionProps
            {
                Runtime = new Runtime("dotnet8", RuntimeFamily.DOTNET_CORE), // To support DOTNET_8 runtime https://github.com/aws/aws-lambda-dotnet/issues/1611,
                FunctionName = "StableDiffusionXLG1Handler",
                //Where to get the code
                Code = Code.FromAsset(".\\src\\TextToImageLambdaFunction\\bin\\Debug\\net8.0"),
                Handler = "TextToImageLambdaFunction::TextToImageLambdaFunction.Function::StableDiffusionXLG1Handler",
                Environment = new Dictionary<string, string>
                {
                    ["ENVIRONMENT"] = <environment name>,
                    ["BUCKET"] = <Amazon S3 Bucket name>
                },
                Timeout = Duration.Seconds(900)
            });

            // Assign permissions to Lambda to invoke Bedrock model
            string[] actions = {
                "bedrock:*"
            };

            var policy = new PolicyStatement(new PolicyStatementProps()
            {
                Sid = "BedrockPermissionForLambda",
                Actions = actions,
                Effect = Effect.ALLOW,
                Resources = new string[] { "*" }
            });
            
            stableDiffusionXLG1Handler.AddToRolePolicy(policy);
            
            ...

Creating Lambda function to process request prompt

Below is the AWS Lambda function code to invoke Stable Diffusion XL Model using Amazon Bedrock API to generate an image using model inference parameters.

    private const string StableDiffusionXLModelId = "stability.stable-diffusion-xl-v1";

    ...
    
    public async Task<APIGatewayProxyResponse> StableDiffusionXLG1Handler(APIGatewayProxyRequest request, ILambdaContext context)
    {
        var requestBody = JsonNode.Parse(request.Body);
        var prompt = requestBody!["cdk"]?.ToString();
        var stylePreset = requestBody["stylePreset"]?.ToString();
        var seed = Convert.ToInt32(requestBody["seed"]?.ToString());

        // it will prepare the InvokeModelRequest payload, which will be used by Bedrock client to invoke the model. 
        var payload = PrepareRequestPayload(prompt, stylePreset, seed);

        ...
        // Invoke the Stable Diffusion XL Model
            var response = await BedrockClient.InvokeModelAsync(new InvokeModelRequest
            {
                ModelId = StableDiffusionXLModelId,
                Body = AWSSDKUtils.GenerateMemoryStreamFromString(payload),
                ContentType = "application/json",
                Accept = "application/json"
            });

        ...
     }
            
    // Prepares a request payload with inference parameters
    private static string PrepareRequestPayload(string? prompt, string? stylePreset, int seed)
    {
        var jsonPayload = new JsonObject
        {
            { "text_prompts", new JsonArray
                {
                    new JsonObject
                    {
                        { "text", prompt }
                    }
                }
            },
            { "seed", seed }
        };

        if (!string.IsNullOrEmpty(stylePreset))
        {
            jsonPayload.Add("style_preset", stylePreset);
        }

        return jsonPayload.ToString();
    }
    
    ...

In the preceding code snippet, the BedrockClient.InvokeModelAsynyc method from AWS SDK is called to invoke the Stable Diffusion XL model with the prepared payload. If the model invocation is successful (HTTP status code 200), it prepares a pre-signed URL for the generated image and returns it in the response.

Deployment

Clone the GitHub repository image-generation-using-amazon-bedrock-api-with-dotnet that contains the AWS CDK constructs and AWS Lambda function code using the below command:

git clone https://github.com/aws-samples/image-generation-using-amazon-bedrock-api-with-dotnet.git

In command prompt/terminal, move to src directory and run the below command to build the solution.

dotnet build

Proceed to the directory containing the cdk.json file within your source code repository.

If this is your initial deployment utilizing the AWS CDK, it is necessary to initialize the CDK environment by executing the following command-line interface (CLI) command:

cdk bootstrap -—init

Subsequently, synthesize the AWS CloudFormation template by running the following command:

cdk synth

To initiate the provisioning of AWS resources, execute the following command:

cdk deploy

Testing

Once you finish the deployment by running cdk deploy command, You can test the implemented functionality using two ways:

Using AWS Console

Login to the AWS console and navigate to API Gateway
Search for your deployed API (e.g. BedrockImageGenerationRestAPI) and open it
Find the ‘image’ resource and click on the ‘POST’ method
Scroll down to the client box under the Method Request section
Construct a test payload like:

{
  "prompt":"An image for girls having fun with with beautiful colors, nice mountain view, with a river and some drinks",
  "seed":0,
  "stylePreset":"cinematic" 
}

Click on the “Test” button at the bottom to invoke the API method with the test payload
Check the request and response details to verify if it worked as expected

Using any Client

You can also test by invoking the API endpoint from any client like Postman. Make sure to replace your API endpoint URL, headers, and payload data and method as “POST”.

https://{api-id}.execute-api.region.amazonaws.com/DEV/image

{
"prompt":"An image for girls having fun with with beautiful colors, nice mountain view, with a river and some drinks",
"seed":0,
"stylePreset":"cinematic"
}

Once you get the successful response which contains an S3 pre-signed URL, copy that and open in the browser. You should be able to see your image.

Security & Monitoring

Bedrock security & monitoring

In order to ensure use of responsible AI, apply Amazon Bedrock GuardRails to mitigate the risk of undesirable image generation. You can monitor Amazon Bedrock using Amazon CloudWatch, which collects raw data and processes it into readable, near real-time metrics. For further readings, visit Monitor Amazon Bedrock and Security in Amazon Bedrock.

To further improve security of this solution,

Enable access logs for Amazon S3 bucket and Amazon API Gateway
Restrict Amazon S3 bucket access to TextToImageLambdaFunction AWS Lambda function
Enable Amazon CloudWatch logging
Use a custom role instead of default AWSLambdaBasicExecutionRole
Use Amazon API gateway authorization

Troubleshooting

If you encounter issues when invoking your API, here are some tips:

When receiving Lambda timeout errors, you can adjust the timeout property in your AWS CloudFormation template to allow more time for execution.
If you get an access error for a specific model ID, check that the necessary capabilities are enabled for that foundation model in the AWS region you are using. Go to the AWS Console to enable model access.
For errors about a model version reaching end of life, verify you have the correct model ID referenced. Use the latest Amazon Bedrock model IDs available to avoid retired models.

Cleanup

To save costs, delete the resources you created as part of the blog post. These resources can be cleaned up using below commands.

aws s3 rm s3://{Amazon S3 Bucket Name} —recursive

Note: The S3 bucket used in this solution might not empty while using the cdk destroy command for clean-up, so you need to ensure that the buckets are empty before running the destroy command. You can find the Amazon S3 bucket name from cdk deploy command output.

cdk destroy

Model Availability by AWS Region

When using Amazon Bedrock, be aware that the models you can access depend on which region you are working in. Not all models are available in every region – there is variation in model availability across regions. Please check the documentation at Model support by AWS Region for details on which models are available in each region. Consult the region availability matrix to see which models can be leveraged in your region before building solutions relying on certain models.

Conclusion

This post demonstrated how to build a serverless solution for generating images using the Stable Diffusion model in Amazon Bedrock with .NET. The sample application we utilized is available on GitHub. As a .NET developer working on AWS, cloning this repository and experimenting with Amazon Bedrock will provide hands-on learning opportunities.

We encourage you to learn and experiment with Amazon Bedrock, and other AWS AI services which you can utilize in your .NET code.

.NET on AWS Blog