Centralized logging for Windows containers on Amazon ECS using Fluent Bit

Introduction

Today, Amazon Web Services (AWS) announced the support for Fluent Bit container images for Windows operating system. This support eliminates the need for Windows customers to implement any custom logging solutions in their application code or manage custom agents on their Windows instances to scrape the logs. For more details about the supported Windows versions and the image tags for Fluent Bit, please visit our documentation here.

Fluent Bit is a fast and flexible log processor and router supported by various operating systems. It’s used to route logs to various AWS destinations, such as Amazon CloudWatch, Amazon Kinesis Data Firehose destinations, Amazon Simple Storage Service (Amazon S3), and Amazon OpenSearch. In addition to the AWS destinations, Fluent Bit supports common partner solutions such as Datadog, Splunk, custom HTTP servers, and many more. For more details about container log routing, please visit the blog link here. AWS already supports the Fluent Bit container images based on Linux operating system. With today’s release on Windows support, it now allows customers to have a centralized mechanism for processing and routing their logs across Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Kubernetes Service (Amazon EKS) for both Linux and Windows workloads.

In this post, we cover how Amazon ECS customers can deploy Fluent Bit containers on their Windows instances running in Amazon ECS to stream Internet Information Services (IIS) logs generated by the Windows tasks to Amazon CloudWatch logs to centralize logging. You can also use Fluent Bit on Amazon EKS and create a unified logging experience across Amazon ECS and Amazon EKS. For more information about deploying Fluent Bit on Amazon EKS, visit our blog post here.

Architecture overview

Architecture of setup used in this post

In order to set up Fluent Bit on Amazon ECS, we’ll use the following approach:

• Run Fluent Bit as a standalone container on the instance such that:
  • It listens on port 24224 using the forward input plug-in.
  • We expose the port 24224 to the host so that docker runtime send logs to Fluent Bit using this exposed port.
  • Fluent Bit has its own configuration that enables it to send the logs records to specified destinations.
  • For production usage, we suggest running Fluent Bit task as an Amazon ECS Service with Daemon scheduling strategy. This ensures that a single instance of Fluent Bit always runs on your container instances in the cluster. For step-by-step guide, visit our Amazon ECS tutorial.
  • Fluent Bit task should be started prior to your application tasks so that it can capture all the application logs.
• All other Amazon ECS task containers are launched using the fluentd docker logging driver.
  • Docker connects to the TCP socket 24224 on localhost inside the host namespace.
  • Amazon ECS agent adds labels to the containers, which include cluster name, task definition family name, task definition revision number, task ARN, and the container name. We’ll add the same to the log record using labels option of the fluentd docker logging driver.
  • Because we are setting async option of the fluentd logging driver as true, even if the Fluent Bit container is restarted, docker buffers the logs until Fluent Bit container is restarted. You can increase the buffer limit by setting the fluentd-buffer-limit option.

Therefore, the end-to-end workflow is as follows:

• Fluent Bit container is started and listens on port 24224, which is exposed to the host.
• Fluent Bit uses the Task AWS Identity and Access Management (AWS IAM) role credentials as specified in its task definition.
• Other Amazon ECS tasks are launched on the same instance, which use fluentd docker logging driver to connect to the Fluent Bit container on port 24224.
• When logs are generated by application containers, docker runtime tags those records, adds additional metadata specified in labels, and forwards them on port 24224 in the host namespace.
• Fluent Bit receives the log record on port 24224 since it’s exposed to the host namespace.
• Fluent Bit perform its internal processing and routes the logs as specified.

We have created some default Fluent Bit configurations to help you get started with minimal effort. For more information, visit the documentation here.

In this post, we’ll use the default Amazon CloudWatch Fluent Bit configuration, which would:

• Create a new log group for each Amazon ECS cluster and Amazon ECS task definition family.
• Create a new log stream for each task container in above generated log group whenever a new task is launched. Each stream is marked with the task-id to which the container belongs.
• Add additional metadata such as Amazon ECS cluster name, Amazon ECS task ARN, Amazon ECS task container name, Amazon ECS task definition family, and the Amazon ECS task definition revision number in each log entry.

For example, if you have the following task definitions with the containers as:

ecs_task_1 -> container_1 and container_2
ecs_task_2 -> container_3

Assuming each of these task definitions has a single task running in an Amazon ECS cluster named windows, then we get the following Amazon CloudWatch log groups and streams:

• /aws/ecs/windows.ecs_task_1
  • task-out.<TASK_ID>.container_1
  • task-out.<TASK_ID>.container_2
• /aws/ecs/windows.ecs_task_2
  • task-out.<TASK_ID>.container_3

Prerequisites

Prerequisites and assumptions:

• The latest version of the AWS Command Line Interface (AWS CLI) is installed and configured. For more information, see Installing the AWS Command Line Interface.
• The aws-for-fluent-bit container image is available for the following Windows operating systems and therefore, you should be using the same:
  • Windows Server 2019 Core
  • Windows Server 2019 Full
  • Windows Server 2022 Core
  • Windows Server 2022 Full
• The steps in Set up to use Amazon ECS have been completed.
• You have a cluster. In this post, the cluster name is FluentBit-cluster. For more information about creating Amazon ECS cluster, visit our docs here.
• A Virtual Private Cloud (VPC) with a public subnet where the Amazon EC2 instance is launched. You can use your default VPC. You can also use a private subnet provided that Amazon CloudWatch endpoints are reachable from inside the private subnet. For more information regarding VPC, visit our documentation here.
• In order to build a Windows container image, you need to have created an Amazon Elastic Compute Cloud (Amazon EC2) Windows instance with Docker installed, which is based on the same Windows version as the Amazon ECS container instance. Alternatively, you can also use Amazon EC2 Image Builder for building your container image.
• An Amazon ECR repository to store the Windows container images. For more information about creating the same, visit our documentation here.
• Amazon ECS Exec prerequisites are completed if you wish to access your task containers to generate the logs. For more information, visit our documentation here.

Solution overview

In this post, we’ll complete the following tasks:

1. Create the IAM access roles
2. [Optional] Build a Windows application container image containing IIS and LogMonitor
3. Create an Amazon ECS Windows container instance
4. Configuration used for Fluent Bit
5. Register a Windows Fluent Bit task definition which routes the logs to Amazon CloudWatch
6. Run the above registered ecs-windows-fluent-bit task definition
7. Register a Windows task definition with application containers which generates the logs
8. Run the above registered windows-iis-task task definition
9. [Optional] Access the IIS pods to generate logs
10. Verify the logs on Amazon CloudWatch
11. Cleaning up

Walkthrough

1. Create the AWS IAM access roles

Create the required AWS IAM roles.

1. Create a new role named buildInstanceRole, which is used by your Windows build instance to create and push Amazon ECR images. If you are running in a production environment, then we recommend that you customize the role policies for minimal permissions.

For this post, you need to push docker image to Amazon ECR. Therefore, attach the following inline policy to your build instance IAM role.

{
    "Version": "2012-10-17",
    "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "ecr:InitiateLayerUpload",
            "ecr:PutImage",
            "ecr:UploadLayerPart",
            "ecr:CompleteLayerUpload"
        ],
        "Resource": "*"
    }
    ]
}

To attach the inline policy to your instance IAM role, perform the following operations:

• Save the above content in a file named ecr-push-policy.json.
• Run the following command to attach the inline policy to above created buildInstanceRole IAM role.

aws iam put-role-policy --role-name buildInstanceRole --policy-name ecr-push --policy-document file://ecr-push-policy.json

2. Create the Amazon ECS container instance role named ecsInstanceRole. For more information, see Amazon ECS container instance IAM role.

3. Create an IAM role for the Fluent Bit task named fluentTaskRole. For more information, see Creating an IAM role and policy for your tasks.

The AWS IAM permissions granted in this IAM role are assumed by the task containers. In order to allow Fluent Bit to send logs to Amazon CloudWatch, you need to attach the following permissions to the task AWS IAM role.

{
    "Version": "2012-10-17",
    "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "logs:CreateLogStream",
            "logs:CreateLogGroup",
            "logs:DescribeLogStreams",
            "logs:PutLogEvents"
        ],
        "Resource": "*"
    }
    ]
}

To attach the inline policy to your task AWS IAM role:

• Save the above content in a file named fluent-bit-policy.json.
• Run the following command to attach the inline policy to fluentTaskRole AWS IAM role.

aws iam put-role-policy --role-name fluentTaskRole --policy-name fluent-bit-policy --policy-document file://fluent-bit-policy.json

4. [Optional] Create an AWS IAM role for the Windows task named iisTaskRole, which is used to generate the logs. This is required only if you want you access your Windows task containers for generating logs as described in Step 9.

In order to turn on Amazon ECS Exec for your task, you need to attach the following permissions to the task IAM role. For more information about Amazon ECS Exec, see Using Amazon ECS Exec for debugging in Amazon ECS guide.

{
   "Version": "2012-10-17",
   "Statement": [
       {
       "Effect": "Allow",
       "Action": [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:OpenDataChannel"
       ],
      "Resource": "*"
      }
   ]
}

To attach the inline policy to your task AWS IAM role:

• Save the above content in a file named ecs-exec-policy.json.
• Run the following command to attach the inline policy to iisTaskRole IAM role.

aws iam put-role-policy --role-name iisTaskRole --policy-name ecs-exec-policy --policy-document file://ecs-exec-policy.json

2. [Optional] Build a Windows application container image containing IIS and LogMonitor

If you already have a Windows container image for your application, you can skip this section.

2.1 To test the functionality explained in this post, create a Windows container image containing IIS and LogMonitor. By default, Windows containers send logs to Event Tracing for Windows (ETW), Event Log, and custom logs files. However, log processors such as Fluent Bit and fluentd fetch containers logs from a STDOUT pipeline which doesn’t exist on Windows containers. LogMonitor is an open-source plugin created by Microsoft to create a STDOUT pipeline within the Windows container so such tools can successfully fetch logs in the same way they do on a Linux environment.

For more instructions about how to use LogMonitor, see the GitHub repository or Microsoft post.

In the example below, we have a Dockerfile to build the Windows container image containing IIS and LogMonitor.

# We are using "windowsservercore" tag of the IIS image.
# Since this is a multi-arch image, we will get the base layer as the host instance version.
FROM mcr.microsoft.com/windows/servercore/iis:windowsservercore
 
# Set powershell as default shell
SHELL ["powershell", "-NoLogo", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
 
# Add X-Forward-For Header to IIS Default website log
RUN Add-WebConfigurationProperty -pspath 'MACHINE/WEBROOT/APPHOST' -filter "system.applicationHost/sites/siteDefaults/logFile/customFields" -name "." "-value @{logFieldName='X-Forwarded-For';sourceName='X-Forwarded-For';sourceType='RequestHeader'}" 
 
# Add STDOUT LogMonitor binary and config in json format
COPY LogMonitor.exe LogMonitorConfig.json 'C:\LogMonitor\'
WORKDIR /LogMonitor
 
ENTRYPOINT ["C:\\LogMonitor\\LogMonitor.exe", "powershell.exe"]
CMD ["C:\\ServiceMonitor.exe w3svc;"]

LogMonitorConfig.json

This sample LogMonitorConfig configuration retrieves all the logs files with the extension .log saved in C:\inetpub\logs and sub directories, including the IIS access logs.

{
  "LogConfig": {
    "sources": [
      {
        "type": "EventLog",
        "startAtOldestRecord": true,
        "eventFormatMultiLine": false,
        "channels": [
          {
            "name": "system",
            "level": "Error"
          }
        ]
      },
      {
        "type": "File",
        "directory": "c:\\inetpub\\logs",
        "filter": "*.log",
        "includeSubdirectories": true
      },
      {
        "type": "ETW",
        "providers": [
          {
            "providerName": "IIS: WWW Server",
            "ProviderGuid": "3A2A4E84-4C21-4981-AE10-3FDA0D9B0F83",
            "level": "Information"
          },
          {
            "providerName": "Microsoft-Windows-IIS-Logging",
            "ProviderGuid": "7E8AD27F-B271-4EA2-A783-A47BDE29143B",
            "level": "Information",
            "keywords": "0xFF"
          }
        ]
      }
    ]
  }
}

As the build completes, push the image to your Amazon ECR repository.

3. Create an Amazon ECS Windows container instance

Create an Amazon ECS Windows container instance.

To create an Amazon ECS instance;

1. Use the aws ssm get-parameters command to retrieve the AMI ID for the Region that hosts your VPC. For more information, see Retrieving Amazon ECS-Optimized AMI metadata.
2. Use the Amazon EC2 console to launch the instance.
   1. Open the Amazon EC2 Console at https://console.thinkwithwp.com/ec2/.
   2. From the navigation bar, select the Region to use.
   3. From the EC2 Dashboard, choose Launch instance.
   4. For Name, enter a unique name.
   5. For Application and OS Images (Amazon Machine Image), choose the AMI that you retrieved in the first step.
   6. For Instance type, choose t3.xlarge.
   7. For Key pair (login), choose a key pair.
   8. Under Network settings, for Security group, choose an existing security group, or create a new one.
   9. Under Network settings, for Auto-assign Public IP, select Enable.
   10. Under Advanced details, for IAM instance profile, choose ecsInstanceRole.
   11. Configure your Amazon ECS container instance with the following user data. Under Advanced Details, paste the following script into the User data field, replacing cluster_name with the name of your cluster (FluentBit-cluster in this tutorial).

       <powershell>
       Import-Module ECSTools
       Initialize-ECSAgent -Cluster cluster_name -EnableTaskENI -EnableTaskIAMRole -LoggingDrivers '["awslogs","fluentd"]'
       </powershell>
       

   12. When you are ready, select the acknowledgment field, and then choose Launch Instances.
   13. A confirmation page lets you know that your instance is launching. Choose View Instances to close the confirmation page and return to the console.

4. Configuration used for Fluent Bit

To ensure that the Windows customers can get started quickly, we have crafted default configurations for the following Fluent Bit output destinations which are present in the container images:

• Amazon CloudWatch
• Amazon S3
• Amazon Kinesis Data Streams
• Amazon Kinesis Data Firehose

In order to use the same, you need to ensure that:

• Your task AWS IAM role has sufficient permissions as mentioned in the Fluent Bit output plugin documentation.
• You have set the environment variables required for the specific configuration as documented here.
• You have overridden the entrypoint of the Windows Fluent Bit image with the specific configuration which you want to use.
• You have configured your Amazon ECS task definition for your application tasks to use fluentd logging driver and the following labels are configured in labels option of fluentd docker logging driver:
  • amazonaws.ecs.cluster
  • amazonaws.ecs.container-name
  • amazonaws.ecs.task-arn
  • amazonaws.ecs.task-definition-family
  • amazonaws.ecs.task-definition-version

These labels are added by Amazon ECS agent to each container.

In this post, we’ll show the above steps for Amazon CloudWatch default configuration but it can be changed easily to other destinations following similar mechanism. For more details about each default configuration, please visit our documentation here.

The default Amazon CloudWatch Windows Fluent Bit configuration is:

[SERVICE]
    Flush               5
    Log_Level           info
    Daemon              off

[INPUT]
    Name                forward
    Listen              0.0.0.0
    Port                24224
    Buffer_Chunk_Size   1M
    Buffer_Max_Size     6M
    Tag_Prefix          ecs.

# ECS agent adds the following log keys as labels to the docker container.
# We would use fluentd logging driver to add these to log record while sending it to Fluent Bit.
[FILTER]
    Name                modify
    Match               ecs.*
    Rename              com.amazonaws.ecs.cluster ecs_cluster
    Rename              com.amazonaws.ecs.container-name ecs_container_name
    Rename              com.amazonaws.ecs.task-arn ecs_task_arn
    Rename              com.amazonaws.ecs.task-definition-family ecs_task_definition_family
    Rename              com.amazonaws.ecs.task-definition-version ecs_task_definition_version

[FILTER]
    Name                rewrite_tag
    Match               ecs.*
    Rule                $ecs_task_arn ^([a-z-:0-9]+)/([a-zA-Z0-9-_]+)/([a-z0-9]+)$  out.$3.$ecs_container_name false
    Emitter_Name        re_emitted

[OUTPUT]
    Name                cloudwatch_logs
    Match               out.*
    region              region
    log_group_name      fallback-group
    log_group_template  /aws/ecs/$ecs_cluster.$ecs_task_definition_family
    log_stream_prefix   task-
    auto_create_group   On

Every log that gets into Fluent Bit has a tag that you specify, or it is automatically generated when you do not supply one. The tags can be used for routing the log entry to different filters and output plugins. For additional information, see Tag in the Fluent Bit Official Manual.

In this Fluent Bit configuration:

• We have configured forward input plugin such that it listens for incoming traffic on TCP port 24224. Each log entry received on that port would have a tag which would be modified by this plugin to prefix the same with ecs. string.
• As a next step in Fluent Bit internal pipeline, the log entry would be routed to modify filter using the Match regex. This filter replaces the keys within log record json to the format which can be consumed by Fluent Bit.
• The modified log entry is consumed by the rewrite_tag filter. This filter changes the tag of the log record to the format out.<TASK_ID>.<CONTAINER_NAME>.
• The new tag is routed to output cloudwatch_logs plugin, which creates the log groups and streams as described earlier in the post. This is accomplished using log_group_template and log_stream_prefix options of the Amazon CloudWatch output plugin.

5. Register a Windows Fluent Bit task definition which routes the logs to Amazon CloudWatch

Register a Windows Fluent Bit task definition which routes the logs to Amazon CloudWatch.

To register a task definition:

1. Create a file named fluent-bit.json with the following contents.

Replace the following variables:

• task-iam-role with the Amazon Resource Name (ARN) of your task AWS IAM role (fluentTaskRole created in step 1).
• region with the Region where your task runs.

{
  "family": "ecs-windows-fluent-bit",
  "taskRoleArn": "task-iam-role",
  "containerDefinitions": [
    {
      "name": "fluent-bit",
      "image": "public.ecr.aws/aws-observability/aws-for-fluent-bit:windowsservercore-latest",
      "cpu": 512,
      "portMappings": [
        {
          "hostPort": 24224,
          "containerPort": 24224,
          "protocol": "tcp"
        }
      ],
      "entryPoint": [
        "Powershell",
        "-Command"
      ],
      "command": [
        "C:\\entrypoint.ps1 -ConfigFile C:\\ecs_windows_forward_daemon\\cloudwatch.conf"
      ],
      "environment": [
        {
          "name": "AWS_REGION",
          "value": "region"
        }
      ],
      "memory": 512,
      "essential": true,
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/fluent-bit-logs",
          "awslogs-region": "region",
          "awslogs-stream-prefix": "flb",
          "awslogs-create-group": "true"
        }
      }
    }
  ],
  "memory": "512",
  "cpu": "512"
}

This task definition overrides the default entrypoint with a custom entrypoint command:

Powershell -Command C:\\entrypoint.ps1 -ConfigFile C:\\ecs_windows_forward_daemon\\cloudwatch.conf

Additionally, as documented in the configuration here, we have added the environment variable AWS_REGION to the container definition.

Note: This task definition exposes Fluent Bit container port 24224 to the host port 24224. Verify that this port is not open in your Amazon EC2 instance security group to prevent access from outside. For more information about security groups, refer to Amazon VPC docs here.

1. Run the following command to register the task definition.

Replace the following variables:

• • region with the Region where your task definition needs to be registered

aws ecs register-task-definition --cli-input-json file://fluent-bit.json --region region

You can list the task definitions for your account by running the list-task-definitions command. The output of displays the family and revision values that you can use together with run-task or start-task.

6. Run the registered ecs-windows-fluent-bit task definition

After you register a task definition for your account, you can run a task in the cluster. For this tutorial, you run one instance of the ecs-windows-fluent-bit:1 task definition in your FluentBit-cluster cluster. For production usage, you can consider running the ecs-windows-fluent-bit:1 task definition using Amazon ECS Service with Daemon scheduling strategy. This ensures that a single instance of Fluent Bit always runs on your container instances in the cluster.

To run a task

1. Run the following command to run the ecs-windows-fluent-bit:1 task definition that you registered in the previous step.

Replace the following variables:

• region with the Region where your task runs.

aws ecs run-task --cluster FluentBit-cluster --task-definition ecs-windows-fluent-bit:1 --region region

Note: Because ecs-windows-fluent-bit uses awslogs logging driver, your container instance needs to have the necessary permissions.

2. Run the following command to list your tasks.

Replace the following variables:

• region with the Region where your task runs.

aws ecs list-tasks --cluster FluentBit-cluster --region region

7. Register a Windows task definition with application containers which generates the logs

Register a task definition which generates the logs. This task definition deploys Windows container image containing IIS and LogMonitor which we had created in Step 3.

To register a task definition

1. Create a file named windows-iis-task.json with the following contents.

Replace the following variables:

• [Optional] iis-task-iam-role with the ARN of your IIS task AWS IAM role (iisTaskRole created in step 1).
• iis_container_image with the Windows container image created in Step 2.

Note: This task uses the default network mode; however, you can also use the awsvpc network mode with the task.

{
  "family": "windows-iis-task",
  "taskRoleArn": "iis-task-iam-role",
  "containerDefinitions": [
    {
      "name": "sample-container",
      "image": "iis_container_image",
      "cpu": 512,
      "memory": 512,
      "essential": true,
      "portMappings": [ {"protocol": "tcp", "containerPort": 80 } ],
      "logConfiguration": {
        "logDriver": "fluentd",
        "options": {
          "fluentd-address": "localhost:24224",
          "tag": "{{ index .ContainerLabels \"com.amazonaws.ecs.task-definition-family\" }}",
          "fluentd-async": "true",
          "labels": "com.amazonaws.ecs.cluster,com.amazonaws.ecs.container-name,com.amazonaws.ecs.task-arn,com.amazonaws.ecs.task-definition-family,com.amazonaws.ecs.task-definition-version"
        }
      }
    }
  ],
  "memory": "512",
  "cpu": "512"
}

In this task definition, we configure the container to use fluentd docker logging driver. The fluentd logging driver connects to port 24224 on which Fluent Bit forward input plugin is listening. Amazon ECS agent labels each Amazon ECS container with tags, which include the cluster name, task ARN, task definition family name, task definition revision number, and the task container name. We’ll pass these key-value labels to Fluent Bit.

2. Run the following command to register the task definition.

Replace the following variables:

• region with the Region where your task runs

aws ecs register-task-definition --cli-input-json file://windows-iis-task.json --region region

You can list the task definitions for your account by running the list-task-definitions command. The output of displays the family and revision values that you can use together with run-task or start-task.

8. Run the registered windows-iis-task task definition

After you register the windows-iis-task task definition, run multiple counts of the same in your FluentBit-cluster cluster.

To run a task

1. Run the windows-iis-task:1 task definition you registered in the previous step.

Replace the following variables:

• region with the Region where your task runs

aws ecs run-task --cluster FluentBit-cluster --task-definition windows-iis-task:1 --count 2 --enable-execute-command --region region

If you don’t want to access task containers using Amazon ECS Exec, then omit --enable-execute-command from above command.

2. Run the following command to list your tasks.

Replace the following variables:

• region with the Region where your task runs.

aws ecs list-tasks --cluster FluentBit-cluster --region region

9. [Optional] Access the IIS pods to generate logs

You can either wait for actual traffic to hit your Windows pod hosting the IIS webserver, or you can log into your container and force the logs.

9 Run the following command to log into your container-:

Replace the following variables:

• task_id with the task id of any of the windows-iis-task started in Step 8.
• region with the Region where your task runs.

aws ecs execute-command --cluster FluentBit-cluster --task task_id --container sample-container --interactive --region region --command powershell.exe

9 From inside the container, run the following command to hit the web server:

Invoke-WebRequest localhost -UseBasicParsing

10. Verify the logs on Amazon CloudWatch

In order to verify your Fluent Bit setup, check for the following log groups in the Amazon CloudWatch Console:

• /ecs/fluent-bit-logs – This contains the log stream corresponding to the Fluent Bit daemon container which is running on the container instance.
• /aws/ecs/FluentBit-cluster.windows-iis-task – This is the log group which corresponds to all the tasks launched for windows-iis-task task definition family inside FluentBit-cluster Amazon ECS cluster.
  • task-out.<FIRST_TASK_ID>.sample-container – This log stream contains all the logs generated by sample-container task container.
  • task-out.<SECOND_TASK_ID>.sample-container – This log stream contains all the logs generated by sample-container task container.

Inside the task-out.<TASK_ID>.sample-container log stream, each log entry would have the following fields:

{
    "source": "stdout",
    "ecs_task_arn": "arn:aws:ecs:us-west-2:173460346027:task/FluentBit-cluster/13eb23d9bed742478a37847cd0ed99a3",
    "container_name": "/ecs-windows-iis-task-1-sample-container-c0d0fee0a5ceeff91100",
    "ecs_cluster": "FluentBit-cluster",
    "ecs_container_name": "sample-container",
    "ecs_task_definition_version": "1",
    "container_id": "61f5e64561b77eaa769f848962e0d388f21f50378951bbbcfeba10162166ef5a",
    "log": "2022-11-04 19:58:44 ::1 GET / - 80 - ::1 Mozilla/5.0+(Windows+NT;+Windows+NT+10.0;+en-US)+WindowsPowerShell/5.1.17763.2931 - 200 0 0 1275 -\r",
    "ecs_task_definition_family": "windows-iis-task"
}

If you see these log groups and streams in the Amazon CloudWatch console, then Fluent Bit setup is verified.

Cleaning up

When you have finished, clean up the resources associated with it to avoid incurring charges for resources that you aren’t using.

To clean up the tutorial resources

1. Stop the windows-iis-task task and the ecs-windows-fluent-bit task. For more information, see Stopping tasks using the new console.
2. Run the following command to delete the log groups created in this post. For more information about deleting log groups see delete-log-group in the AWS Command Line Interface Reference.

   aws logs delete-log-group --log-group-name /ecs/fluent-bit-logs
   aws logs delete-log-group --log-group-name /aws/ecs/FluentBit-cluster.windows-iis-task
   

3. Run the following command to terminate the Amazon EC2 instance created in this tutorial. Replace INSTANCE_ID with the id of Amazon EC2 instance created in Step 3.

   aws ec2 terminate-instances --instance-ids <INSTANCE_ID>

4.  Run the following command to delete the private Amazon ECR repository. Replace ECR_REPOSITORY_NAME with the Amazon ECR repository used in Step 2.

   aws ecr delete-repository --repository-name <ECR_REPOSITORY_NAME> --force

5. Run the following command to detach the inline AWS IAM policies and delete the IAM roles created in this tutorial.

   aws iam delete-role-policy --role-name buildInstanceRole --policy-name ecr-push
   aws iam delete-role --role-name buildInstanceRole
   
   aws iam delete-role --role-name ecsInstanceRole
   
   aws iam delete-role-policy --role-name fluentTaskRole --policy-name fluent-bit-policy
   aws iam delete-role --role-name fluentTaskRole
   
   aws iam delete-role-policy --role-name iisTaskRole --policy-name ecs-exec-policy
   aws iam delete-role --role-name iisTaskRole
   

6. Run the following command to delete the Amazon ECS cluster.

   aws ecs delete-cluster --cluster FluentBit-cluster 

Conclusion

In this post, we showed you how to deploy Fluent Bit on your Windows instances in Amazon ECS.

Using Fluent Bit as a log router is a beneficial way to centralize logging across Amazon ECS for both Linux and Windows workloads. Fluent Bit can send the logs to various destination solutions allowing flexibility for customers.

Amazon ECS engineers are actively working to improve the experience for running Fluent Bit on Amazon ECS. For running Fluent Bit as a Daemon Service, track the progress in this roadmap issue. For the Firelens feature of Amazon ECS, track the progress in this roadmap issue.