Deploying Amazon EKS Windows managed node groups

Introduction

To help customers run their Windows applications in a more streamlined manner, we launched the support for Amazon EKS Managed Node Group (MNG) support for Windows containers on December 15, 2022. Amazon Elastic Kubernetes Service (Amazon EKS) MNGs automate the provisioning and lifecycle management of nodes (Amazon Elastic Compute Cloud [Amazon EC2] instances) for Amazon EKS Kubernetes clusters. With Amazon EKS MNGs, you don’t need to separately provision or register the Amazon EC2 instances that provide compute capacity to run your Kubernetes applications. You can create, automatically update, or terminate nodes for your cluster with a single operation. Amazon EKS automatically drains nodes using the Kubernetes API (Application Programming Interface) during terminations or updates. Updates respect the pod disruption budgets that you set for your pods.

To help align operations teams, Windows MNGs are enabled using the same workflows and tools as Linux MNGs. Today, we support full and core AMI (Amazon Machine Image) family versions of Windows Server 2019 and 2022. To get started, you can use eksctl or the AWS Management Console to easily deploy a new MNG using the following AMIs!

WINDOWS_CORE_2019_x86_64
WINDOWS_FULL_2019_x86_64
WINDOWS_CORE_2022_x86_64
WINDOWS_FULL_2022_x86_64

With the ability to use a custom launch template, teams can continue to implement common requirements like integration with Active Directory and container image caching strategies. There are no additional costs to use Amazon EKS MNGs, you only pay for the AWS resources you provision.

This post provides step-by-step instructions on how to get started with Windows-based MNGs.

Getting started with managed node groups

Prerequisites

Create an AWS Identity and Access Management (AWS IAM) user or role that has permissions to create and managed Amazon EKS This AWS IAM security principal is used in the Getting Started section below for AWS Command Line Interface (AWS CLI) configuration
Follow the Getting Started section in the Amazon EKS documentation to install aws cli, kubectl, and eksctl on your dev machine
Ensure you are using eksctl v0.124.0 or higher to operate Windows MGNs

Walkthrough

1. Create a new Amazon EKS Cluster with Windows-managed node groups

eksctl makes it easy to setup and manage an Amazon EKS cluster with Windows MNGs. Whether you are provisioning a new cluster or adding onto an existing, eksctl can help. eksctl automatically patches the ConfigMap to enable Windows IP address management when a Windows node group is created.

1.1 Create a file that contains the following Amazon EKS cluster configuration details and name it as eks-windows-cluster.yaml

Note: This creates a cluster in us-west-2 region using Kubernetes version 1.23. If you are planning to use Windows Server 2022, then please set your cluster version to 1.23 or higher. If you are planning to utilize a different region, then you’ll need to adjust that here.

1.2 In this example, we configure taints on the Windows MNG nodes. If you are planning to use a mixed cluster with Windows and Linux nodes, then taints can help ensure Linux pods are not scheduled onto Windows nodes. If you would like to use an existing virtual private cloud (VPC), you ‘ll specify a vpc and subnet block within the cluster configuration. To learn more, visit creating and managing clusters with eksctl.


---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: eks-windows-mng-demo
  region: us-west-2
  version: '1.23'
managedNodeGroups:
  - name: linux-ng
    instanceType: t3.large
    minSize: 1
  - name: windows-managed-ng-2022
    amiFamily: WindowsServer2022FullContainer
    instanceType: m5.large
    volumeSize: 50
    minSize: 2
    maxSize: 4
    taints:
      - key: os
        value: "windows"
        effect: NoSchedule

availabilityZones: ["us-west-2a", "us-west-2b"]

1.3 Run the following command to create your Amazon EKS cluster with a Windows MNG and wait for the provisioning process to complete.

eksctl create cluster -f eks-windows-cluster.yaml

Creating your Amazon EKS cluster and Windows-managed node group eksctl switches the context that kubectl uses to become the cluster that you just provisioned. This allows you to manage your new cluster and provision a sample application.

kubectl config current-context

Update kubectl context

1.4 Navigate to the AWS Management Console to see the created Windows MNG and the associated AWS resources automatically created, including the auto scaling group and launch template. The console details information about your nodes including running pods, kubelet version, and AMI release version. If you review the launch template details, then you’ll see that the AMI and user data are missing. These configuration items are managed behind the scenes ,unless you are using a custom launch template.

Windows-Managed Node Group Console experience

2. Deploy a sample application

Now that our Amazon EKS cluster is online and running a Windows MNG, we can deploy a sample application.

2.1 Create a file that contains the sample application configuration details and save it as windows-server-iis-ltsc2022.yaml

Note: We include a toleration within our deployment manifest that allows the scheduler to schedule pods with matching taints, which in our case, is os=windows. Node selector is applied within a single template asking the scheduler to run the pods on a set of nodes. NoSchedule taints affect all pods and instruct the scheduler to block scheduling pods that don’t have a matching toleration.


apiVersion: apps/v1
kind: Deployment
metadata:
  name: windows-server-iis-ltsc2022
  namespace: windows
spec:
  selector:
    matchLabels:
      app: windows-server-iis-ltsc2022
      tier: backend
      track: stable
  replicas: 2
  template:
    metadata:
      labels:
        app: windows-server-iis-ltsc2022
        tier: backend
        track: stable
    spec:
      containers:
      - name: windows-server-iis-ltsc2022
        image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2022
        ports:
        - name: http
          containerPort: 80
        imagePullPolicy: IfNotPresent
        command:
        - powershell.exe
        - -command
        - "Add-WindowsFeature Web-Server; Invoke-WebRequest -UseBasicParsing -Uri 'https://dotnetbinaries.blob.core.windows.net/servicemonitor/2.0.1.6/ServiceMonitor.exe' -OutFile 'C:\\ServiceMonitor.exe'; echo '<html><body><br/><br/><H1>Our first pods running on Windows managed node groups! Powered by Windows Server LTSC 2022.<H1></body><html>' > C:\\inetpub\\wwwroot\\iisstart.htm; C:\\ServiceMonitor.exe 'w3svc'; "
      nodeSelector:
        kubernetes.io/os: windows
      tolerations:
          - key: "os"
            operator: "Equal"
            value: "windows"
            effect: "NoSchedule"
---
apiVersion: v1
kind: Service
metadata:
  name: windows-server-iis-ltsc2022-service
  namespace: windows
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: windows-server-iis-ltsc2022
    tier: backend
    track: stable
  sessionAffinity: None
  type: LoadBalancer

2.2 We create a new namespace to deploy our pods into called windows.

kubectl create namespace windows

2.3 Run the following command to deploy our sample application. The pod takes approximately 5 minutes to download and extract the IIS (Internet Information Services) base image.

kubectl apply -f windows-server-iis-ltsc2022.yaml

2.4 You can check the state of your deployment, service, and pods by running the following command.

kubectl -n windows get svc,deploy,pods

2.5 With your pods in a running state, you can retrieve the Load balancer URL from your service.

kubectl -n windows get svc -o jsonpath='{.items[].status.loadBalancer.ingress[].hostname}'

2.6 Copy the Load Balancer URL into your browser of choice and you should be presented with a basic HTML site.

Sample app deployment verification

3. Adding a Windows managed node group to an existing cluster

If you are already using Windows self-managed node groups and plan to switch to Windows MNGs, then you can provision a new MNG by using the following eksctl command. To learn more, visit migrating to a new node group for detailed instructions.

eksctl create nodegroup --cluster eks-windows-mng-demo --region us-west-2 --name windows-mng --node-ami-family WindowsServer2022FullContainer --node-type m5.large --nodes 1 --nodes-min 1 --nodes-max 10

4. Updating a Windows managed node group

Updating a MNG makes it easy to update the existing node group with the latest patched AMI in your cluster’s Region. You can use the same workflows to upgrade your nodes to newer versions of Kubernetes. The upgrade process is well defined for eksctl and the AWS Management Console. To understand the managed node update behavior in more depth, visit the managed node update behavior documentation. In our example, you use the following command to issue a MNG update to the latest patched AMI.

eksctl upgrade nodegroup --name windows-managed-ng-2022 --cluster eks-windows-mng-demo --region us-west-2

Updating a managed node group using eksctl

5. Design considerations for Windows Managed Node Groups

If the benefits of MNGs make sense for your organization, then here are a few things to keep in mind:

Patching is a shared responsibility. You’ll use the update-nodegroup-version API to replace the AMI used in the launch template with the latest AMI in your clusters region. The latest Amazon EKS optimized AMI versions are published monthly. You can use AWS Systems Manager Parameter Store Public parameters to locate the latest AMI ID for your region. The entire upgrade process can be found in the Amazon EKS Public documentation.
If you use a custom Launch Template you’ll need to add back in the bootstrap script to register your nodes with your cluster. You can find the steps to do this in the specifying an AMI public documentation.
Using a custom Launch Template allows customers to implement design requirements like deploying Active Directory Group Managed Service Accounts (gMSA), Implementing a caching strategy for their container images, and enabling Direct Server Return for improved application performance. There are additional patching steps when using a custom AMI that need to be followed.

For more considerations, visit the MNGs concepts page.

Cleaning up

When you’ve finished, clean up the resources associated with the example deployment to avoid incurring unwanted charges.

First, we need to remove the service we created. To do so we must get the name using kubectl.

kubectl get svc -n windows

Take the name of the service and input it into the next command.

kubectl delete svc <NAMEOFSERVICE> -n windows

Finally, we remove the cluster and the associated MNGs with a single command.

eksctl delete cluster --name eks-windows-mng-demo --region us-west-2

If this command times out, then you can run the command above again to show the cluster has been successfully removed.

Conclusion

In this post, we showed you how to use Windows MNGs to remove undifferentiated heavy lifting of managing the provisioning and lifecycle of Windows Kubernetes nodes. We aligned operations to use the same tools for Linux and Windows. The services provide node updates and terminations, which automatically drain nodes to ensure that your applications stay available. They are free to use minus the resources you provision. With the added flexibility of using custom launch templates, MNGs should be the default provisioning method organizations use in the future.

Containers