Containers

App Mesh Integration with AWS ALB Ingress Controller

NOTICE: October 04, 2024 – This post no longer reflects the best guidance for configuring a service mesh with Amazon EKS and its examples no longer work as shown. Please refer to newer content on Amazon VPC Lattice.

——–

AWS App Mesh is a service mesh that provides application-level networking to make it easy for your services to communicate with each other across multiple types of compute infrastructure. App Mesh standardizes how your services communicate, giving you end-to-end visibility and ensuring high-availability for your applications.

The AWS ALB Ingress controller is a controller that triggers the creation of an ALB and the necessary supporting AWS resources whenever a Kubernetes user declares an Ingress resource on the cluster. The Ingress resource uses the ALB to route HTTP[s] traffic to different endpoints within the cluster.

With App Mesh, internal traffic (aka east-west traffic) in EKS is managed by Envoy side car which is controlled by App Mesh control plane, however external access (aka north-south traffic) is not managed by App Mesh, there are several options to connect the north-south traffic to east-west traffic as following:

  • Gloo as Ingress gateway.
  • ALB Ingress Controller with the gateway application in App Mesh.
  • Ingress-nginx as ingress to App Mesh, the ingress-nginx-controller will be set as virtual node in mesh.

This blog will illustrate how to setup AWS ALB Ingress Controller with the AWS App Mesh example application Color App

How it works

 

  • ALB: AWS Application Load Balancer, created and maintained by ALB ingress controller, traffic will be routed to Color Gateway service. The target group of ALB will be pointed to each pod of gateway in ALB ingress controller IP mode.
  • Color Gateway: A simple http service written in Go that is exposed to external clients and that responds to http://service-name:port/color. The gateway responds with a color retrieved from color-teller and a histogram of colors observed at the server that responded up to the point when you made the request.
  • ColorTeller – A simple http service written in Go that is configured to return a color. Multiple variants of the service are deployed. Each service is configured to return a specific color.

Walkthrough

1. Create EKS cluster with eksctl

Please follow this document to install and configure AWS CLI, kubectl, and eksctl tools. Then create EKS cluster and worker nodes by following these commands:

export REGION=us-west-2
eksctl create cluster --region $REGION --name appmesh-alb --appmesh-access

It may take 15 mins to create EKS cluster and worker nodes. After it’s finished, you may run following command to validate EKS cluster:

kubectl get svc

expected output should be like this:

NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.100.0.1   <none>        443/TCP   10m

now create an IAM OIDC provider and associate it with the cluster:

eksctl utils associate-iam-oidc-provider --cluster=appmesh-alb --approve

2. Install appmesh controller

Run following kubectl command install appmesh controller:

kubectl apply -f https://raw.githubusercontent.com/aws/aws-app-mesh-controller-for-k8s/master/deploy/all.yaml

check controller resources:

kubectl rollout status deployment app-mesh-controller -n appmesh-system
kubectl get crd

expected output should be like this:

deployment "app-mesh-controller" successfully rolled out
NAME                               CREATED AT
eniconfigs.crd.k8s.amazonaws.com   2019-09-28T13:32:59Z
meshes.appmesh.k8s.aws             2019-09-28T13:44:17Z
virtualnodes.appmesh.k8s.aws       2019-09-28T13:44:17Z
virtualservices.appmesh.k8s.aws    2019-09-28T13:44:17Z

3. Install appmesh side car injector

export MESH_NAME=color-mesh
export MESH_REGION=$REGION
curl https://raw.githubusercontent.com/aws/aws-app-mesh-inject/master/scripts/install.sh | bash

4. install color teller sample application

kubectl apply -f https://raw.githubusercontent.com/aws/aws-app-mesh-controller-for-k8s/master/examples/color.yaml

5. install AWS ALB Ingress controller

Run following commands to install ALB ingress controller, you may refer to this blog to get detail explanation for each command:

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.4/docs/examples/rbac-role.yaml
PolicyARN=$(aws iam create-policy \
    --policy-name ALBIngressControllerIAMPolicy \
    --policy-document https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.4/docs/examples/iam-policy.json \
    | jq -r ".Policy.Arn")
eksctl create iamserviceaccount \
    --cluster=appmesh-alb \
    --namespace=kube-system \
    --name=alb-ingress-controller \
    --attach-policy-arn=$PolicyARN \
    --override-existing-serviceaccounts \
    --approve
curl -sS "https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.4/docs/examples/alb-ingress-controller.yaml" \
     | sed "s/# - --cluster-name=devCluster/- --cluster-name=appmesh-alb/g" \
     | kubectl apply -f -

6. Create ingress for color teller application

create new file appmesh-alb-ingress.yaml with following content:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/healthcheck-path: /color
    kubernetes.io/ingress.class: alb
  name: colorgateway
  namespace: appmesh-demo
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: colorgateway
          servicePort: 9080
        path: /color

then run following command to create ingress resource

kubectl apply -f appmesh-alb-ingress.yaml

it may take a few minutes to create new ALB resource and make it ready, you can get the endpoint of ALB:

kubectl -n appmesh-demo describe ing/colorgateway

then you can access the sample application with above ALB endpoint:

http://{ALB_endpoint}/color

expected output should be like this:

{"color":"white", "stats": {"black":0.33,"blue":0.33,"white":0.33}}

7. Clean-up

kubectl delete -f appmesh-alb-ingress.yaml
eksctl delete cluster appmesh-alb

Conclusion

Using ALB as ingress of App Mesh is one of the simplest ways to route external traffic into mesh, the deployment process is same as deploying ALB Ingress Controller in EKS or self-build Kubernetes. In the Color App sample application, gateway is an App Mesh virtual node, which exposed as the entry of the mesh, since ALB is not in the mesh, it is necessary to have this gateway bridge external and internal traffic.

Reference: