Advanced EKS concepts

Resources

Kubernetes Scaling

Overview:

In EC2, an autoscaling group might operate and scale if CPU > 50% (the specified policy).

In k8s, the Horizontal Pod Autoscaling will run based on the pod CPU (as opposed to the node).

It keeps CPU utilization of the pod within limits. What happens if the node or max pod is reached? Another autoscaler comes into play: the cluster autoscaler. It will horizontally scale the nodes.

The resources part of a manifest file for a deployment that helps with the autoscaling.

resources:
  requests:
    cpu: 500m # also 0.5
    memory: 256Mi # 1 mebibyte = 1.04848 megabyte
  limits:
    cpu: 1000m # also 1
    memory: 512Mi
copy

1 vCPU = 1000m (milicore)

One pod requesting CPU of 500m (half of 1 vCPU)

One pod CPU limit 1000m (1 vCPU)

So far, there is no HPA declaration. We need it in the manifest file as well.

# V1 used in demo
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: nginxext
  namespace: default
spec:
  minReplicas: 1
  maxReplicas: 10
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx
  # 50% of 500m (denoted above) = 250m before scaling
  targetCPUUtilizationPercentage: 50
---
# V2
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: nginxext
  namespace: default
spec:
  minReplicas: 1
  maxReplicas: 5
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx
  metrics:
    - type: External
      external:
        metricName: nginx.net.request_per_s
        metricSelector:
          matchLabels:
            kube_container_name: nginx
        targetAverageValue: 50
copy

HPA Demo

There is a Kubernetes Metrics server to help see the data. You also need node larger than t3.micro.

The demo itself runs a loop to scale up the pods.

Run kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml to add the metrics server.

Once installed, you can see the utilisation on the targets.

Cluster AutoScaler in EKS

Resource Video

This handles the Node scaling. If you request many replicas that has desired mebibytes for resources that is larger than the single node, then even with no CPU usage the node with scale.

Vertical Pod Autoscaler

DO NOT USE IN PRODUCTION. It restarted the pod.

Goldilocks is a tool used with VPA to help with defining what resources should be used.

We can use a GitHub example to setup VPA.

Namespace - What and Why

If two different teams accidentally deployed two apps of the same name in the same namespace, one will override the other and if someone deploys a badly optimised app, it will affect the other pods.

You can think of them as isolated, virtual clusters.

You can also assign process power per namespace (other apps cannot accidentally exhaust the other namespace).

Ingress - What and Why

The example given is what happens when you deploy multiple services under different subdomains.

This can be solved with ingress. It is an API service to help manage internal and external http(s) access to Kubernetes services running in a cluster.

Supports:

Find more on ALB here.

In the example, we have the following setup:

Then we create an ingress controller in two parts:

An example from the AWS Load Balancer controller workshop lesson:

---
apiVersion: v1
kind: Namespace
metadata:
  name: game-2048
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: game-2048
  name: deployment-2048
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: app-2048
  replicas: 5
  template:
    metadata:
      labels:
        app.kubernetes.io/name: app-2048
    spec:
      containers:
        - image: alexwhen/docker-2048
          imagePullPolicy: Always
          name: app-2048
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  namespace: game-2048
  name: service-2048
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  type: NodePort
  selector:
    app.kubernetes.io/name: app-2048
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  namespace: game-2048
  name: ingress-2048
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  rules:
    - http:
        paths:
          - path: /*
            backend:
              serviceName: service-2048
              servicePort: 80
copy

If we changed the path, it would add a rule to the existing LoadBalancer.

The Ingress Controller requires the correct IAM Policy. Require Service Account and IAM Role for the alb-ingress-controller pod.

EKS Cost Optimization

How to deal with expensive bills. The 4 commandments:

Open Source Tools to help with pricing

Third Party

EKS Tools Ecosystem

Stricly popular opensource and 3rd party tools.

Devops

Security

Ingress

Logging

Monitoring

Cost optimisation

EKS Public Roadmap

If you go to aws/containers-roadmap on GitHubb, you can see the public roadmap.