Exposed K8S Apis

A finding by the Shadow Foundation uncovered close to half a million k8s endpoints on the internet which can be targets to exploits. One factor is that by default these clusters are built with public IPs since cloud providers are outside your network and not all companies can have ExpressRoutes or dedicated point to point connectivity. To increase the security and have easier routing of your kubernetes cluster you can create a private cluster. In Azure Kubernetes Service the private cluster assigns an internal IP to your k8s API but NGINX defaults to external IP so in this article I walk through configuring NGINX to have internal IPs as well to keep it all inside the network.

I. Helm Install controller

At the time of this doc I was installing an ingress on K8S 1.21.9 so had to use the beta endpoints to install the ingress controller in Azure Kubernetes Service so that it can use an internal IP. I used the command below so you can replace the names and IPs accordingly.

helm install ingress-nginx/ingress-nginx --namespace something --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-internal"="true" --set controller.service.loadBalancerIP="192.168.1.200" --generate-name

II. Ingress Annotation

Here are the ingress annotations I used to get it up and running. I created an A Record so that I can use a cert on the ingress so I added the host and TLS settings here as well.

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-world-ingress-static
  annotations:

    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS  
spec:
  rules:
  - host: dnsname.ingeniumcode.io
    http:
      paths:
      - backend:
          serviceName: serviceNameofpod
          servicePort: ### 
        path: / 
  tls:
    - hosts:
      - dnsname.ingeniumcode.io
      secretName: k8sSecretCert

III. Sources

https://docs.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli

Exposed K8S Api Endpoints: https://threatpost.com/380k-kubernetes-api-servers-exposed-to-public-internet/179679/