如何使用 Terraform 配置 EKS ALB

Question

我很难让 EKS 向公共互联网公开 IP 地址。我需要自己设置 ALB，还是作为 EKS 集群的一部分免费获得？如果必须自己做，是需要在terraform模板文件中定义还是在kubernetes对象yaml中定义？

这是我在 Terraform 中定义的 EKS 集群以及我认为所需的权限。

// eks.tf

resource "aws_iam_role" "eks_cluster_role" {
  name = "${local.env_name}-eks-cluster-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = {
          Service = "eks.amazonaws.com"
        },
        Action = "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_cluster_role.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSVPCResourceController" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.eks_cluster_role.name
}

resource "aws_kms_key" "eks_key" {
  description             = "EKS KMS Key"
  deletion_window_in_days = 7
  enable_key_rotation     = true

  tags = {
    Environment = local.env_name
    Service     = "EKS"
  }
}

resource "aws_kms_alias" "eks_key_alias" {
  target_key_id = aws_kms_key.eks_key.id
  name          = "alias/eks-kms-key-${local.env_name}"
}

resource "aws_eks_cluster" "eks_cluster" {
  name                      = "${local.env_name}-eks-cluster"
  role_arn                  = aws_iam_role.eks_cluster_role.arn
  enabled_cluster_log_types = ["api", "audit", "authenticator", "controllerManager", "scheduler"]

  vpc_config {
    subnet_ids = [aws_subnet.private_a.id, aws_subnet.private_b.id]
  }

  encryption_config {
    resources = ["secrets"]

    provider {
      key_arn = aws_kms_key.eks_key.arn
    }
  }

  tags = {
    Environment = local.env_name
  }
}

resource "aws_iam_role" "eks_node_group_role" {
  name = "${local.env_name}-eks-node-group"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = {
          Service = "ec2.amazonaws.com"
        },
        Action = "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "eks-node-group-AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.eks_node_group_role.name
}

resource "aws_iam_role_policy_attachment" "eks-node-group-AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.eks_node_group_role.name
}

resource "aws_iam_role_policy_attachment" "eks-node-group-AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.eks_node_group_role.name
}

resource "aws_eks_node_group" "eks_node_group" {
  instance_types  = var.node_group_instance_types
  node_group_name = "${local.env_name}-eks-node-group"
  node_role_arn   = aws_iam_role.eks_node_group_role.arn
  cluster_name    = aws_eks_cluster.eks_cluster.name
  subnet_ids      = [aws_subnet.private_a.id, aws_subnet.private_b.id]

  scaling_config {
    desired_size = 1
    max_size     = 1
    min_size     = 1
  }

  // Ensure that IAM Role permissions are created before and deleted after EKS Node Group handling.
  // Otherwise, EKS will not be able to properly delete EC2 Instances and Elastic Network Interfaces.
  depends_on = [
    aws_iam_role_policy_attachment.eks-node-group-AmazonEC2ContainerRegistryReadOnly,
    aws_iam_role_policy_attachment.eks-node-group-AmazonEKS_CNI_Policy,
    aws_iam_role_policy_attachment.eks-node-group-AmazonEKSWorkerNodePolicy,
  ]

这是我的 kubernetes 对象 yaml：

# hello-kubernetes.yaml

apiVersion: v1
kind: Service
metadata:
  name: hello-kubernetes
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: hello-kubernetes
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-kubernetes
spec:
  replicas: 3
  selector:
    matchLabels:
      app: hello-kubernetes
  template:
    metadata:
      labels:
        app: hello-kubernetes
    spec:
      containers:
      - name: hello-kubernetes
        image: paulbouwer/hello-kubernetes:1.9
        ports:
        - containerPort: 8080
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: hello-ingress
spec:
  backend:
    serviceName: hello-kubernetes
    servicePort: 80

我已经运行 terraform apply 并且集群已启动并正在运行。我已经安装了eksctl 和kubectl 并运行kubectl apply -f hello-kubernetes.yaml。 pod、服务和入口似乎运行良好。

$ kubectl get pods
NAME                                READY   STATUS             RESTARTS   AGE
hello-kubernetes-6cb7cd595b-25bd9   1/1     Running            0          6h13m
hello-kubernetes-6cb7cd595b-lccdj   1/1     Running            0          6h13m
hello-kubernetes-6cb7cd595b-snwvr   1/1     Running            0          6h13m

$ kubectl get services
NAME               TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
hello-kubernetes   LoadBalancer   172.20.102.37   <pending>     80:32086/TCP   6h15m

$ kubectl get ingresses
NAME            CLASS    HOSTS   ADDRESS   PORTS   AGE
hello-ingress   <none>   *                 80      3h45m

我缺少什么以及它属于哪个文件？

Answer 1

通常的方法是放置一个 ALB 并将流量重定向到 EKS 集群，并使用 ALB 入口控制器对其进行管理。这个入口控制器将充当集群和您的 ALB 之间的通信，这是非常直接的 AWS 文档

EKS w/ALB

如果 ALB 不适合您的应用程序需求，其他解决方案可能是使用带有 NLB 的 NGINX 入口控制器，如下文所述

NGINX w/NLB

Answer 2

这也发生在我身上，在所有设置之后，我无法看到入口地址。调试此问题的最佳方法是检查入口控制器的日志。你可以这样做：

使用以下命令获取 Ingress 控制器 po 名称：kubectl get po -n kube-system 使用以下命令检查 po 的日志：kubectl logs -n kube-system 这将为您指出您看不到地址的确切问题。

如果您没有找到任何以 ingress 名称运行的 po，那么您必须先创建 ingress 控制器。

Answer 3

您需要按照installation instructions安装AWS Load Balancer Controller；首先您需要创建 IAM 角色和权限，这可以通过 Terraform 完成；那么你需要应用 Kubernetes Yaml 来将控制器安装到你的集群中，这可以通过 Helm 或 Kubectl 来完成。

您还需要注意 subnet tagging ，例如创建面向公共或私有的负载均衡器。