我在这里添加我的答案是因为我偶然发现了同样的问题,并且接受了答案(以及上面的其他答案),但没有提供问题的完整解决方案 - 没有代码示例。它们只是我必须用来进行更深入研究的指导方针。有些问题确实很容易被忽略 - 如果没有代码示例,很难得出结论(尤其是在创建 IAM 角色时与条件/StringEquals 相关的部分)
创建将与角色绑定的服务帐户的全部目的是从集群内创建 aws 资源的可能性(最常见的情况是负载均衡器,或用于将日志推送到 cloudwatch 的角色)。
所以,问题是我们如何才能做到这一点,使用 terraform,而不是使用 eks 命令。
我们需要做的是:
- 创建 eks oidc(可以使用 terraform 完成)
- 创建 AWS IAM 角色(可以使用 terraform 完成),创建和使用适当的策略
- 创建 k8s 服务帐户(需要使用 kubectl 命令完成 - 或使用 kubernetes 资源使用 terraform 完成
- 使用我们创建的 IAM 角色注释 k8s 服务帐户(这意味着我们正在将 k8s 服务帐户与 IAM 角色相关联)
完成此设置后,我们的 k8s 服务帐户将具有 k8s 集群角色和 k8s 集群角色绑定(这将允许该服务帐户在 k8s 中执行操作),并且我们的 k8s 服务帐户将附加 IAM 角色,这将允许在集群外执行操作(例如创建 aws 资源)
让我们从它开始吧。下面的假设是您的 eks 集群已经使用 terraform 创建,我们专注于在 eks 集群周围创建工作服务帐户所需的资源。
创建 eks_oidc
### First we need to create tls certificate
data "tls_certificate" "eks-cluster-tls-certificate" {
url = aws_eks_cluster.eks-cluster.identity[0].oidc[0].issuer
}
# After that create oidc
resource "aws_iam_openid_connect_provider" "eks-cluster-oidc" {
client_id_list = ["sts.amazonaws.com"]
thumbprint_list = [data.tls_certificate.eks-cluster-tls-certificate.certificates[0].sha1_fingerprint]
url = aws_eks_cluster.eks-cluster.identity[0].oidc[0].issuer
}
现在,让我们使用所有必要的策略创建 AWS IAM 角色。
以下 Terraform 声明性代码将:
- 创建 ALBIngressControllerIAMPolicy 策略
- 创建 alb-ingress-controller-role 角色
- 将 ALBIngressControllerIAMPolicyr 策略附加到 alb-ingress-controller-role 角色
- 将现有的 AmazonEKS_CNI_Policy 策略附加到角色
请注意,我在这里使用后缀作为 alb 入口控制器,因为这是我在集群中的角色的主要用途。您可以更改角色的策略名称,也可以更改策略的权限访问权限,以及您打算使用它执行的操作。
data "aws_caller_identity" "current" {}
locals {
account_id = data.aws_caller_identity.current.account_id
eks_oidc = replace(replace(aws_eks_cluster.eks-cluster.endpoint, "https://", ""), "/\\..*$/", "")
}
# Policy which will allow us to create application load balancer from inside of cluster
resource "aws_iam_policy" "ALBIngressControllerIAMPolicy" {
name = "ALBIngressControllerIAMPolicy"
description = "Policy which will be used by role for service - for creating alb from within cluster by issuing declarative kube commands"
policy = jsonencode({
Version = "2012-10-17",
Statement = [
{
Effect = "Allow",
Action = [
"elasticloadbalancing:ModifyListener",
"wafv2:AssociateWebACL",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DescribeInstances",
"wafv2:GetWebACLForResource",
"elasticloadbalancing:RegisterTargets",
"iam:ListServerCertificates",
"wafv2:GetWebACL",
"elasticloadbalancing:SetIpAddressType",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:SetWebAcl",
"ec2:DescribeInternetGateways",
"elasticloadbalancing:DescribeLoadBalancers",
"waf-regional:GetWebACLForResource",
"acm:GetCertificate",
"shield:DescribeSubscription",
"waf-regional:GetWebACL",
"elasticloadbalancing:CreateRule",
"ec2:DescribeAccountAttributes",
"elasticloadbalancing:AddListenerCertificates",
"elasticloadbalancing:ModifyTargetGroupAttributes",
"waf:GetWebACL",
"iam:GetServerCertificate",
"wafv2:DisassociateWebACL",
"shield:GetSubscriptionState",
"ec2:CreateTags",
"elasticloadbalancing:CreateTargetGroup",
"ec2:ModifyNetworkInterfaceAttribute",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"ec2:RevokeSecurityGroupIngress",
"elasticloadbalancing:DescribeTargetGroupAttributes",
"shield:CreateProtection",
"acm:DescribeCertificate",
"elasticloadbalancing:ModifyRule",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:DescribeRules",
"ec2:DescribeSubnets",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"waf-regional:AssociateWebACL",
"tag:GetResources",
"ec2:DescribeAddresses",
"ec2:DeleteTags",
"shield:DescribeProtection",
"shield:DeleteProtection",
"elasticloadbalancing:RemoveListenerCertificates",
"tag:TagResources",
"elasticloadbalancing:RemoveTags",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:DescribeListeners",
"ec2:DescribeNetworkInterfaces",
"ec2:CreateSecurityGroup",
"acm:ListCertificates",
"elasticloadbalancing:DescribeListenerCertificates",
"ec2:ModifyInstanceAttribute",
"elasticloadbalancing:DeleteRule",
"cognito-idp:DescribeUserPoolClient",
"ec2:DescribeInstanceStatus",
"elasticloadbalancing:DescribeSSLPolicies",
"elasticloadbalancing:CreateLoadBalancer",
"waf-regional:DisassociateWebACL",
"elasticloadbalancing:DescribeTags",
"ec2:DescribeTags",
"elasticloadbalancing:*",
"elasticloadbalancing:SetSubnets",
"elasticloadbalancing:DeleteTargetGroup",
"ec2:DescribeSecurityGroups",
"iam:CreateServiceLinkedRole",
"ec2:DescribeVpcs",
"ec2:DeleteSecurityGroup",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:SetSecurityGroups",
"elasticloadbalancing:DescribeTargetGroups",
"shield:ListProtections",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:DeleteListener"
],
Resource = "*"
}
]
})
}
# Create IAM role
resource "aws_iam_role" "alb-ingress-controller-role" {
name = "alb-ingress-controller"
assume_role_policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Federated": "${aws_iam_openid_connect_provider.eks-cluster-oidc.arn}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"${replace(aws_iam_openid_connect_provider.eks-cluster-oidc.url, "https://", "")}:sub": "system:serviceaccount:kube-system:alb-ingress-controller",
"${replace(aws_iam_openid_connect_provider.eks-cluster-oidc.url, "https://", "")}:aud": "sts.amazonaws.com"
}
}
}
]
}
POLICY
depends_on = [aws_iam_openid_connect_provider.eks-cluster-oidc]
tags = {
"ServiceAccountName" = "alb-ingress-controller"
"ServiceAccountNameSpace" = "kube-system"
}
}
# Attach policies to IAM role
resource "aws_iam_role_policy_attachment" "alb-ingress-controller-role-ALBIngressControllerIAMPolicy" {
policy_arn = aws_iam_policy.ALBIngressControllerIAMPolicy.arn
role = aws_iam_role.alb-ingress-controller-role.name
depends_on = [aws_iam_role.alb-ingress-controller-role]
}
resource "aws_iam_role_policy_attachment" "alb-ingress-controller-role-AmazonEKS_CNI_Policy" {
role = aws_iam_role.alb-ingress-controller-role.name
policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
depends_on = [aws_iam_role.alb-ingress-controller-role]
}
上面执行terraform后,就成功创建了terraform部分资源。现在我们需要创建一个 k8s 服务帐户并将 IAM 角色与该服务帐户绑定。
创建集群角色、集群角色绑定和服务帐号
你可以使用
https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/master/docs/examples/rbac-role.yaml
直接(来自 master 分支),但考虑到我们需要注释 iam arn,我倾向于下载此文件,对其进行更新并将其存储为我的 kubectl 配置文件中的更新。
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/name: alb-ingress-controller
name: alb-ingress-controller
rules:
- apiGroups:
- ""
- extensions
resources:
- configmaps
- endpoints
- events
- ingresses
- ingresses/status
- services
- pods/status
verbs:
- create
- get
- list
- update
- watch
- patch
- apiGroups:
- ""
- extensions
resources:
- nodes
- pods
- secrets
- services
- namespaces
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/name: alb-ingress-controller
name: alb-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: alb-ingress-controller
subjects:
- kind: ServiceAccount
name: alb-ingress-controller
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/name: alb-ingress-controller
name: alb-ingress-controller
namespace: kube-system
annotations:
eks.amazonaws.com/role-arn: <ARN OF YOUR ROLE HERE>
...
在此文件的底部,您会注意到需要放置 ANR 角色的注释。
仔细检查
就是这样。之后,您就有了一个与 iam 角色相关联的 k8s 服务帐户。
检查:
kubectl get sa -n kube-system
kubectl describe sa alb-ingress-controller -n kube-system
你应该得到类似这样的输出(注解是最重要的部分,因为它确认了iam角色的附加):
Name: alb-ingress-controller
Namespace: kube-system
Labels: app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=alb-ingress-controller
Annotations: eks.amazonaws.com/role-arn: <YOUR ANR WILL BE HERE>
meta.helm.sh/release-name: testrelease
meta.helm.sh/release-namespace: default
Image pull secrets: <none>
Mountable secrets: alb-ingress-controller-token-l4pd8
Tokens: alb-ingress-controller-token-l4pd8
Events: <none>
从现在开始,您可以使用此服务来管理您附加的策略允许的内部 k8s 资源和外部资源。
就我而言,如前所述,我使用它(除了其他东西)来创建 alb 入口控制器和负载均衡器,因此所有前缀都带有“alb-ingress”