通过 terraform 部署 helm chart 时无法传递服务注释

Question

我看到了一些关于如何在通过 terraform 部署 helm chart 时传递注释的示例，但没有一个按预期工作，在这种情况下，我试图创建一个服务，在特定子网上创建一个私有 ip，但是，它创建一个公共IP。 我的地形文件：

locals {
  helm_general = {
    # Reference values
    # https://github.com/elastic/helm-charts/blob/master/elasticsearch/values.yaml
    elasticsearch = {
      name      = "elasticsearch"
      chart     = "elastic/elasticsearch"
      tag       = "7.14.0"
      namespace = "elasticsearch"
      set = [
        {
          name  = "nodeSelector.agentpool"
          value = "general"
        },      
        {
          name  = "replicas"
          value = "1"
        },
        {
          name  = "minimumMasterNodes"
          value = "1"
        },
        {
          name  = "image"
          value = "docker.elastic.co/elasticsearch/elasticsearch"
        },
        {
          name  = "imageTag"
          value = "7.14.0"
        },
        {
          name  = "resources.requests.cpu"
          value = "10m"
        },
        {
          name  = "resources.requests.memory"
          value = "128Mi"
        },
        {
          name  = "volumeClaimTemplate.reosources.requests.storage"
          value = "4Gi"
        },
        {
          name  = "persistence.enabled"
          value = "false"
        },
        {
          name  = "service.type"
          value = "LoadBalancer"
        },
        {
          name  = "service.annotations\\.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal"
          value = "true"
        },
        {
          name  = "service.annotations\\.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal-subnet"
          value = "somesubnet"
        },          
      ]
      timeout = "900"
    }
  }
}

Helm 部署

resource "helm_release" "helm" {
  provider  = helm.general
  for_each  = local.helm_general
  name      = each.value.name
  chart     = each.value.chart
  namespace = format(each.value.namespace)
  dynamic "set" {
    iterator = item
    for_each = each.value.set == null ? [] : each.value.set

    content {
      name  = item.value.name
      value = item.value.value
    }
  }
  depends_on = [kubernetes_namespace.general]
}

计划/应用输出 https://i.imgur.com/uAdmblM.png

而且目前正在部署的是公共 ip 而不是私有 ip：

Namespace:                elasticsearch                                                     
Labels:                   app=elasticsearch-master                                          
                          app.kubernetes.io/managed-by=Helm                                 
                          chart=elasticsearch                                               
                          heritage=Helm                                                     
                          release=elasticsearch                                             
Annotations:              meta.helm.sh/release-name: elasticsearch                          
                          meta.helm.sh/release-namespace: elasticsearch                     
Selector:                 app=elasticsearch-master,chart=elasticsearch,release=elasticsearch
Type:                     LoadBalancer                                                      
IP Families:              <none>                                                            
IP:                       xx                                                      
IPs:                      xxx                                                     
LoadBalancer Ingress:     redacted public ip                                                      
Port:                     http  9200/TCP                                                    
TargetPort:               9200/TCP                                                          
NodePort:                 http  32083/TCP                                                   
Endpoints:                                                                                  
Port:                     transport  9300/TCP                                               
TargetPort:               9300/TCP                                                          
NodePort:                 transport  32638/TCP                                              
Endpoints:                                                                                  
Session Affinity:         None                                                              
External Traffic Policy:  Cluster                                                           
Events:                                                                                     
  Type    Reason                Age        From                Message                      
  ----    ------                ----       ----                -------                      
  Normal  EnsuringLoadBalancer  1s         service-controller  Ensuring load balancer       
  Normal  EnsuredLoadBalancer   <invalid>  service-controller  Ensured load balancer        

我一直关注的参考资料：

https://github.com/hashicorp/terraform-provider-helm/issues/125 https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release

编辑：

错误信息：

Error: unable to decode "": resource.metadataOnlyObject.ObjectMeta: v1.ObjectMeta.Annotations: ReadString: expects " or n, but found t, error found in #10 byte of ...|nternal":true},"labe|..., bigger context
..|beta.kubernetes.io/azure-load-balancer-internal":true},"labels":{"app":"elasticsearch-master","chart|...

  with helm_release.helm["elasticsearch"],
  on aks-general-helm.tf line 1, in resource "helm_release" "helm":
   1: resource "helm_release" "helm" {

Answer 1

我刚刚遇到了类似的问题，这对我有用：

{
 name  = "service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal"
 value = "true"
},

我认为问题在于它是如何连接的。弹性官方的图表清单中的服务是service.annotations:{}，因此您需要附加.service，然后使用\\。