GKE + CloudSQL MYSQL 实例上的 Wordpress 部署 -- MySQL 连接错误：(2002) 连接被拒绝

Question

我正在尝试完成以下在 GKE 上部署 Wordpress 的教程： https://cloud.google.com/kubernetes-engine/docs/tutorials/persistent-disk

我使用 terraform 来配置 gcp 资源，而不是教程推荐的 gcp。这是导致 CrashLoopBackOff 状态的部署。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wordpress
  template:
    metadata:
      labels:
        app: wordpress
    spec:
      containers:
        - image: wordpress
          name: wordpress
          env:
          - name: WORDPRESS_DB_HOST
            value: 127.0.0.1:3306
          # These secrets are required to start the pod.
          - name: WORDPRESS_DB_USER
            valueFrom:
              secretKeyRef:
                name: cloudsql-db-credentials
                key: username
          - name: WORDPRESS_DB_PASSWORD
            valueFrom:
              secretKeyRef:
                name: cloudsql-db-credentials
                key: password
          ports:
            - containerPort: 80
              name: wordpress
          volumeMounts:
            - name: wordpress-persistent-storage
              mountPath: /var/www/html
        # Change archtek-wordpress:us-west1:archtek-wordpress-postgres-instance here to include your GCP
        # project, the region of your Cloud SQL instance and the name
        # of your Cloud SQL instance. The format is
        # ::
        - name: cloudsql-proxy
          image: gcr.io/cloudsql-docker/gce-proxy:1.11
          command: ["/cloud_sql_proxy",
                    "-instances=archtek-wordpress:us-west1:archtek-wordpress-mysql-instance=tcp:3306",
                    # If running on a VPC, the Cloud SQL proxy can connect via Private IP. See:
                    # https://cloud.google.com/sql/docs/mysql/private-ip for more info.
                    # "-ip_address_types=PRIVATE",
                    "-credential_file=/secrets/cloudsql/key.json"]
          securityContext:
            runAsUser: 2  # non-root user
            allowPrivilegeEscalation: false
          volumeMounts:
            - name: cloudsql-instance-credentials
              mountPath: /secrets/cloudsql
              readOnly: true
          imagePullPolicy: Always
      volumes:
        - name: wordpress-persistent-storage
          persistentVolumeClaim:
            claimName: wordpress-volumeclaim
        - name: cloudsql-instance-credentials
          secret:
            secretName: cloudsql-instance-credentials

当我描述 pod 时，我在日志中看到以下内容：

wordpress-54c68dbf59-5djfx wordpress MySQL Connection Error: (2002) Connection refused

为了排除凭据无效的想法，我使用了用于创建 cloudsql-db-credentials 的用户名和密码，即我的部署 yaml 中引用的 k8s 机密，并运行了它。

$: gcloud sql connect archtek-wordpress-mysql-instance -u wordpress

我可以连接，没问题。但我发现我也不能这样做：

$: mysql -u wordpress -p'$CLOUD_SQL_PASSWORD' \                                                                    ()
    -h 35.197.7.98       -P 3306 \
    -D archtek-wordpress:us-west1:archtek-wordpress-mysql-instance -v

返回：

ERROR 2003 (HY000): Can't connect to MySQL server on '35.197.7.98' (60)

我知道，当使用gcloud 客户端连接到 cloudsql 数据库时，它会在身份验证前 5 分钟将 ip 列入白名单，这可能解释了为什么 mysql 客户端无法进行身份验证。但是，我不确定这个理由是否适用于我在集群中的部署。是否也需要将其列入白名单，cloudsql 才能接受身份验证请求？

这是用于配置 cloudsql 实例的 terraform 文件：

resource "google_sql_database_instance" "postgres" {
  name             = "archtek-wordpress-mysql-instance"
  database_version = "MYSQL_5_7"
  settings {
    tier              = "db-f1-micro"
    availability_type = "ZONAL"
  }
}

Answer 1

您尝试从GKE集群外部连接时遇到的错误：

错误 2003 (HY000)：无法连接到“35.197.7.98”上的 MySQL 服务器 (60)

这是因为您连接的站点 (IP) 没有被授权这样做。使用：

• $ gcloud sql connect ...

允许在 5 分钟内连接到 SQL 实例。

在GKE 中使用CloudSQL proxy 时，您无需授权要连接的网络。

你可以在GCP -> SQL -> Instance -> Connections看到授权部分。

此外，您可以查看 pod 的日志（wordpress 和 cloudsql-proxy）以确定哪个 pod 导致了问题（$ kubectl describe 除外）：

• $ kubectl logs POD_NAME -c wordpress

---

更多关于CloudSQL和sql-proxy的参考：

• Cloud.google.com: SQL: MySQL: Diagnose issues

• Cloud.google.com: SQL: MySQL: sql-proxy: Troubleshooting

---

假设您使用 github 页面中的 .tf 文件为 CloudSQL 实例创建了用户，则可能是它失败的原因。负责创建用户wordpress的部分主机参数错误（以下示例已编辑）：

resource "google_sql_user" "users" {
  name     = "wordpress"
  instance = google_sql_database_instance.postgres.name
  # host     = "*" <- BAD
  host      = "%" # <- GOOD
  password = random_password.password.result
}

我无法使用以下参数连接到服务器：host = "*"。将其从 "*" 更改为 "%" 解决了我的问题。

---

我设法创建了类似于官方GKE 指南部分的.tf 文件：

• Cloud.google.com: Kubernetes Engine: Tutorials: Persistent disk

将 Terraform 连接到 GCP 项目的指南：

• Terraform.io: Google: Guides: Getting started

使用的文件：

• main.tf
• vpc.tf - 创建一个VPC（基于评论中链接的github）
• gke.tf - 在新的 VPC 中创建一个 GKE 集群
• mysql.tf - 创建一个 CloudSQL 实例和一个用户 wordpress
• pvc.tf - 为 Wordpress 部署创建一个 PVC
• sa.tf - 创建一个 ServiceAccount 并将其绑定到访问 CloudSQL 实例所需的权限
• secret.tf - 为上面的 ServiceAccount 创建一个密钥，并为 Wordpress 和 CloudSQL pod 创建 Kubernetes 秘密
• deployment.tf - 创建将运行 Wordpress 和 cloudsql-proxy 的部署

每次添加新文件时，我都会运行以下命令（按上述顺序）：

• $ terraform init
• $ terraform apply

main.tf:

provider "google" {
  project = "ENTER-YOUR-PROJECT-ID"
  region  = "europe-west3"
  zone    = "europe-west3-c"
}

variable project {
  type = string
  default = "ENTER-YOUR-PROJECT-ID"
}

variable zone {
  type = string
  default = "europe-west3-c"
}

variable region {
  type = string
  default = "europe-west3"
}

vpc.tf:

resource "google_compute_network" "terraform-network" {
  name                    = "terraform-network"
  auto_create_subnetworks = "false"
}

resource "google_compute_subnetwork" "terraform-subnet" {
  name          = "terraform-subnet"
  region        = var.region
  network       = google_compute_network.terraform-network.name
  ip_cidr_range = "10.0.0.0/24"

}

gke.tf:

resource "google_container_cluster" "gke-terraform" {
  name     = "gke-terraform"
  location = var.zone
  initial_node_count = 1

  network    = google_compute_network.terraform-network.name
  subnetwork = google_compute_subnetwork.terraform-subnet.name
}

我也跑了：

• $ gcloud container clusters get-credentials gke-terraform --zone=europe-west3-c

mysql.tf

resource "google_sql_database_instance" "cloudsql" {
  name             = "cloudsql-terraform"
  database_version = "MYSQL_5_7"
  settings {
    tier              = "db-f1-micro"
    availability_type = "ZONAL"
  }
}

data "google_sql_database_instance" "cloudsql" {
    name = "cloudsql-terraform"
}

resource "random_password" "wordpress-cloudsql-password" {
  length           = 18
  special          = true
  override_special = "_%@"
}


resource "local_file" "password-file" {
  content  = random_password.wordpress-cloudsql-password.result
  filename = "./password-file"
}

resource "google_sql_user" "cloudsql-wordpress-user" {
  name     = "wordpress"
  instance = google_sql_database_instance.cloudsql.name
  host     = "%"
  password = random_password.wordpress-cloudsql-password.result
}

pvc.tf:

resource "google_compute_disk" "terraform-pd" {
  name  = "terraform-disk"
  type  = "pd-standard"
  zone  = "europe-west3-c"
}

resource "kubernetes_persistent_volume" "terraform-pv" {

  metadata {
    name = "wordpress-pv"
  }
  spec {
    capacity = {
      storage = "10Gi"
    }
    storage_class_name = "standard"
    access_modes = ["ReadWriteOnce"]
    persistent_volume_source {
      gce_persistent_disk {
        pd_name = google_compute_disk.terraform-pd.name
      }
    }
  }
}

resource "kubernetes_persistent_volume_claim" "terraform-pvc" {
  metadata {
    name = "wordpress-pvc"
  }
  spec {
    access_modes = ["ReadWriteOnce"]
    storage_class_name = "standard"
    resources {
      requests = {
        storage = "10Gi"
      }
    }
    volume_name = kubernetes_persistent_volume.terraform-pv.metadata.0.name
  }
}

sa.tf:

resource "google_service_account" "cloudsql-proxy-terraform" {
  account_id   = "cloudsql-proxy-terraform"
  display_name = "cloudsql-proxy-terraform"
}

data "google_service_account" "cloudsql-proxy-terraform" {
  account_id = "cloudsql-proxy-terraform"
}

resource "google_project_iam_binding" "cloudsql-proxy-binding" {
  project = var.project
  role    = "roles/cloudsql.client"

  members = [
    "serviceAccount:${google_service_account.cloudsql-proxy-terraform.email}",
  ]
}

secret.tf:

resource "google_service_account_key" "cloudsql-proxy-key" {
  service_account_id = google_service_account.cloudsql-proxy-terraform.name
}

resource "kubernetes_secret" "cloudsql-instance-credentials-terraform" {
  metadata {
    name = "cloudsql-instance-credentials-terraform"
  }
  data = {
    "key.json" = base64decode(google_service_account_key.cloudsql-proxy-key.private_key)
  }
}

resource "kubernetes_secret" "cloudsql-db-credentials-terraform" {
  metadata {
    name = "cloudsql-db-credentials-terraform"
  }
  data = {
    "username" = "wordpress"
    "password" = random_password.wordpress-cloudsql-password.result
  }
}

deployment.tf:

resource "kubernetes_deployment" "wordpress-deployment" {
  metadata {
    name = "wordpress-deployment"
    labels = {
      app = "wordpress"
    }
  }

  spec {
    
    replicas = 1

    selector {
      match_labels = {
        app = "wordpress"
      }
    }

    template {
      metadata {
        labels = {
          app = "wordpress"
        }
      }

      spec {

          container {

            image = "wordpress"
            name  = "wordpress"

            env {
                  name = "WORDPRESS_DB_HOST"
                  value = "127.0.0.1:3306"
            }

            env {
                  name = "WORDPRESS_DB_USER"
                  value_from {
                      secret_key_ref {
                          name = kubernetes_secret.cloudsql-db-credentials-terraform.metadata.0.name
                          key = "username"
                      }  
                  }
            }

            env {
                  name = "WORDPRESS_DB_PASSWORD"
                  value_from  {
                      secret_key_ref  {
                          name = kubernetes_secret.cloudsql-db-credentials-terraform.metadata.0.name
                          key = "password"
                      }  
                  }
            }

            port {
              name           = "http"
              container_port = 80
              protocol       = "TCP"
            }

            volume_mount {
              mount_path = "/var/www/html"
              name       = "wordpress-persistent-storage"
            }
            
            

          }
          
          container { 
            image = "gcr.io/cloudsql-docker/gce-proxy:1.11"
            name  = "cloudsql-proxy"

            command = ["/cloud_sql_proxy", 
            "-instances=${google_sql_database_instance.cloudsql.connection_name}=tcp:3306",
            "-credential_file=/secrets/cloudsql/key.json"]
            
            security_context {
                run_as_user = 2 
                allow_privilege_escalation = "false"
            }

            volume_mount {
                mount_path = "/secrets/cloudsql"
                name       = "cloudsql-instance-credentials-terraform"
                read_only  = "true"
            }



          }

        volume {
            name = "wordpress-persistent-storage"
            persistent_volume_claim {
              claim_name = "wordpress-pvc"
            } 
        }

        volume {
            name = "cloudsql-instance-credentials-terraform"
            secret {
                secret_name = "cloudsql-instance-credentials-terraform"
            }
        }
        
        }
      }
}
}

检查资源创建是否正确 ($ kubectl logs POD_NAME -c CONTAINER_NAME) 后，您可以使用以下命令公开您的 Wordpress：

• $ kubectl expose deployment wordpress-deployment --type=LoadBalancer --port=80