Terraform 计划在导入 aws 资源后显示差异

Question

这是我原来问题的后续问题：Terraform (0.12.29) import not working as expected; import succeeded but plan shows destroy & recreate

一些背景：我们有 terraform 代码来创建各种 AWS 资源。其中一些资源是按 AWS 账户创建的，因此被构造为存储在我们项目的账户范围文件夹中。当时我们只有一个 AWS 区域。现在我们的应用程序是多区域的，因此将为每个 AWS 账户的每个区域创建这些资源。

为了做到这一点，我们现在已将这些 TF 脚本移至将按区域运行的区域范围文件夹。由于这些资源不再属于“帐户范围”，我们已将它们从帐户范围 Terraform 状态中删除。现在当我尝试将这些资源导入到region scope

我的导入（从 xyz-region-scope 目录运行）和 terraform 计划：

terraform import module.buckets.random_id.cloudtrail_bucket_suffix cqLFzQ
terraform import module.buckets.aws_s3_bucket.cloudtrail_logging_bucket "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
terraform import module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
terraform import module.buckets.module.access_logging_bucket.aws_s3_bucket.default "ab-xyz-stage-access-logging-9d8e94ff"
terraform import module.buckets.module.access_logging_bucket.random_id.bucket_suffix  nY6U_w
terraform import module.encryption.module.data_key.aws_iam_policy.decrypt "arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_decrypt"
terraform import module.encryption.module.data_key.aws_iam_policy.encrypt "arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_encrypt"


mycompanymachine:xyz-region-scope kuldeepjain$ ../scripts/terraform.sh xyz-stage plan -no-color
+ set -o posix
+ IFS='
    '
++ blhome
+ BASH_LIB_HOME=/usr/local/lib/mycompany/ab/bash_library/0.0.1-SNAPSHOT
+ source /usr/local/lib/mycompany/ab/bash_library/0.0.1-SNAPSHOT/s3/bucket.sh
+ main xyz-stage plan -no-color
+ '[' 3 -lt 2 ']'
+ local env=xyz-stage
+ shift
+ local command=plan
+ shift
++ get_region xyz-stage
++ local env=xyz-stage
++ shift
+++ aws --profile xyz-stage configure get region
++ local region=us-west-2
++ '[' -z us-west-2 ']'
++ echo us-west-2
+ local region=us-west-2
++ _get_bucket xyz-stage xyz-stage-tfstate
++ local env=xyz-stage
++ shift
++ local name=xyz-stage-tfstate
++ shift
+++ _get_bucket_list xyz-stage xyz-stage-tfstate
+++ local env=xyz-stage
+++ shift
+++ local name=xyz-stage-tfstate
+++ shift
+++ aws --profile xyz-stage --output json s3api list-buckets --query 'Buckets[?contains(Name, `xyz-stage-tfstate`) == `true`].Name'
++ local 'bucket_list=[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ _count_buckets_in_json '[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ local 'json=[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ shift
+++ echo '[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ jq '. | length'
++ local number_of_buckets=1
++ '[' 1 == 0 ']'
++ '[' 1 -gt 1 ']'
+++ echo '[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ jq -r '.[0]'
++ local bucket_name=ab-xyz-stage-tfstate-5b8873b8
++ echo ab-xyz-stage-tfstate-5b8873b8
+ local tfstate_bucket=ab-xyz-stage-tfstate-5b8873b8
++ get_config_file xyz-stage us-west-2
++ local env=xyz-stage
++ shift
++ local region=us-west-2
++ shift
++ local config_file=config/us-west-2/xyz-stage.tfvars
++ '[' '!' -f config/us-west-2/xyz-stage.tfvars ']'
++ config_file=config/us-west-2/default.tfvars
++ echo config/us-west-2/default.tfvars
+ local config_file=config/us-west-2/default.tfvars
+ export TF_DATA_DIR=state/xyz-stage/
+ TF_DATA_DIR=state/xyz-stage/
+ terraform get
+ terraform plan -var-file=config/us-west-2/default.tfvars -var-file=variables.tfvars -var-file=../globals.tfvars -var profile=xyz-stage -var region=us-west-2 -var tfstate_bucket=ab-xyz-stage-tfstate-5b8873b8 -no-color
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

module.encryption.module.data_key.data.null_data_source.key: Refreshing state...
module.buckets.module.access_logging_bucket.data.template_file.dependencies: Refreshing state...
module.buckets.data.template_file.dependencies: Refreshing state...
data.aws_caller_identity.current: Refreshing state...
module.buckets.module.access_logging_bucket.data.aws_caller_identity.current: Refreshing state...
module.encryption.module.data_key.data.aws_kms_alias.default: Refreshing state...
module.buckets.data.aws_caller_identity.current: Refreshing state...
module.encryption.module.data_key.data.aws_region.current: Refreshing state...
module.encryption.module.data_key.data.aws_caller_identity.current: Refreshing state...
module.buckets.module.access_logging_bucket.data.aws_kms_alias.encryption_key_alias: Refreshing state...
module.buckets.module.access_logging_bucket.random_id.bucket_suffix: Refreshing state... [id=nY6U_w]
module.buckets.module.access_logging_bucket.aws_s3_bucket.default: Refreshing state... [id=ab-xyz-stage-access-logging-9d8e94ff]
module.buckets.random_id.cloudtrail_bucket_suffix: Refreshing state... [id=cqLFzQ]
module.buckets.module.access_logging_bucket.data.template_file.encryption_configuration: Refreshing state...
module.encryption.module.data_key.data.aws_iam_policy_document.encrypt: Refreshing state...
module.encryption.module.data_key.data.aws_iam_policy_document.decrypt: Refreshing state...
module.encryption.module.data_key.aws_iam_policy.decrypt: Refreshing state... [id=arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_decrypt]
module.encryption.module.data_key.aws_iam_policy.encrypt: Refreshing state... [id=arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_encrypt]
module.buckets.aws_s3_bucket.cloudtrail_logging_bucket: Refreshing state... [id=ab-xyz-stage-cloudtrail-logging-72a2c5cd]
module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail: Refreshing state...
module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket: Refreshing state... [id=ab-xyz-stage-cloudtrail-logging-72a2c5cd]

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
 <= read (data resources)

Terraform will perform the following actions:


  # module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "restrict_access_cloudtrail"  {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "s3:GetBucketAcl",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd",
            ]
          + sid       = "AWSCloudTrailAclCheck"

          + principals {
              + identifiers = [
                  + "cloudtrail.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
      + statement {
          + actions   = [
              + "s3:PutObject",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd/*",
            ]
          + sid       = "AWSCloudTrailWrite"

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "bucket-owner-full-control",
                ]
              + variable = "s3:x-amz-acl"
            }

          + principals {
              + identifiers = [
                  + "cloudtrail.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
    }

  # module.buckets.aws_s3_bucket.cloudtrail_logging_bucket will be updated in-place
  ~ resource "aws_s3_bucket" "cloudtrail_logging_bucket" {
      + acl                         = "private"
        arn                         = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd"
        bucket                      = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
        bucket_domain_name          = "ab-xyz-stage-cloudtrail-logging-72a2c5cd.s3.amazonaws.com"
        bucket_regional_domain_name = "ab-xyz-stage-cloudtrail-logging-72a2c5cd.s3.us-west-2.amazonaws.com"
      + force_destroy               = false
        hosted_zone_id              = "Z3BJ6K6RIION7M"
        id                          = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
        region                      = "us-west-2"
        request_payer               = "BucketOwner"
        tags                        = {
            "mycompany:finance:accountenvironment"   = "xyz-stage"
            "mycompany:finance:application"          = "ab-platform"
            "mycompany:finance:billablebusinessunit" = "my-dev"
            "name"                                = "Cloudtrail logging bucket"
        }

        lifecycle_rule {
            abort_incomplete_multipart_upload_days = 0
            enabled                                = true
            id                                     = "intu-lifecycle-s3-int-tier"
            tags                                   = {}

            transition {
                days          = 32
                storage_class = "INTELLIGENT_TIERING"
            }
        }

        logging {
            target_bucket = "ab-xyz-stage-access-logging-9d8e94ff"
            target_prefix = "logs/cloudtrail-logging/"
        }

        versioning {
            enabled    = false
            mfa_delete = false
        }
    }

  # module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket will be updated in-place
  ~ resource "aws_s3_bucket_policy" "cloudtrail_logging_bucket" {
        bucket = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
        id     = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
      ~ policy = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "s3:GetBucketAcl"
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "cloudtrail.amazonaws.com"
                        }
                      - Resource  = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd"
                      - Sid       = "AWSCloudTrailAclCheck"
                    },
                  - {
                      - Action    = "s3:PutObject"
                      - Condition = {
                          - StringEquals = {
                              - s3:x-amz-acl = "bucket-owner-full-control"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "cloudtrail.amazonaws.com"
                        }
                      - Resource  = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd/*"
                      - Sid       = "AWSCloudTrailWrite"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
    }

  # module.buckets.module.access_logging_bucket.aws_s3_bucket.default will be updated in-place
  ~ resource "aws_s3_bucket" "default" {
      + acl                         = "log-delivery-write"
        arn                         = "arn:aws:s3:::ab-xyz-stage-access-logging-9d8e94ff"
        bucket                      = "ab-xyz-stage-access-logging-9d8e94ff"
        bucket_domain_name          = "ab-xyz-stage-access-logging-9d8e94ff.s3.amazonaws.com"
        bucket_regional_domain_name = "ab-xyz-stage-access-logging-9d8e94ff.s3.us-west-2.amazonaws.com"
      + force_destroy               = false
        hosted_zone_id              = "Z3BJ6K6RIION7M"
        id                          = "ab-xyz-stage-access-logging-9d8e94ff"
        region                      = "us-west-2"
        request_payer               = "BucketOwner"
        tags                        = {
            "mycompany:finance:accountenvironment"   = "xyz-stage"
            "mycompany:finance:application"          = "ab-platform"
            "mycompany:finance:billablebusinessunit" = "my-dev"
            "name"                                = "Access logging bucket"
        }

      - grant {
          - permissions = [
              - "READ_ACP",
              - "WRITE",
            ] -> null
          - type        = "Group" -> null
          - uri         = "http://acs.amazonaws.com/groups/s3/LogDelivery" -> null
        }
      - grant {
          - id          = "0343271a8c2f184152c171b223945b22ceaf5be5c9b78cf167660600747b5ad8" -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }

      - lifecycle_rule {
          - abort_incomplete_multipart_upload_days = 0 -> null
          - enabled                                = true -> null
          - id                                     = "intu-lifecycle-s3-int-tier" -> null
          - tags                                   = {} -> null

          - transition {
              - days          = 32 -> null
              - storage_class = "INTELLIGENT_TIERING" -> null
            }
        }

        versioning {
            enabled    = false
            mfa_delete = false
        }
    }

Plan: 0 to add, 3 to change, 0 to destroy.

如您所见，terraform plan 输出显示 Plan：0 添加，3 更改，0 销毁。

我的问题是：

1. 为什么即使策略没有变化，它也会尝试为cloudtrail_logging_bucket 删除新的aws_s3_bucket_policy。请参阅下面的屏幕截图和 TF 代码 cloudtrail_bucket.tf。 cloudtrail_bucket_suffix 的旧帐户范围 (LEFT) 与我当前的远程 TF 状态 (RIGHT) 的差异片段：
2. 对于显示module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail will be read during apply 的资源。它显示+ 符号，这是否意味着它会在这里修改某些内容，或者只是按照它所说的那样阅读它。
3. 为什么它在删除grant 和lifecycle_rule 的地方显示这个module.buckets.module.access_logging_bucket.aws_s3_bucket.default will be updated in-place ~ resource "aws_s3_bucket" "default"。见下文s3_bucket.tf

TF 代码：

cloudtrail_bucket.tf：

data "aws_caller_identity" "current" {}

resource "random_id" "cloudtrail_bucket_suffix" {
  keepers = {
    # Keep the suffix per account id / environment
    aws_account_id = "${data.aws_caller_identity.current.account_id}"
    env = "${var.environment}"
  }

  byte_length = "4"
}


resource "aws_s3_bucket" "cloudtrail_logging_bucket" {
  bucket = "ab-${var.environment}-cloudtrail-logging-${random_id.cloudtrail_bucket_suffix.hex}"
  acl = "private"

  depends_on = [data.template_file.dependencies]

  tags = {
    name = "Cloudtrail logging bucket"
    "mycompany:finance:accountenvironment" = "${var.environment}"
    "mycompany:finance:application" = "${module.constants.finance_application}"
    "mycompany:finance:billablebusinessunit" = "${module.constants.finance_billablebusinessunit}"
  }

  lifecycle {
    ignore_changes = [ "server_side_encryption_configuration" ]
  }

  logging {
    target_bucket =  "${module.access_logging_bucket.name}"
    target_prefix = "logs/cloudtrail-logging/"
  }

  lifecycle_rule {
    enabled = "true"

    transition {
      days          = 32
      storage_class = "INTELLIGENT_TIERING"
    }
  }

}

resource "aws_s3_bucket_policy" "cloudtrail_logging_bucket" {
  bucket = "${aws_s3_bucket.cloudtrail_logging_bucket.id}"
  policy = "${data.aws_iam_policy_document.restrict_access_cloudtrail.json}"
}


data aws_iam_policy_document "restrict_access_cloudtrail" {

  statement {
    sid = "AWSCloudTrailAclCheck"
    effect = "Allow"
    actions = ["s3:GetBucketAcl"]
    resources = [ "${aws_s3_bucket.cloudtrail_logging_bucket.arn}" ]
    principals {
      identifiers = ["cloudtrail.amazonaws.com"]
      type = "Service"
    }

  }

  statement {
    sid = "AWSCloudTrailWrite"
    effect = "Allow"
    actions = ["s3:PutObject"]
    resources = [ "${aws_s3_bucket.cloudtrail_logging_bucket.arn}/*" ]
    principals {
      identifiers = ["cloudtrail.amazonaws.com"]
      type = "Service"
    }
    condition {
      test = "StringEquals"
      values = ["bucket-owner-full-control"]
      variable = "s3:x-amz-acl"
    }

  }
}

s3_bucket.tf

resource "random_id" "bucket_suffix" {
    keepers = {
        # Keep the suffix per account id / environment
        aws_account_id = "${data.aws_caller_identity.current.account_id}"
        env = "${var.environment}"
    }

    byte_length = "${var.byte_length}"
}

resource "aws_s3_bucket" "default" {
    bucket = "ab-${var.environment}-${var.name}-${random_id.bucket_suffix.hex}"
    acl = "${var.acl}"

    depends_on = [data.template_file.dependencies]

    tags = {
        name = "${var.name_tag}"
        "mycompany:finance:accountenvironment" = "${var.environment}"
        "mycompany:finance:application" = "${module.constants.finance_application}"
        "mycompany:finance:billablebusinessunit" = "${module.constants.finance_billablebusinessunit}"
    }

    lifecycle {
        ignore_changes = [ "server_side_encryption_configuration" ]
    }

    logging {
        target_bucket = "${lookup(var.logging, "target_bucket", "ab-${var.environment}-${var.name}-${random_id.bucket_suffix.hex}")}"
        target_prefix = "logs/${lookup(var.logging, "target_folder_name", "access-logging")}/"
    }
}

我的环境：

Local machine: macOS v10.14.6

Terraform v0.12.29
+ provider.aws v3.14.1
+ provider.null v2.1.2
+ provider.random v2.3.1
+ provider.template v2.1.2

Answer 1

如果 terraform-code 与导入的现有资源不同，则可能会显示这种差异。例如如果有人通过在 AWS 管理控制台中单击并指向更改资源，而无需编辑/应用代码。 terraform import 仅将资源导入到 tfstate，但不创建 terraform 代码。

在此示例中，您可以在 AWS 控制台/cli 中验证 S3 存储桶“默认”是否实际配置了日志记录。按照计划，现有的bucket没有配置为登录AWS，但是你的TF-code包含了这个，所以会改。

您确定您的 TF 代码与导入资源的所有属性完全匹配吗？

为了进一步调查，您也有必要发布相应的 tf 代码。

Answer 2

只是回答我自己的问题，以便我可以提及我对每个问题所做的：

对于2nd question，它不会使用terraform apply 修改任何内容，只是按照消息中的说明进行读取。

对于我的3rd question 在这里的一个单独的 SO 线程中询问它：Terraform - Updating S3 Access Control: Question on replacing acl with grant 并继续我在答案中提到的解决方案。

对于1st question，仍然不清楚它为什么显示差异。我尝试将现有状态与terraform state pull 进行比较，并检查为什么更新完成但没有帮助。但是为此运行 terraform apply 效果很好，并且它没有对政策做出任何改变，这正是我所期望的。