Terraform 错误创建 Lambda 函数：ResourceConflictException 与刚刚由 Terraform 创建的资源应用答案

【问题标题】：Terraform Error creating Lambda function: ResourceConflictException with the resource just created by Terraform applyTerraform 错误创建 Lambda 函数：ResourceConflictException 与刚刚由 Terraform 创建的资源应用
【发布时间】：2021-01-22 15:04:06
【问题描述】：

我正在我们账户的每个 AWS 区域中部署 Lambda 函数，并遇到奇怪的问题，即应用失败并针对某些 AWS 区域显示以下错误消息

应用 Terraform 时出错

Error: Error creating Lambda function: ResourceConflictException: Function already exist: log-forwarder
{
  RespMetadata: {
    StatusCode: 409,
    RequestID: "8cfd7260-7c4a-42d2-98c6-6619c7b2804f"
  },
  Message_: "Function already exist: log-forwarder",
  Type: "User"
}

上述 Lambda 函数刚刚由失败的同一 Terraform Apply 创建。

terraform plan 和 init 不会抛出任何关于 TF 配置问题的错误。

plan 和 init 都运行成功。

下面是我的目录结构

.
├── log_forwarder.tf
├── log_forwarder_lambdas
│   └── main.tf
└── providers.tf

下面是我的providers.tf文件

provider "aws" {
  region  = "us-east-1"
  version = "3.9.0"
}
provider "aws" {
  alias   = "us-east-2"
  region  = "us-east-2"
  version = "3.9.0"
}
provider "aws" {
  alias   = "us-west-2"
  region  = "us-west-2"
  version = "3.9.0"
}  
provider "aws" {
  alias   = "us-west-1"
  region  = "us-west-1"
  version = "3.9.0"
}
provider "aws" {
  alias   = "ca-central-1"
  region  = "ca-central-1"
  version = "3.9.0"
}


... with all the AWS Regions.

下面是log_forwarder.tf的tf配置

terraform {
  required_version = "0.12.25"

  backend "s3" {
    All the backend Config
  }
}


resource "aws_iam_role" "log_forwarder" {
  name = "LogForwarder"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": ["lambda.amazonaws.com"]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF
}

resource "aws_iam_role_policy" "log_forwarder" {
  name = "LogForwarder"
  role = aws_iam_role.log_forwarder.id

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "lambda:ListTags",
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": [
        "arn:aws:logs:*",
        "arn:aws:lambda:*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "lambda:InvokeFunction"
      ],
      "Resource": "*"
    },
        {
    "Sid": "AWSDatadogPermissionsForCloudtrail",
    "Effect": "Allow",
    "Action": ["s3:ListBucket", "s3:GetBucketLocation", "s3:GetObject", "s3:ListObjects"],
    "Resource": [
        "arn:aws:s3:::BucketName",
        "arn:aws:s3:::BucketName/*"
    ]
}

  ]
}
EOF
}


module "DDLogForwarderUSEast1" {
  source                = "./log_forwarder_lambdas"
  dd_log_forwarder_role = aws_iam_role.log_forwarder.arn
  region                = "us-east-1"

}

module "DDLogForwarderUSEast2" {
  source                = "./log_forwarder_lambdas"
  dd_log_forwarder_role = aws_iam_role.log_forwarder.arn
  providers             = { aws = aws.us-east-2 }
  region                = "us-east-2"
}

module "DDLogForwarderUSWest1" {
  source                = "./log_forwarder_lambdas"
  dd_log_forwarder_role = aws_iam_role.log_forwarder.arn
  providers             = { aws = aws.us-west-1 }
  region                = "us-west-1"
}

module "DDLogForwarderUSWest2" {
  source                = "./log_forwarder_lambdas"
  dd_log_forwarder_role = aws_iam_role.log_forwarder.arn
  region                = "us-west-2"
  providers             = { aws = aws.us-west-2 }
}

module "DDLogForwarderAPEast1" {
  source                = "./log_forwarder_lambdas"
  dd_log_forwarder_role = aws_iam_role.log_forwarder.arn
  providers             = { aws = aws.ap-east-1 }
  region                = "ap-east-1"
}

module "DDLogForwarderAPSouth1" {
  source                = "./log_forwarder_lambdas"
  dd_log_forwarder_role = aws_iam_role.log_forwarder.arn
  region                = "ap-south-1"
  providers             = { aws = aws.ap-south-1 }
}

... All AWS Regions with different providers

log_forwarder_lambdas/main.tf的TF配置

variable "region" {}

variable "account_id" {
  default = "AWS Account Id"
}

variable "dd_log_forwarder_role" {}

variable "exclude_at_match" {
  default = "([A-Z]* RequestId: .*)"
}

data "aws_s3_bucket" "cloudtrail_bucket" {
  count  = var.region == "us-west-2" ? 1 : 0
  bucket = "BucketName"
}


resource "aws_lambda_function" "log_forwarder" {
  filename      = "${path.cwd}/log_forwarder_lambdas/aws-dd-forwarder-3.16.3.zip"
  function_name = "log-forwarder"
  role          = var.dd_log_forwarder_role
  description   = "Gathers logs from targetted Cloudwatch Log Groups and sends them to DataDog"
  handler       = "lambda_function.lambda_handler"
  runtime       = "python3.7"
  timeout       = 600
  memory_size   = 1024
  layers        = ["arn:aws:lambda:${var.region}:464622532012:layer:Datadog-Python37:11"]


  environment {
    variables = {
      DD_ENHANCED_METRICS = false
      EXCLUDE_AT_MATCH    = var.exclude_at_match
    }
  }
}

resource "aws_cloudwatch_log_group" "log_forwarder" {
  name              = "/aws/lambda/${aws_lambda_function.log_forwarder.function_name}"
  retention_in_days = 90
}

resource "aws_lambda_permission" "cloudtrail_bucket" {
  count         = var.region == "us-west-2" ? 1 : 0
  statement_id  = "AllowExecutionFromS3Bucket"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.log_forwarder.arn
  principal     = "s3.amazonaws.com"
  source_arn    = element(data.aws_s3_bucket.cloudtrail_bucket.*.arn, count.index)
}

resource "aws_s3_bucket_notification" "cloudtrail_bucket_notification" {
  count  = var.region == "us-west-2" ? 1 : 0
  bucket = element(data.aws_s3_bucket.cloudtrail_bucket.*.id, count.index)


  lambda_function {
    lambda_function_arn = aws_lambda_function.log_forwarder.arn
    events              = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.cloudtrail_bucket, aws_cloudwatch_log_group.log_forwarder]
}

在这种情况下，我使用的是 TF 0.12.25。

到目前为止我尝试过的事情。

每次运行 Terraform init/plan/apply 循环时，从根模块中删除 .terraform 文件夹
我已尝试尽可能多地重构代码。
我正在本地运行 TF 计划/应用周期，没有任何 CI。

【问题讨论】：

标签： terraform terraform-provider-aws

【解决方案1】：

乍一看，似乎 Lambda 函数可能不在您的 Terraform 状态（无论出于何种原因）。您是否更改了后端/从后端删除了数据？

运行 terraform show 和/或 terraform state show 并查看冲突的 Lambda 函数是否处于您的状态。
如果不是，但它已经存在于 AWS 中，您可以导入它。
见这里：https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function#import

更新：

根据您的评论，由于资源存在于 AWS 中但不在状态中，因此这是一个预期错误。（Terraform 不知道资源存在，因此尝试创建它；AWS 知道它已经存在，因此返回错误。）

你有两个选择：

删除AWS中的资源，再次运行Terraform；或
将现有资源导入 Terraform（推荐）。

尝试类似：

terraform import module.DDLogForwarderUSEast1.aws_lambda_function.log-forwarder log-forwarder

（如果在其他地区尝试此操作，请确保您设置了正确的提供商/地区！）

【讨论】：

这是我最初的思考过程，不知何故 Lambda 函数已经存在于 AWS 中，而我的 TF 正在抱怨它。但是我仔细检查了目标区域但没有成功。我还在完全独立的 AWS 帐户中启动了上面的基础设施，只是为了分开，但仍然是同样的错误。
定义成功？ > 但是我仔细检查了目标区域，但没有成功。
那么你的terraform show 的输出是什么？有没有这些功能？
当我运行 TF apply to above code 时，它开始创建所有必要的资源，并在运行结束时失败并显示 Resource Already exists 错误消息。如果我转到 TF 错误中提到的目标区域，我可以看到 lambda 函数已由 TF Apply 循环（几分钟前）创建。但是terraform state list 没有这样的资源。我已经直观地检查了 s3 后端中的 terraform 状态，并且它还缺少刚刚由 TF 应用周期创建的资源。