解决 AWS CloudFormation 中的循环依赖关系

Question

以下 AWS CloudFormation 给出循环依赖错误。我的理解是依赖关系是这样流动的：rawUploads -> generatePreview -> previewPipeline -> rawUploads。虽然rawUploads 似乎并不依赖于generatePreview，但我猜CF 需要知道在创建存储桶时要触发什么lambda，即使触发器是在CloudFormation 模板的lambda 部分中定义的。

我在网上找到了一些讨论类似问题的资源，但它似乎不适用于这里。 https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-circular-dependency-cloudformation/

我有哪些方法可以打破这种循环依赖链？可编写脚本的解决方案是可行的，但手动更改的多个部署不适合我的用例。

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  rawUploads:
    Type: 'AWS::S3::Bucket'
  previewAudioFiles:
    Type: 'AWS::S3::Bucket'

  generatePreview:
    Type: AWS::Serverless::Function
    Properties:
      Handler: generatePreview.handler
      Runtime: nodejs6.10
      CodeUri: .
      Environment:
        Variables:
          PipelineId: !Ref previewPipeline
      Events:
        BucketrawUploads:
          Type: S3
          Properties:
            Bucket: !Ref rawUploads
            Events: 's3:ObjectCreated:*'

  previewPipeline:
    Type: Custom::ElasticTranscoderPipeline
    Version: '1.0'
    Properties:
      ServiceToken:
        Fn::Join:
        - ":"
        - - arn:aws:lambda
          - Ref: AWS::Region
          - Ref: AWS::AccountId
          - function
          - aws-cloudformation-elastic-transcoder-pipeline-1-0-0
      Name: transcoderPipeline
      InputBucket:
        Ref: rawUploads
      OutputBucket:
        Ref: previewAudioFiles

Answer 1

一种方法是为 S3 存储桶指定明确的名称，以便稍后您可以简单地使用存储桶名称，而不是依赖 Ref: bucketname。如果您想要自动生成的存储桶名称，这显然是有问题的，在这种情况下，谨慎的做法是从某个前缀加上（唯一）堆栈名称生成存储桶名称，例如：

InputBucket: !Join ["-", ['rawuploads', Ref: 'AWS::StackName']]

另一种选择是使用单个 CloudFormation 模板，但分两个阶段 - 第一个阶段创建基础资源（以及任何非循环的 refs），然后将剩余的 refs 添加到模板并进行堆栈更新。显然不理想，所以我更喜欢第一种方法。

在需要引用 ARN 的情况下，您也可以使用第一种技术，例如：

!Join ['/', ['arn:aws:s3:::logsbucket', 'AWSLogs', Ref: 'AWS:AccountId', '*']]

在使用此技术时，您可能还需要考虑使用DependsOn，因为您已删除有时会导致问题的隐式依赖项。

Answer 2

这篇文章最终帮助了我：https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-destination-s3/

我最终在 CloudFormation 中配置了一个 SNS 主题。存储桶将推送有关此主题的事件，而 Lambda 函数会侦听此主题。这样依赖图如下：

S3 bucket -> SNS topic -> SNS topic policy
Lambda function -> SNS topic
Lambda function -> transcoder pipeline

类似的东西（省略了一些政策）

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  SNSTopic:
    Type: AWS::SNS::Topic
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Id: MyTopicPolicy
        Version: '2012-10-17'
        Statement:
        - Sid: Statement-id
          Effect: Allow
          Principal:
            AWS: "*"
          Action: sns:Publish
          Resource:
            Ref: SNSTopic
          Condition:
            ArnLike:
              aws:SourceArn:
                !Join ["-", ['arn:aws:s3:::rawuploads', Ref: 'AWS::StackName']]
      Topics:
      - Ref: SNSTopic

  rawUploads:
    Type: 'AWS::S3::Bucket'
    DependsOn: SNSTopicPolicy
    Properties:
      BucketName: !Join ["-", ['rawuploads', Ref: 'AWS::StackName']]
      NotificationConfiguration:
        TopicConfigurations:
          - Topic:
              Ref: "SNSTopic"
            Event: 's3:ObjectCreated:*'

  previewAudioFiles:
    Type: 'AWS::S3::Bucket'


  generatePreview:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Join ["-", ['generatepreview', Ref: 'AWS::StackName']]
      Handler: generatePreview.handler
      Runtime: nodejs6.10
      CodeUri: .
      Environment:
        Variables:
          PipelineId: !Ref previewPipeline
      Events:
        BucketrawUploads:
          Type: SNS
          Properties:
            Topic: !Ref "SNSTopic"

  previewPipeline:
    Type: Custom::ElasticTranscoderPipeline
    DependsOn: 'rawUploads'
    Version: '1.0'
    Properties:
      ServiceToken:
        Fn::Join:
        - ":"
        - - arn:aws:lambda
          - Ref: AWS::Region
          - Ref: AWS::AccountId
          - function
          - aws-cloudformation-elastic-transcoder-pipeline-1-0-0
      Name: transcoderPipeline
      InputBucket:
        !Join ["-", ['arn:aws:s3:::rawuploads', Ref: 'AWS::StackName']]
      OutputBucket:
        Ref: previewAudioFiles