运行 AWS ECS 容器时 ELB 运行状况检查失败

Question

我目前正在尝试通过 CloudFormation 模板将应用程序部署到 AWS ECS 上。 Docker 映像存储在 AWS ECR 中并部署到由 Application Load Balancer 提供的 ECS 服务中。

我的服务启动了，我的负载均衡器也创建好了，但是ECS服务里面的任务反复失败，报错：

Task failed ELB health checks in (target-group arn:aws:elasticloadbalancing:us-east-1:...

我检查了我的安全组——ECS服务安全组包含负载均衡器安全组，负载均衡器创建成功。

我已经手动尝试在 ECR 上拉取我的图像并运行它 - 没有问题。我错过了什么？我的模板如下。

Resources:
  ECSRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [ecs.amazonaws.com]
          Action: ['sts:AssumeRole']
      Path: /
      Policies:
      - PolicyName: ecs-service
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action:
              # Rules which allow ECS to attach network interfaces to instances
              # on your behalf in order for awsvpc networking mode to work right
              - 'ec2:AttachNetworkInterface'
              - 'ec2:CreateNetworkInterface'
              - 'ec2:CreateNetworkInterfacePermission'
              - 'ec2:DeleteNetworkInterface'
              - 'ec2:DeleteNetworkInterfacePermission'
              - 'ec2:Describe*'
              - 'ec2:DetachNetworkInterface'

              # Rules which allow ECS to update load balancers on your behalf
              # with the information sabout how to send traffic to your containers
              - 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer'
              - 'elasticloadbalancing:DeregisterTargets'
              - 'elasticloadbalancing:Describe*'
              - 'elasticloadbalancing:RegisterInstancesWithLoadBalancer'
              - 'elasticloadbalancing:RegisterTargets'
            Resource: '*'

  # This is a role which is used by the ECS tasks themselves.
  ECSTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [ecs-tasks.amazonaws.com]
          Action: ['sts:AssumeRole']
      Path: /
      Policies:
        - PolicyName: AmazonECSTaskExecutionRolePolicy
          PolicyDocument:
            Statement:
            - Effect: Allow
              Action:
                # Allow the ECS Tasks to download images from ECR
                - 'ecr:GetAuthorizationToken'
                - 'ecr:BatchCheckLayerAvailability'
                - 'ecr:GetDownloadUrlForLayer'
                - 'ecr:BatchGetImage'

                # Allow the ECS tasks to upload logs to CloudWatch
                - 'logs:CreateLogStream'
                - 'logs:PutLogEvents'
              Resource: '*'
  TaskDef:
    Type: AWS::ECS::TaskDefinition
    Properties: 
      Cpu: 4096    
      Memory: 30720
      ContainerDefinitions: 
        - Image: !Ref ECRImageUrl
          Name: !Sub "${ProjectName}-ecsContainer"
          PortMappings: 
            - ContainerPort: 4000
              HostPort: 4000
              Protocol: tcp
      Family: !Sub "${ProjectName}-taskDef"
      ExecutionRoleArn: !Ref ECSTaskExecutionRole
      RequiresCompatibilities: 
        - FARGATE
      NetworkMode: awsvpc

  Cluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Sub "${ProjectName}-ECSCluster"

  Service:
    Type: AWS::ECS::Service
    DependsOn:
      - LoadBalancerListener
    Properties:
      Cluster: !Ref Cluster
      DesiredCount: 2
      LaunchType: FARGATE
      ServiceName: !Sub "${ProjectName}-ECSService"
      TaskDefinition: !Ref TaskDef
      NetworkConfiguration:
        AwsvpcConfiguration:
          SecurityGroups: 
            - !Ref FargateContainerSecurityGroup
          AssignPublicIp: ENABLED
          Subnets: !Split [',', {'Fn::ImportValue': !Sub '${VPCStackName}-PublicSubnets'}]
      LoadBalancers:
        - ContainerName: !Sub "${ProjectName}-ecsContainer"
          ContainerPort: 4000
          TargetGroupArn: !Ref TargetGroup

  FargateContainerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the Fargate containers
      VpcId:
        Fn::ImportValue: 
          !Sub '${VPCStackName}-VPC'
  EcsSecurityGroupIngressFromPublicALB:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Ingress from the public ALB
      GroupId: !Ref 'FargateContainerSecurityGroup'
      IpProtocol: -1
      SourceSecurityGroupId: !Ref 'PublicLoadBalancerSG'
  EcsSecurityGroupIngressFromSelf:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Ingress from other containers in the same security group
      GroupId: !Ref 'FargateContainerSecurityGroup'
      IpProtocol: -1
      SourceSecurityGroupId: !Ref 'FargateContainerSecurityGroup'
  PublicLoadBalancerSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the public facing load balancer
      VpcId: 
        Fn::ImportValue: 
          !Sub '${VPCStackName}-VPC'
      SecurityGroupIngress:
          - CidrIp: 0.0.0.0/0
            IpProtocol: -1

  ACMCertificate:
    Type: AWS::CertificateManager::Certificate
    Properties: 
      DomainName: !Sub ${ProjectName}.${DomainName}
      ValidationMethod: DNS

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    DependsOn:
      - LoadBalancer
    Properties:
      TargetType: ip
      Name: !Sub "${ProjectName}-ECSService"
      Port: 4000
      Protocol: HTTP
      VpcId: 
        Fn::ImportValue: 
          !Sub '${VPCStackName}-VPC'

  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Scheme: internet-facing
      Subnets: !Split [',', {'Fn::ImportValue': !Sub '${VPCStackName}-PublicSubnets'}]
      SecurityGroups: 
        - !Ref PublicLoadBalancerSG

  LoadBalancerListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    DependsOn:
      - LoadBalancer
    Properties:
      DefaultActions:
        - TargetGroupArn: !Ref TargetGroup
          Type: 'forward'
      LoadBalancerArn: !Ref LoadBalancer
      Port: 443
      Protocol: HTTP

Answer 1

事实证明，我的安全组不够宽松。来自网络负载均衡器的流量被视为来自其原始来源，因此如果您的 NLB 对所有流量开放，您的 Fargate 容器也应该如此。这解决了我的问题：

  FargateContainerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the Fargate containers
      VpcId:
        Fn::ImportValue: 
          !Sub '${VPCStackName}-VPC'
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: !Ref ApplicationPort
          ToPort: !Ref ApplicationPort
          CidrIp: 0.0.0.0/0

Answer 2

健康检查功能自动在端口 80 调用 / 并期望 200 状态码作为响应。它在 EC2-> 目标组 -> 您的 ecs 目标组中可用。您必须确保您的端口为 4000，并在运行状况检查中调整默认路径和响应状态代码。

此外，您始终可以尝试使用您正在使用的端口 4000 上的公共 ip 或 DNS 连接到您的 ec2 实例，看看是否可行。

如果 ec2 实例无法在端口 4000 上运行，请对 docker 部署进行故障排除。谈话定义或参数有问题。

如果目标组 trlargets 或运行状况检查配置出现问题。

希望这会有所帮助。

Answer 3

在经历了很多痛苦和磨难之后，我发现 ALB 本身需要与安全组 (SG) 相关联，该安全组 (SG) 允许端口上的流量由 ECS 动态分配。您应该有一个自动定义的 SG 来定义这些端口范围。将此 SG 与您的 ALB 相关联，您的健康检查将开始通过（假设其他一切都正确连接）。

此外，请确保您的任务定义将网络模式设置为“bridge”并且“hostPort”值设置为 0——这表明 ECS 在底层 EC2 实例上动态分配一个端口并将其映射到你的容器端口。