免责声明:我知道这个问题已经问过一段时间了,但我还是要回答以防万一它可能对其他人有所帮助......它肯定会对我有所帮助
假设您正在尝试重现 link you put in the question 的内容,那么我认为可能是错误的:
当然是部分答案:aws_appautoscaling_target.my_custom_resource.resource_id
如果您创建了一个 aws_api_gateway_deployment 资源来重现 aws-sample 的 MyApi 部分,那么您很幸运!
你可以用这个:
resource "aws_appautoscaling_target" "my_custom_resource" {
# ...
resource_id = "${aws_api_gateway_deployment.gateway.invoke_url}/scalableTargetDimensions/${var.stream}"
# ...
如果您想了解如何让这个 ^ 工作的详细信息,请参阅下文...
但请记住,这可能还不够!
可能也是解决方案的一部分:创建aws_appautoscaling_target 所需的内部链接和权限
始终假设您正在关注same example,并且您已经使用terraform 实现了大部分 所有...
还有
免责声明:我不确定此答案其余部分的确切原因。我怀疑这与AWS Auto Scaling服务的内部API的要求有关。
TL;DR:在注册自动缩放目标之前,一切都需要连接并正常工作
这部分:
README.md 的 this other project that covers the integration you're trying to achieve 暗示了这一点。
长版;我在实施中必须解决的问题:
- Lambda 响应格式
- API 网关设置和接线
- 正确的 IAM 权限
- Lambda 和 APIGateway 工作正常
1。确保你有正确的 lambda 响应
lambda 的返回值的返回格式对于 aws_appautoscaling_target 的创建至关重要。
它需要完全:
returningJson = {
"actualCapacity": float(actualCapacity),
"desiredCapacity": float(desiredCapacity),
"dimensionName": resourceName,
"resourceName": resourceName,
"scalableTargetDimensionId": resourceName,
"scalingStatus": scalingStatus,
"version": "MyVersion"
}
try:
returningJson['failureReason'] = failureReason
except:
pass
(这是示例中定义的方式)...
在我的实现中,我曾尝试过它(在一切部署和完成之前),我认为我可以从 GET 调用中获取更多数据,用于度量和监控......
原来,最后一切都搞定了,我只需要恢复返回函数,就全部连接成功了。
2。 API 网关设置和接线
这部分给我带来了麻烦。我认为 Auto Scaling API 连接并找到目标以便注册它是完全正确的(这就是您要创建的资源所做的)
@987654326 @ 是一个有效的 openapi.yaml 定义。
我建议把它放在一个像这样的文件中:
openapi.yaml.template
swagger: '2.0'
info:
title: "${NAME}"
paths:
'/scalableTargetDimensions/{scalableTargetDimensionId}':
get:
tags:
- ScalableTargets
x-tags:
- tag: ScalableTargets
security:
- sigv4: []
x-amazon-apigateway-any-method:
produces:
- application/json
consumes:
- application/json
x-amazon-apigateway-integration:
httpMethod: POST
type: aws_proxy
uri: ${INTEGRATION_URI}
responses: {}
patch:
tags:
- ScalableTargets
x-tags:
- tag: ScalableTargets
security:
- sigv4: []
x-amazon-apigateway-any-method:
security:
- sigv4: []
produces:
- application/json
consumes:
- application/json
x-amazon-apigateway-integration:
httpMethod: POST
type: aws_proxy
uri: ${INTEGRATION_URI}
responses: {}
securityDefinitions:
sigv4:
type: apiKey
name: Authorization
in: header
x-amazon-apigateway-authtype: awsSigv4
然后你可以这样使用它:
# API Gateway
resource "aws_api_gateway_rest_api" "gateway" {
name = var.rest_api_name
body = templatefile("${path.module}/openapi.yaml.template",
{
NAME = var.rest_api_name,
INTEGRATION_URI = var.integration_uri
}
)
}
resource "aws_api_gateway_deployment" "gateway" {
depends_on = [
aws_api_gateway_rest_api.gateway,
]
lifecycle {
create_before_destroy = true
}
rest_api_id = aws_api_gateway_rest_api.gateway.id
stage_name = var.stage_name
}
3。正确的 IAM 权限
cloudformation 模板中定义的权限以及您可以使用 terraform 创建的权限不是精确匹配,并且似乎需要一些调整才能使集成工作...(我怀疑这与某些 AWS 魔法有关,但我发现转移到 terraform 总体上相对容易)
所以这是我最终创建的role 和policies:
# Lambda
resource "aws_lambda_function" "lambda" {
# ...
role = aws_iam_role.kinesis_autoscaler_lambda_role.arn
# ...
}
resource "aws_iam_role" "kinesis_autoscaler_lambda_role" {
name = "${var.env}-kinesis-scaler-lambda-role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_lambda_permission" "kinesis_api" {
statement_id = "AllowKinesisAPIInvoke"
function_name = aws_lambda_function.lambda.function_name
action = "lambda:InvokeFunction"
principal = "apigateway.amazonaws.com"
source_arn = "${aws_api_gateway_deployment.gateway.execution_arn}/GET/scalableTargetDimensions/{scalableTargetDimensionId}"
}
resource "aws_lambda_permission" "kinesis_api_patch" {
statement_id = "AllowKinesisAPIPatchInvoke"
function_name = aws_lambda_function.lambda.function_name
action = "lambda:InvokeFunction"
principal = "apigateway.amazonaws.com"
source_arn = "${aws_api_gateway_deployment.gateway.execution_arn}/PATCH/scalableTargetDimensions/{scalableTargetDimensionId}"
}
# Permissions
resource "aws_iam_policy" "lambda_access_stream" {
name = "${var.stream_name}-access-stream-policy"
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "KinesisConsumerAccess",
"Effect": "Allow",
"Action": [
"kinesis:DescribeStreamConsumer"
],
"Resource": "${aws_kinesis_stream.stream.arn}/consumer/*:*"
},
{
"Sid": "KinesisStreamAccess",
"Effect": "Allow",
"Action": [
"kinesis:DescribeLimits",
"kinesis:DescribeStream",
"kinesis:DescribeStreamConsumer",
"kinesis:DescribeStreamSummary",
"kinesis:UpdateShardCount"
],
"Resource": "${aws_kinesis_stream.stream.arn}"
},
{
"Sid": "SSMParameterStoreGet",
"Effect": "Allow",
"Action": [
"ssm:GetParameter"
],
"Resource": [
"${aws_ssm_parameter.number_of_shards.arn}",
]
},
{
"Sid": "SSMParameterStorePut",
"Effect": "Allow",
"Action": [
"ssm:PutParameter"
],
"Resource": [
"${aws_ssm_parameter.number_of_shards.arn}"
]
}
]
}
POLICY
}
# I ended up splitting the policies for `module` reasons...
resource "aws_iam_policy_attachment" "attach_lambda_stream_access" {
name = "${aws_iam_role.kinesis_autoscaler_lambda_role.name}_attach_lambda_stream_access"
roles = [
aws_iam_role.kinesis_autoscaler_lambda_role.name
]
policy_arn = aws_iam_policy.lambda_access_stream.arn
}
最后一个很重要。这是我为使 lambda 工作所做的最后一次调整的结果。
resource "aws_iam_policy" "lambda_access_scaling" {
name = "${var.stream_name}-lambda-access-scaling-policy"
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "FindScalingPolicyARNAndAlarms",
"Effect": "Allow",
"Action": [
"application-autoscaling:DescribeScalingPolicies",
"cloudwatch:DescribeAlarms"
],
"Resource": "*"
},
{
"Sid": "UpdateAlarms",
"Effect": "Allow",
"Action": [
"cloudwatch:PutMetricAlarm",
"cloudwatch:DeleteAlarms"
],
"Resource": [
"${aws_cloudwatch_metric_alarm.alarm_out.arn}",
"${aws_cloudwatch_metric_alarm.alarm_in.arn}"
]
}
]
}
POLICY
}
resource "aws_iam_policy_attachment" "attach_scaling_access" {
name = "${aws_iam_role.kinesis_autoscaler_lambda_role.name}_attach_scaling_access"
roles = [
aws_iam_role.kinesis_autoscaler_lambda_role.name
]
policy_arn = aws_iam_policy.lambda_access_scaling.arn
}
另外,您希望拥有custom_appautoscaling...的正确权限...
# For reference:
resource "aws_appautoscaling_target" "kinesis_stream" {
min_capacity = var.min_number_of_shard
max_capacity = var.max_number_of_shard
resource_id = "${aws_api_gateway_deployment.gateway.invoke_url}/scalableTargetDimensions/${var.stream_name}"
role_arn = aws_iam_role.custom_appautoscaling_service_role.arn
scalable_dimension = "custom-resource:ResourceType:Property"
service_namespace = "custom-resource"
depends_on = [
aws_iam_policy_attachment.attach_base_policy,
]
lifecycle {
ignore_changes = [
# This is because the "assume_role_policy" becomes the actual
# Role "AWSServiceRoleForApplicationAutoScaling_CustomResource"
# at runtime and is always attemted to be recreated
role_arn,
]
}
}
# Actual policies:
resource "aws_iam_role" "custom_appautoscaling_service_role" {
name = "${var.stream_name}-assume-custom-resource"
assume_role_policy = <<-EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "custom-resource.application-autoscaling.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_iam_policy" "base_policy" {
name = "${var.stream_name}_base_policy"
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DescribeAlarms",
"Effect": "Allow",
"Action": [
"cloudwatch:DescribeAlarms"
],
"Resource": "*"
},
{
"Sid": "InvokeApiGateway",
"Effect": "Allow",
"Action": [
"execute-api:Invoke*"
],
"Resource": [
"${aws_api_gateway_deployment.gateway.execution_arn}/scalableTargetDimensions/${var.stream_name}"
]
}
]
}
POLICY
}
resource "aws_iam_policy_attachment" "attach_base_policy" {
name = "${var.stream_name}_attach_base_policy"
roles = [
aws_iam_role.custom_appautoscaling_service_role.name
]
policy_arn = aws_iam_policy.base_policy.arn
}
resource "aws_iam_policy" "alarms_modification" {
name = "${var.stream_name}_alarms_modification"
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "UpdateAlarms",
"Effect": "Allow",
"Action": [
"cloudwatch:PutMetricAlarm",
"cloudwatch:DeleteAlarms"
],
"Resource": [
"${aws_cloudwatch_metric_alarm.alarm_out.arn}",
"${aws_cloudwatch_metric_alarm.alarm_in.arn}"
]
}
]
}
POLICY
}
resource "aws_iam_policy_attachment" "attach_alarms_modification" {
name = "${var.stream_name}_attach_alarms_modification"
roles = [
aws_iam_role.custom_appautoscaling_service_role.name
]
policy_arn = aws_iam_policy.alarms_modification.arn
}
4。 Lambda 和 APIGateway 工作正常
最后,如果所有这些仍然不起作用,您可能需要对 lambda 和 APIGateway 进行故障排除...
我最终在 lambda 控制台中使用了 apigateway-aws-proxy Event Template。稍微编辑一下字段,使其看起来像这样:
{
"body": {
"desiredCapacity": "1"
},
"resource": "/{proxy+}",
"path": "/scalableTargetDimensions/my-custom-stream",
"httpMethod": "PATCH",
...
"requestContext" {
...
"path": "/<STAGE_NAME_SEE_API_GATEWAY_DEPLOYMENT_RESOURCE>/scalableTargetDimensions/my-custom-stream",
"resourcePath": "/{proxy+}",
"httpMethod": "PATCH",
}
对于这一部分,我还创建了一个具有base64encoded 主体的内容(因为我猜它是通过 APIGateway 传递的方式?)...
无论如何,我最终对 lambda 进行了微调,以便它同时接受两者,这样就可以知道 lambda 是否真的正常工作。
{
"body": "eyJkZXNpcmVkQ2FwYWNpdHkiOiIyIn0=",
"resource": "/{proxy+}",
"path": "/scalableTargetDimensions/my-custom-stream",
"httpMethod": "PATCH",
"isBase64Encoded": true,
...
有了这个,您应该能够对 lambda 进行故障排除,确保正确授予所有权限。
要对 APIGateway 进行故障排除,您始终可以使用 postman。它可以很好地处理对 AWS 的身份验证,因此如果您有足够访问您创建的 APIGateway 资源的凭据,您应该能够执行一些 GET 和 PATCH 来手动触发 APIGateway 并测试它集成的一部分。