【问题标题】:what the correct template mapping should be in the Integration Request aws api-getway?集成请求 aws api-gateway 中的正确模板映射应该是什么?
【发布时间】:2021-04-20 15:58:48
【问题描述】:

我正在开发一个事件流系统。而我的解决方案是:api-gatway->firehose data stream->data stream delivery->s3.

1.Send a json request(action is PUT) to api-getway
2.Integrate the request with Kinesis service
3.Valid data in the template mapping of the Integration Request
4.Delivery the data to data stream by firehose data delivery
5.Consume the stream and save into S3

它适用于以下数据 1.请求数据

{
   "Data" : "this is a test"
}

2.集成请求与firehose的模板映射代码

{ 
"StreamName" : "Stream",
"Data" : "$util.base64Encode($input.json('$.Data'))",
"PartitionKey" : "1" 

}

Execution log for request 4ad5f3a1-da7d-436f-bd3f-879ba045c622
Fri Jan 15 15:14:19 UTC 2021 : Starting execution for request: 4ad5f3a1-da7d-436f-bd3f-879ba045c622
Fri Jan 15 15:14:19 UTC 2021 : HTTP Method: PUT, Resource Path: /record
Fri Jan 15 15:14:19 UTC 2021 : Method request path: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request query string: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request headers: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request body before transformations: {
      "property":"1",
      "name":"name"
   }

Fri Jan 15 15:14:19 UTC 2021 : Endpoint request URI: https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:14:19 UTC 2021 : Endpoint request headers: {Authorization=***************************************************************************************************************************************************************************************************************************************************************************************9bb292, X-Amz-Date=20210115T151419Z, x-amzn-apigateway-api-id=450acvde3l, Accept=application/json, User-Agent=AmazonAPIGateway_450acvde3l, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEEAaCXVzLWVhc3QtMSJHMEUCIQC7W693nrtuUGrgUvOpn+PUKTRYTGAzRYVSWUPKoAc4eAIgGp/uH4yV/vynaMQUBNEBTwP79m0P9nVfCmw9CMWjY0Mq5QIIGBACGgw1MzUxMTE0MzMzNDAiDKy2z/4lnGJJJAWaOyrCAnANknvMUIbvf9diTZ4qYsdJ3LjETy9hE0f9pUZTBh7aKRjn2y4hbZc/oXGpsmHcgsgJAlIGD51gD0O5LR+N2EKjQeeo5GijkYvZvdbmxVRAZR27EhFzujCk3okePxAzVb3SeCQDWrBnEWn2VwqX4Y75QWwOgL8ZfcuxNKF/wbks0ukPFBo0p68KQ24PIXOYGMs9IAxHYa195+5mUMSi5Av62QKPpBbUAgZ2IJt9EKvRKyLYCs5XzS4lPCgsIbwouVB38f6UQKdQ9XsmD0cX04ODkdj2/0GHne6ufar7nANd3o08PmALd9mXK4z4gt/OxkZUe2AVLXvNQ4IodzW0WOUe1nFvq6YxpveDVfXKDRSK [TRUNCATED]
Fri Jan 15 15:14:19 UTC 2021 : Endpoint request body after transformations: { 
    "StreamName" : "EventStream",
    "Data" : "IiI=",
    "PartitionKey" : "1" 
}

Fri Jan 15 15:14:19 UTC 2021 : Sending request to https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:14:19 UTC 2021 : Received response. Status: 200, Integration latency: 21 ms
Fri Jan 15 15:14:19 UTC 2021 : Endpoint response headers: {x-amzn-RequestId=c028bf47-0059-8b93-9853-0cccfda9a977, x-amz-id-2=L15V+8AS8gN3HxJwiz0qVqi/UJn2xSWueRMnXdhqgjFw6TeuaRlYX62DZK9pa+O1PcKopTP55aHRLdhX0cQwovKefVpiuRtv, Date=Fri, 15 Jan 2021 15:14:19 GMT, Content-Type=application/x-amz-json-1.1, Content-Length=110}
Fri Jan 15 15:14:19 UTC 2021 : Endpoint response body before transformations: {"SequenceNumber":"49614572211779959518343530489315214421429290892684951554","ShardId":"shardId-000000000000"}
Fri Jan 15 15:14:19 UTC 2021 : Method response body after transformations: {"SequenceNumber":"49614572211779959518343530489315214421429290892684951554","ShardId":"shardId-000000000000"}
Fri Jan 15 15:14:19 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=1-6001b14b-bbccae126175224317a10a4e, Content-Type=application/json}
Fri Jan 15 15:14:19 UTC 2021 : Successfully completed execution
Fri Jan 15 15:14:19 UTC 2021 : Method completed with status: 200 

但不适用于以下: 1.请求数据

{
   "Data" : {
     "property":"1",
     "name":"name"       
   }
}

2.集成请求与firehose的模板映射代码

{ “流名称”:“流”, “数据” : { “属性”:$input.json('$.property'), "名称" : $input.json('$.name') }, “分区键”:“1” }

Execution log for request 05b707cc-d95c-40bf-8b75-375048697414
Fri Jan 15 15:39:34 UTC 2021 : Starting execution for request: 05b707cc-d95c-40bf-8b75-375048697414
Fri Jan 15 15:39:34 UTC 2021 : HTTP Method: PUT, Resource Path: /record
Fri Jan 15 15:39:34 UTC 2021 : Method request path: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request query string: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request headers: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request body before transformations: {
      "property":"1",
      "name":"name"
   }

Fri Jan 15 15:39:34 UTC 2021 : Endpoint request URI: https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:39:34 UTC 2021 : Endpoint request headers: {Authorization=***************************************************************************************************************************************************************************************************************************************************************************************a43c70, X-Amz-Date=20210115T153934Z, x-amzn-apigateway-api-id=450acvde3l, Accept=application/json, User-Agent=AmazonAPIGateway_450acvde3l, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEEAaCXVzLWVhc3QtMSJFMEMCHw0tTLi/gwtxUjwAQSl/CIY8aie2nmayl+Qsm6/i520CID9iXCFafaQTh4YqE1/tzvKgMO5IlYgFJrcNbAQB2Nc+KuUCCBkQAhoMNTM1MTExNDMzMzQwIgxeE22ch5hdFy3nB+UqwgJyKnQpnLuEY3zpcbRdEO5jks7yfx2+o1xfIz9Kga0S1PojPfzxh5aD/PthhP8D0jutv96ZVe8p52TwfSnv/z3YeDCFzsnw/U9kGFzGVt1pY2JMB4sg1vU7li8pFP/qiUQ3QA8cXbp4nWeE3kQGlPG4pjH0MsOvowTxM8G6yKosvCdD8fVyCJxWIjFnn1+dK9GGV/MnZlnaVqc57z0n0nrHgLjxBzDcDKJ5/xrgcqcYmUETFj8NyDJ9ESzCp0PhKJV9tGF4LgxbAgffe2Yw/3qpQyB6JNqJrZEczADp3gL0rjIBXhbnx5Yizs9MBMtoB9L22mAwEeqJRx4lK12wOqQZ5+0homLCugauYVoy3juNv/zW [TRUNCATED]
Fri Jan 15 15:39:34 UTC 2021 : Endpoint request body after transformations: { 
    "StreamName" : "EventStream",
    "Data" : { 
        "property" : "1",
        "name" : "name"
    },
    
    "PartitionKey" : "1" 
}

Fri Jan 15 15:39:34 UTC 2021 : Sending request to https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:39:34 UTC 2021 : Received response. Status: 400, Integration latency: 2 ms
Fri Jan 15 15:39:34 UTC 2021 : Endpoint response headers: {x-amzn-RequestId=cc937c57-744b-fca2-94e8-c5214b4386dd, x-amz-id-2=fqg8MLIbHOfeeFXp6wpii3l4yl32mI/5RyTwYQyzw9/OqpdLNqCBvBbTp8x7Q4YWAroefbfHb5IUEYeD68SQqh2bq87nL4vp, connection=close, Date=Fri, 15 Jan 2021 15:39:34 GMT, Content-Type=application/x-amz-json-1.1, Content-Length=99}
Fri Jan 15 15:39:34 UTC 2021 : Endpoint response body before transformations: {"__type":"SerializationException","Message":"Start of structure or map found where not expected."}
Fri Jan 15 15:39:34 UTC 2021 : Method response body after transformations: {"__type":"SerializationException","Message":"Start of structure or map found where not expected."}
Fri Jan 15 15:39:34 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=1-6001b736-328415f2db20383946bffd9e, Content-Type=application/json}
Fri Jan 15 15:39:34 UTC 2021 : Successfully completed execution
Fri Jan 15 15:39:34 UTC 2021 : Method completed with status: 200

参考:https://www.youtube.com/watch?v=0UxiV5sUlcA

【问题讨论】:

  • 我的错,我忘了在模板映射中编码。 :( 代码应该是:

标签: node.js amazon-web-services amazon-s3 stream amazon-kinesis-firehose


【解决方案1】:

我的错,我忘了在模板映射中进行编码。 代码应该是:

{ 
  "StreamName" : "Stream", 
  "Data" : "$util.base64Encode(
        { 
           "property" : $input.json('$.property'), 
           "name" : $input.json('$.name') 
        }
  ), 
  "PartitionKey" : "1" 
 }

【讨论】:

猜你喜欢
  • 2018-01-20
  • 2016-01-21
  • 1970-01-01
  • 2019-08-02
  • 2018-05-02
  • 2018-01-25
  • 2017-11-30
  • 2020-09-28
  • 2019-08-11
相关资源
最近更新 更多