创建 AWS DMS 迁移任务时出现 TableMapping 参数问题

Question

我正在使用 python boto3 创建 AWS DMS 迁移任务。指定 TableMapping 参数时出现错误。 代码：

response = dmsc.create_replication_task(
ReplicationTaskIdentifier='Abhisek-task12',
SourceEndpointArn='',
TargetEndpointArn='',
ReplicationInstanceArn='',
MigrationType='full-load',
TableMappings="file:////home/abhisek/shellscript/FinalScripts-copy/table- mapping-blog.json" 

json 文件：table-mapping-blog.json

{
  "TableMappings": 
  [
   {
    "Type": "Include", 
    "SourceSchema": "cloudthat_blog", 
    "SourceTable": "%" 
    }
   ]
  }

错误：

调用时发生错误（InvalidParameterValueException） CreateReplicationTask 操作：无效的表映射文档

我是否遗漏了 json 文件中的任何内容？

Answer 1

我也遇到过同样的问题。终于得到了this is a documentation bug for dms的信息。该文档描述了 TableMappings 值将是一个带有“file://mappings.json”前缀的 json 文件。但它实际上接受原始 json 字符串值。这是我已经成功完成mysql-mysql同构迁移的代码示例。

create_task.py：

ReplicationTaskIdentifier='simple-task'
SourceEndpointArn=''
TargetEndpointArn=''
# MigrationType='full-load'|'cdc'|'full-load-and-cdc'
MigrationType='full-load'

TableMappings="""{
    "rules": [
        {
            "rule-type": "selection",
            "rule-id": "1",
            "rule-name": "1",
            "object-locator": {
                "schema-name": "dms_mysql",
                "table-name": "%"
            },
            "rule-action": "include"
        }
    ]
}"""


response = client.create_replication_task(ReplicationTaskIdentifier=ReplicationTaskIdentifier,
                                          SourceEndpointArn=SourceEndpointArn,
                                          TargetEndpointArn=TargetEndpointArn,
                                          ReplicationInstanceArn=ReplicationInstanceArn,
                                          MigrationType=MigrationType,
                                          TableMappings=TableMappings
                                          ) 

为了清楚起见，我不知道您是如何获得这种 json 格式的，我从 AWS DMS ui 中的手动任务创建中获取了我的 json 字符串。尽管他们没有太多关于 json 格式的详细信息，但我的意思是 AWS API 接受或不接受的 json 密钥是什么。希望对你有帮助

Answer 2

文档确实以这种方式出现，我遇到了与您和@Hakuna-Matata 相同的问题，但是您可以将 JSON 文件传递​​到 CLI，我遇到了同样的错误，只是删除了数组和“表映射”标签

   {
    "Type": "Include", 
    "SourceSchema": "cloudthat_blog", 
    "SourceTable": "%" 
    }

我没有尝试过上述方法，但这对我来说就像一个冠军：

{ 
"rules": [ 
    { "rule-type": "selection", 
      "rule-id": "1", 
      "rule-name": "1", 
      "object-locator": 
                    { 
                        "schema-name": "SCHEMA-NAME", 
                        "table-name": "TABLE-NAME" 
                    }, 
      "rule-action": "include", 
      "filters": 
                [
                     { 
                        "filter-type": "source", 
                        "column-name": "DATE-COLUMN-TO-FILTER", 
                        "filter-conditions": 
                            [ 
                                { 
                                    "filter-operator": "gte", 
                                    "value": "2018-03-15 00:00:00" 
                                } 
                            ] 
                    } 
                ] 
    } 
   ] 
}

我对 --replication-task-settings 做了同样的事情

{  
   "TargetMetadata":{  
      "TargetSchema":"",
      "SupportLobs":false,
      "FullLobMode":false,
      "LobChunkSize":64,
      "LimitedSizeLobMode":true,
      "LobMaxSize":32,
      "LoadMaxFileSize":0,
      "ParallelLoadThreads":0,
      "ParallelLoadBufferSize":0,
      "BatchApplyEnabled":false,
      "TaskRecoveryTableEnabled":false
   },
   "FullLoadSettings":{  
      "TargetTablePrepMode":"DO_NOTHING",
...

然后运行它就可以了

aws dms create-replication-task  --replication-task-identifier dmsdynamodbtask-name --source-endpoint-arn my-source-endpoint-arn --target-endpoint-arn my-target-endpoint-arn --replication-instance-arn my-replication-instance-arn --migration-type full-load-and-cdc --table-mappings file://./TableMappings/my-table.json --replication-task-settings file://./ReplicationTaskSettings.json