如何从 JSON 中提取特定部分并将其转换为 CSV答案

【问题标题】：How to extract specific section from JSON and convert it to CSV如何从 JSON 中提取特定部分并将其转换为 CSV
【发布时间】：2018-03-12 15:14:22
【问题描述】：

部分 output.json（文件有多个相同的部分-仅角色/策略名称不同）完整链接：https://1drv.ms/u/s!AizscpxS0QM4hJl_VRQaWbm6D8T8_w

{
  "AssumeRolePolicyDocument": {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Action": "sts:AssumeRoleWithSAML",
        "Effect": "Allow",
        "Condition": {
          "StringEquals": {
            "SAML:aud": "https://signin.aws.amazon.com/saml"
          }
        },
        "Principal": {
          "Federated": "arn:aws:iam::279052847476:saml-provider/LastPass"
        }
      }
    ]
  },
  "RoleId": "AROAI3SYPHW7ZSS4TSO6Y",
  "CreateDate": "2017-05-23T22:29:53Z",
  "InstanceProfileList": [],
  "RoleName": "AWSCorpAdmin",
  "Path": "/",
  "AttachedManagedPolicies": [
    {
      "PolicyName": "AdministratorAccess",
      "PolicyArn": "arn:aws:iam::aws:policy/AdministratorAccess"
    }
  ],
  "RolePolicyList": [],
  "Arn": "arn:aws:iam::279052847476:role/AWSCorpAdmin"
}

如何获取以下输出（将 IAM 角色映射到策略）并使用 jq 将其转换为 csv 文件？

jq -r '.AssumeRolePolicyDocument[]| .fields.RoleName |  @csv' output.json
jq: error (at output.json:11895): Cannot iterate over null (null)

期望的输出：

AWSCorpAdmin,AdministratorAccess

我需要遍历整个 JSON 文件并获取每个角色的策略（以上只是示例）

【问题讨论】：

标签： json bash jq aws-cli

【解决方案1】：

jq解决方案：

jq -r '.RoleDetailList | map(select(.AttachedManagedPolicies | length > 0))[] 
       | .RoleName as $r | .AttachedManagedPolicies[] 
       | [$r, .PolicyName] | @csv' output.json

样本输出：

"ADFS-amtest-ro","pol-amtest-ro"
"adfs-host-role","pol-amtest-ro"
"aws-elasticbeanstalk-ec2-role","AWSElasticBeanstalkWebTier"
"aws-elasticbeanstalk-ec2-role","AWSElasticBeanstalkMulticontainerDocker"
"aws-elasticbeanstalk-ec2-role","AWSElasticBeanstalkWorkerTier"
"aws-elasticbeanstalk-service-role","AWSElasticBeanstalkEnhancedHealth"
"aws-elasticbeanstalk-service-role","AWSElasticBeanstalkService"
"AWSAccCorpAdmin","AdministratorAccess"
"AWScompanyCorpAdmin","AdministratorAccess"
"AWScompanyCorpPowerUser","PowerUserAccess"
"AWSServiceRoleForAutoScaling","AutoScalingServiceRolePolicy"
"AWSServiceRoleForElasticBeanstalk","AWSElasticBeanstalkServiceRolePolicy"
"AWSServiceRoleForElasticLoadBalancing","AWSElasticLoadBalancingServiceRolePolicy"
"AWSServiceRoleForOrganizations","AWSOrganizationsServiceTrustPolicy"
"AWSServiceRoleForRDS","AmazonRDSServiceRolePolicy"
"Cloudyn","ReadOnlyAccess"
"DatadogAWSIntegrationRole","DatadogAWSIntegrationPolicy"
"datadog_alert_metrics_role","AWSLambdaBasicExecutionRole-66abe1f2-cee8-4a90-a026-061b24db1b02"
"dev-instance-role","dev-instance-role-policy"
"ecsInstanceRole","AmazonEC2ContainerServiceforEC2Role"
"ecsServiceRole","AmazonEC2ContainerServiceRole"
"companySAMLUser","AdministratorAccess"
"irole-matlabscheduler","pol-marketdata-rw"
"lambda-ec2-ami-role","lambda-ec2-ami-policy"
"lambda_api_gateway_twilio_processor","AWSLambdaBasicExecutionRole-f47a6b57-b716-4740-b2c6-a02fa6480153"
"lambda_api_gateway_twilio_processor","AWSLambdaSNSPublishPolicyExecutionRole-d31a9f16-80e7-47c9-868a-f162396cccf6"
"OneLoginAdmin","AdministratorAccess"
"OneLoginDev","PowerUserAccess"
"rds-monitoring-role","AmazonRDSEnhancedMonitoringRole"
"role-amtest-ro","pol-amtest-ro"
...

【讨论】：

jq: error (at output.json:12033): Cannot iterate over null (null), doesn't work, 这是更大的 JSON 文件的一部分，AssumePolicyRoleDocumen 的部分更多跨度>
@Milister，除非您发布它，否则无法猜测您的“更大的 json”。如果不是很重要，你可以分享一个完整的 json 链接，比如说通过 pastebin.com，这样我就可以看到并处理它