使用jq提取可变数量的字段

【问题标题】：Extracting variable number of fields using jq使用jq提取可变数量的字段
【发布时间】：2015-12-19 05:30:17
【问题描述】：

我是这个网站的新手，之所以来到这里，是因为我真的在努力解决从 JSON 文件中提取信息的问题。棘手的部分是字段数量可变，所以我无法摆脱简单的语法。

这是一个示例代码：

{
  "addresses": {
    "@count": "1",
    "address_name": {
      "address_spec": {
        "@addr_no": "1",
        "full_address": "Tel Aviv Univ, Eitan Berglas Sch Econ, IL-69978 Tel Aviv, Israel",
        "organizations": {
          "@count": "2",
          "organization": [
            "Tel Aviv Univ",
            {
              "@pref": "Y",
              "#text": "Tel Aviv University"
            }
          ]
        },
        "suborganizations": {
          "@count": "1",
          "suborganization": "Eitan Berglas Sch Econ"
        },
        "city": "Tel Aviv",
        "country": "Israel",
        "zip": {
          "@location": "BC",
          "#text": "IL-69978"
        }
      }
    }
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "1",
    "address_name": {
      "address_spec": {
        "@addr_no": "1",
        "full_address": "MIT, Cambridge, MA 02139 USA",
        "organizations": {
          "@count": "2",
          "organization": [
            "MIT",
            {
              "@pref": "Y",
              "#text": "Massachusetts Institute of Technology (MIT)"
            }
          ]
        },
        "city": "Cambridge",
        "state": "MA",
        "country": "USA",
        "zip": {
          "@location": "AP",
          "#text": "02139"
        }
      }
    }
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "2",
    "address_name": [
      {
        "address_spec": {
          "@addr_no": "1",
          "full_address": "Univ Kentucky, Lexington, KY 40506 USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "Univ Kentucky",
              {
                "@pref": "Y",
                "#text": "University of Kentucky"
              }
            ]
          },
          "city": "Lexington",
          "state": "KY",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "40506"
          }
        }
      },
      {
        "address_spec": {
          "@addr_no": "2",
          "full_address": "Univ Bonn, ZEI, D-5300 Bonn, Germany",
          "organizations": {
            "@count": "2",
            "organization": [
              "Univ Bonn",
              {
                "@pref": "Y",
                "#text": "University of Bonn"
              }
            ]
          },
          "suborganizations": {
            "@count": "1",
            "suborganization": "ZEI"
          },
          "city": "Bonn",
          "country": "Germany",
          "zip": {
            "@location": "BC",
            "#text": "D-5300"
          }
        }
      }
    ]
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "1",
    "address_name": {
      "address_spec": {
        "@addr_no": "1",
        "full_address": "Harvard Univ, Cambridge, MA 02138 USA",
        "organizations": {
          "@count": "2",
          "organization": [
            "Harvard Univ",
            {
              "@pref": "Y",
              "#text": "Harvard University"
            }
          ]
        },
        "city": "Cambridge",
        "state": "MA",
        "country": "USA",
        "zip": {
          "@location": "AP",
          "#text": "02138"
        }
      }
    }
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "3",
    "address_name": [
      {
        "address_spec": {
          "@addr_no": "1",
          "full_address": "Columbia Univ, New York, NY 10027 USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "Columbia Univ",
              {
                "@pref": "Y",
                "#text": "Columbia University"
              }
            ]
          },
          "city": "New York",
          "state": "NY",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "10027"
          }
        }
      },
      {
        "address_spec": {
          "@addr_no": "2",
          "full_address": "NYU, New York, NY USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "NYU",
              {
                "@pref": "Y",
                "#text": "New York University"
              }
            ]
          },
          "city": "New York",
          "state": "NY",
          "country": "USA"
        }
      },
      {
        "address_spec": {
          "@addr_no": "3",
          "full_address": "Univ Pompeu Fabra, Barcelona, Spain",
          "organizations": {
            "@count": "2",
            "organization": [
              "Univ Pompeu Fabra",
              {
                "@pref": "Y",
                "#text": "Pompeu Fabra University"
              }
            ]
          },
          "city": "Barcelona",
          "country": "Spain"
        }
      }
    ]
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "2",
    "address_name": [
      {
        "address_spec": {
          "@addr_no": "1",
          "full_address": "Univ Chicago, Chicago, IL 60637 USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "Univ Chicago",
              {
                "@pref": "Y",
                "#text": "University of Chicago"
              }
            ]
          },
          "city": "Chicago",
          "state": "IL",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "60637"
          }
        }
      },
      {
        "address_spec": {
          "@addr_no": "2",
          "full_address": "Amer Bar Fdn, Chicago, IL 60611 USA",
          "organizations": {
            "@count": "1",
            "organization": "Amer Bar Fdn"
          },
          "city": "Chicago",
          "state": "IL",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "60611"
          }
        }
      }
    ]
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "2",
    "address_name": [
      {
        "address_spec": {
          "@addr_no": "1",
          "full_address": "Ohio State Univ, Columbus, OH 43210 USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "Ohio State Univ",
              {
                "@pref": "Y",
                "#text": "Ohio State University"
              }
            ]
          },
          "city": "Columbus",
          "state": "OH",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "43210"
          }
        }
      },
      {
        "address_spec": {
          "@addr_no": "2",
          "full_address": "Harvard Univ, Cambridge, MA 02138 USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "Harvard Univ",
              {
                "@pref": "Y",
                "#text": "Harvard University"
              }
            ]
          },
          "city": "Cambridge",
          "state": "MA",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "02138"
          }
        }
      }
    ]
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "1",
    "address_name": {
      "address_spec": {
        "@addr_no": "1",
        "full_address": "Univ Chicago, Chicago, IL 60637 USA",
        "organizations": {
          "@count": "2",
          "organization": [
            "Univ Chicago",
            {
              "@pref": "Y",
              "#text": "University of Chicago"
            }
          ]
        },
        "city": "Chicago",
        "state": "IL",
        "country": "USA",
        "zip": {
          "@location": "AP",
          "#text": "60637"
        }
      }
    }
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "2",
    "address_name": [
      {
        "address_spec": {
          "@addr_no": "1",
          "full_address": "Wissensch Zentrum Berlin Sozialforsch, D-1000 Berlin, Germany",
          "organizations": {
            "@count": "1",
            "organization": "Wissensch Zentrum Berlin Sozialforsch"
          },
          "city": "Berlin",
          "country": "Germany",
          "zip": {
            "@location": "BC",
            "#text": "D-1000"
          }
        }
      },
      {
        "address_spec": {
          "@addr_no": "2",
          "full_address": "Harvard Univ, Dept Govt, Cambridge, MA 02138 USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "Harvard Univ",
              {
                "@pref": "Y",
                "#text": "Harvard University"
              }
            ]
          },
          "suborganizations": {
            "@count": "1",
            "suborganization": "Dept Govt"
          },
          "city": "Cambridge",
          "state": "MA",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "02138"
          }
        }
      }
    ]
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}
{
  "addresses": {
    "@count": "2",
    "address_name": [
      {
        "address_spec": {
          "@addr_no": "1",
          "full_address": "NYU, CV Starr Ctr Appl Econ, New York, NY 10003 USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "NYU",
              {
                "@pref": "Y",
                "#text": "New York University"
              }
            ]
          },
          "suborganizations": {
            "@count": "1",
            "suborganization": "CV Starr Ctr Appl Econ"
          },
          "city": "New York",
          "state": "NY",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "10003"
          }
        }
      },
      {
        "address_spec": {
          "@addr_no": "2",
          "full_address": "Princeton Univ, Princeton, NJ 08544 USA",
          "organizations": {
            "@count": "2",
            "organization": [
              "Princeton Univ",
              {
                "@pref": "Y",
                "#text": "Princeton University"
              }
            ]
          },
          "city": "Princeton",
          "state": "NJ",
          "country": "USA",
          "zip": {
            "@location": "AP",
            "#text": "08544"
          }
        }
      }
    ]
  },
  "category_info": {
    "headings": {
      "@count": "1",
      "heading": "Social Sciences"
    },
    "subjects": {
      "@count": "3",
      "subject": [
        {
          "@ascatype": "traditional",
          "#text": "Economics"
        },
        {
          "@ascatype": "extended",
          "#text": "Business & Economics"
        },
        {
          "@ascatype": "traditional",
          "#text": "ECONOMICS"
        }
      ]
    }
  }
}

我希望为每条记录提取一个国家（有些记录有多个国家，这似乎是导致问题的原因）。所以我幼稚的做法是说：

.static_data."fullrecord_metadata".addresses.address_name.country

然而，这给了我几个错误（null 没有键，并且不能用字符串索引数组）。使用keys命令检查：

.static_data."fullrecord_metadata".addresses.address_name | keys

我可以看到数据的结构方式似乎存在问题......

那么，您能否建议我是否可以使用 jq 实际提取每个条目的国家列表？谢谢！

【问题讨论】：

    标签： json jq


    【解决方案1】：

    对于每个输入的顶级 JSON 实体，以下过滤器将递归检查所有对象以查看它们是否具有“国家”键，然后它将报告该顶级实体的不同“国家”值： 

    jq -c '[.. | if type == "object" and has("country") 
             then .country
             else empty end] | unique' 
["Israel"]
["USA"]
["Germany","USA"]
["USA"]
["Spain","USA"]
["USA"]
["USA"]
["USA"]
["Germany","USA"]
["USA"]

    这是一个过滤器，将在您的示例中产生相同的结果，尽管它不完全等效：

    [.. | .country? // empty] | unique

    [有兴趣的读者练习：有什么区别？ :-)]

    【讨论】：

    • 非常感谢！ :) 这对我来说太先进了，无法自己弄清楚。
    • .country? // empty 表示.country如果存在且不为null则应使用，否则将替换为空。 unique 删除重复条目。
    【解决方案2】：

    这是一个使用函数来处理.address_name中的变化的解决方案

     def address_specs:
    if type == "array" then .[].address_spec else .address_spec end
 ;

 .addresses | .address_name | [address_specs | .country] | unique

    【讨论】：

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2023-03-09
      • 1970-01-01
      • 2022-11-24
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多
      热门标签
      Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode