将 xBRL 报告实例映射到 JSON #xbrl #python #json答案

【问题标题】：Mapping XBRL report instances to JSON将 xBRL 报告实例映射到 JSON #xbrl #python #json
【发布时间】：2021-01-01 02:17:18
【问题描述】：

我希望将 xBRL 报告实例（特别是 SEC 生成的财务报告）转换为 Python 字典或 JSON。我花时间使用 bs4（美丽的汤）开发代码，但理想情况下我想利用开源的 Arelle 库。

我的理解是Arelle软件包有一个名为“saveLoadableOIM”的插件。 xBRL.org 发布了一般指南；但是，它没有实际实施。

http://www.xbrl.org/Specification/xbrl-json/CR-2020-05-06/xbrl-json-CR-2020-05-06.html

我发现有关 Arelle 的命令提示符用法的文档已过时且不适用于 Python 3.x。谁能提供如何通过python命令提示符操作Arelle的指导；具体来说，如何将 SEC xBRL 报告实例转换为 JSON？我想要一个能够适应标准分类法未来变化的模型，尤其是 GAAP：

https://www.sec.gov/info/edgar/edgartaxonomies.shtml

如果有用于将以下 MSFT 10-K 的 xBRL 报告实例映射到 JSON 的示例代码将特别有帮助：

https://www.sec.gov/Archives/edgar/data/789019/000156459018019062/msft-20180630.xml

如果现有的 Arelle 库存在限制，我想了解这些限制。

谢谢，雷扎

【问题讨论】：

标签： python json python-3.x xbrl

【解决方案1】：

我必须解决同样的问题，并开发了一个通用的 xBRL 到 JSON 转换器，它可以自动适应美国 GAAP 分类的任何变化。

API 从 10-K（和 10-Q）文件中提取所有 xBRL 财务报表，对其进行标准化并将其转换为 JSON。使用 pandas 数据帧就像一个魅力。

您可以提供

xBRL 文件的 URL，例如MSFT 10-K 归档https://www.sec.gov/Archives/edgar/data/789019/000156459018019062/msft-20180630.xml
文件的 HTML 文件的 URL，例如https://www.sec.gov/Archives/edgar/data/789019/000156459018019062/msft-10k_20180630.htm
申请的登录号，例如0001564590-18-019062

标准化财务报表包括：

收入报表
StatementsOfIncomeParenthetical
综合收入报表
StatementsOfComprehensiveIncomeParenthetical
资产负债表
资产负债表括号
StatementsOfCashFlows
StatementsOfCashFlowsParenthetical
StatementsOfShareholdersEquity
StatementsOfShareholdersEquityParenthetical

ConsolidatedStatementsofOperations 或 ConsolidatedStatementsOfLossIncome 等变体会自动标准化为其根名称，例如StatementsOfIncome.

示例

API 支持 GET HTTP 方法。换句话说，您可以通过提供 xBRL 文件 URL 来发出简单的 GET 请求，然后 API 返回完全转换的 JSON。

将 xBRL 文件转换为 JSON

GET 请求到：

https://api.sec-api.io/xbrl-to-json?
xbrl-url=https://www.sec.gov/Archives/edgar/data/789019/000156459018019062/msft-20180630.xml&
token=YOUR_API_KEY

转换后的 JSON

注意：短版适合此处。

{
  "StatementsOfIncome": {
    "RevenueFromContractWithCustomerExcludingAssessedTax": [
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "value": "274515000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "value": "260174000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "value": "265595000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ProductMember"
        },
        "value": "220747000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ProductMember"
        },
        "value": "213883000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ProductMember"
        },
        "value": "225847000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ServiceMember"
        },
        "value": "53768000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ServiceMember"
        },
        "value": "46291000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ServiceMember"
        },
        "value": "39748000000"
      },

      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:IPhoneMember"
        },
        "value": "137781000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:IPhoneMember"
        },
        "value": "142381000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:IPhoneMember"
        },
        "value": "164888000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:MacMember"
        },
        "value": "28622000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:MacMember"
        },
        "value": "25740000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:MacMember"
        },
        "value": "25198000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:IPadMember"
        },
        "value": "23724000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:IPadMember"
        },
        "value": "21280000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:IPadMember"
        },
        "value": "18380000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:WearablesHomeandAccessoriesMember"
        },
        "value": "30620000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:WearablesHomeandAccessoriesMember"
        },
        "value": "24482000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "aapl:WearablesHomeandAccessoriesMember"
        },
        "value": "17381000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:AmericasSegmentMember"
        },
        "value": "124556000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:AmericasSegmentMember"
        },
        "value": "116914000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:AmericasSegmentMember"
        },
        "value": "112093000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:EuropeSegmentMember"
        },
        "value": "68640000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:EuropeSegmentMember"
        },
        "value": "60288000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:EuropeSegmentMember"
        },
        "value": "62420000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:GreaterChinaSegmentMember"
        },
        "value": "40308000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:GreaterChinaSegmentMember"
        },
        "value": "43678000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:GreaterChinaSegmentMember"
        },
        "value": "51942000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:JapanSegmentMember"
        },
        "value": "21418000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:JapanSegmentMember"
        },
        "value": "21506000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:JapanSegmentMember"
        },
        "value": "21733000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:RestOfAsiaPacificSegmentMember"
        },
        "value": "19593000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:RestOfAsiaPacificSegmentMember"
        },
        "value": "17788000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "us-gaap:StatementBusinessSegmentsAxis",
          "value": "aapl:RestOfAsiaPacificSegmentMember"
        },
        "value": "17407000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "country:US"
        },
        "value": "109197000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "country:US"
        },
        "value": "102266000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "country:US"
        },
        "value": "98061000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "country:CN"
        },
        "value": "40308000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "country:CN"
        },
        "value": "43678000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "country:CN"
        },
        "value": "51942000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "aapl:OtherCountriesMember"
        },
        "value": "125010000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "aapl:OtherCountriesMember"
        },
        "value": "114230000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:StatementGeographicalAxis",
          "value": "aapl:OtherCountriesMember"
        },
        "value": "115592000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2020-06-28",
          "endDate": "2020-09-26"
        },
        "value": "64698000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2020-03-29",
          "endDate": "2020-06-27"
        },
        "value": "59685000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-12-29",
          "endDate": "2020-03-28"
        },
        "value": "58313000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2019-12-28"
        },
        "value": "91819000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-06-30",
          "endDate": "2019-09-28"
        },
        "value": "64040000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-03-31",
          "endDate": "2019-06-29"
        },
        "value": "53809000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-12-30",
          "endDate": "2019-03-30"
        },
        "value": "58015000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2018-12-29"
        },
        "value": "84310000000"
      }
    ],
    "CostOfGoodsAndServicesSold": [
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ProductMember"
        },
        "value": "151286000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ProductMember"
        },
        "value": "144996000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ProductMember"
        },
        "value": "148164000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ServiceMember"
        },
        "value": "18273000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ServiceMember"
        },
        "value": "16786000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "segment": {
          "dimension": "srt:ProductOrServiceAxis",
          "value": "us-gaap:ServiceMember"
        },
        "value": "15592000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "value": "169559000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "value": "161782000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2017-10-01",
          "endDate": "2018-09-29"
        },
        "value": "163756000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2019-09-29",
          "endDate": "2020-09-26"
        },
        "segment": [
          {
            "dimension": "us-gaap:DerivativeInstrumentRiskAxis",
            "value": "us-gaap:ForeignExchangeContractMember"
          },
          {
            "dimension": "us-gaap:ReclassificationOutOfAccumulatedOtherComprehensiveIncomeAxis",
            "value": "us-gaap:ReclassificationOutOfAccumulatedOtherComprehensiveIncomeMember"
          },
          {
            "dimension": "us-gaap:StatementEquityComponentsAxis",
            "value": "us-gaap:AccumulatedGainLossNetCashFlowHedgeParentMember"
          }
        ],
        "value": "-584000000"
      },
      {
        "decimals": "-6",
        "unitRef": "usd",
        "period": {
          "startDate": "2018-09-30",
          "endDate": "2019-09-28"
        },
        "segment": [
          {
            "dimension": "us-gaap:DerivativeInstrumentRiskAxis",
            "value": "us-gaap:ForeignExchangeContractMember"
          },
          {
            "dimension": "us-gaap:ReclassificationOutOfAccumulatedOtherComprehensiveIncomeAxis",
            "value": "us-gaap:ReclassificationOutOfAccumulatedOtherComprehensiveIncomeMember"
          },
          {
            "dimension": "us-gaap:StatementEquityComponentsAxis",
            "value": "us-gaap:AccumulatedGainLossNetCashFlowHedgeParentMember"
          }
        ],
        "value": "-482000000"
      }
    ]
  }
}

【讨论】：

【解决方案2】：

我在 Python 3 下调用 Arelle：

python3 $HOME/Arelle/arelleCmdLine.py

这是在 Linux 上，假设我已在我的主目录中以 Arelle 签出 Arelle。

要加载插件，请使用--plugins 并为其指定Arelle/arelle/plugin 目录下的文件名（不带.py 扩展名）。例如，--plugins=saveLoadableOIM。然后您可以添加--help，您应该会看到帮助消息中包含的其他选项。

这对我有用：

python3 $HOME/Arelle/arelleCmdLine.py --plugins=saveLoadableOIM --saveLoadableOIM=msft.json -f https://www.sec.gov/Archives/edgar/data/789019/000156459018019062/msft-20180630.xml

使用真棒jq提取数据的示例：

jq '[.facts[] | select( .dimensions.concept | test(":GrossProfit$") )] | sort_by(.dimensions.period)[-1]' msft.json

这将获取最新的 GrossProfit 值：

{
  "value": "20343000000",
  "decimals": -6,
  "dimensions": {
    "concept": "us-gaap:GrossProfit",
    "entity": "cik:0000789019",
    "period": "2018-04-01T00:00:00/2018-07-01T00:00:00",
    "unit": "iso4217:USD"
  }
}

我应该注意的是，xBRL-JSON 规范尚未最终确定，并且此 JSON 的格式可能会随着时间的推移而略有变化。我希望一旦 Arelle 可用，它就会更新到最终版本。

【讨论】：