通过python将json数据转储到elasticsearch答案

【问题标题】：Dumping json data into elasticsearch through python通过python将json数据转储到elasticsearch
【发布时间】：2016-05-26 09:49:23
【问题描述】：

我正在尝试将我从 csv 文件解析的一些 json 数据转储到 elasticsearch 中。我检查了数据及其正确的格式。但由于某种原因，我收到以下错误。

{
  "error" : {
    "root_cause" : [ {
      "type" : "parse_exception",
      "reason" : "Failed to derive xcontent"
    } ],
    "type" : "parse_exception",
    "reason" : "Failed to derive xcontent"
  },
  "status" : 400
}

这是我正在使用的命令。我不确定为什么它不起作用

curl -XPUT 'http://192.168.99.100:9200/_bulk?pretty' --data-binary "@data.json"

编辑：这是我的 json 数据的一部分。我知道错误可能来自此，但我不知道它是什么。我已经阅读了 elasticsearch 上的 Bulk_API。

{"index": {"_index": "jobs", "_id": 119556, "_type": "2014_jobs"}}
{"job category": "Logistics / Supply Chain|Purchasing / Merchandising|Human resource consultancy services|Full Time|Executive|Manager|Middle Management|", "closing date": "28-Dec-14", "salary": "Not published", "posting date": "28-Nov-14", "working hours": "-", "company": "", "contact": "+65 66454545 / ctay sg. drakeintl. com", "description": "", "title": "Logistics Category Manager", "job level": "Executive|Manager|Middle Management", "shift pattern": "Day Shift", "job id": "JOB-2014-0119556", "industry": "Human resource consultancy services", "employment type": "Full Time", "min years of experience": "8", "skills": "", "timestamp": "1.41973E+12", "address": "1 RAFFLES PLACE| 20-01 ONE RAFFLES PLACE||Singapore 048616|"}
{"index": {"_index": "jobs", "_id": 119700, "_type": "2014_jobs"}}
{"job category": "F B|Hospitality|Logistics / Supply Chain|Purchasing / Merchandising|Hotels with restaurant|Full Time|Fresh/entry level|Non-executive|", "closing date": "28-Dec-14", "salary": "Not published", "posting date": "28-Nov-14", "working hours": "44 hours a week", "company": "DAISHO DEVELOPMENT SINGAPORE PTE LTD", "contact": "Not available", "description": "", "title": "Culinary Logistic Agent", "job level": "Fresh/entry level|Non-executive", "shift pattern": "No Shift", "job id": "JOB-2014-0119700", "industry": "Hotels with restaurant", "employment type": "Full Time", "min years of experience": "1", "skills": "", "timestamp": "1.41973E+12", "address": "12 MARINA VIEW| 35-00 ASIA SQUARE TOWER 2||Singapore 018961|"}
{"index": {"_index": "jobs", "_id": 118701, "_type": "2014_jobs"}}
{"job category": "Architecture / Interior Design|Architectural services|Permanent|", "closing date": "30-Dec-14", "salary": "Not published", "posting date": "27-Nov-14", "working hours": "8am - 6pm", "company": "LOOK ARCHITECTS PTE. LTD. ", "contact": "jasmin lookarchitects. com", "description": "", "title": "Resident Technical Officer (RTO)", "job level": "-", "shift pattern": "No Shift", "job id": "JOB-2014-0118701", "industry": "Architectural services", "employment type": "Permanent", "min years of experience": "5", "skills": "", "timestamp": "1.41973E+12", "address": "18 BOON LAY WAY| 09-135 TRADEHUB 21||Singapore 609966|"}

【问题讨论】：

你写的是“我已经检查过数据”但是有一个parse_exception ...请提供一些示例代码，更重要的是一些示例数据，否则任何人都可能很难弄清楚问题到底是什么。现在我不得不猜测。
嘿，是的，我在发布此消息后不久就意识到了这一点。我将从我的 json 数据中发布一些代码摘录。我似乎无法弄清楚出了什么问题。我已经阅读了 bulk_api，它遵循了他们的约定。
数据似乎正确，stackoverflow.com/questions/37457267/…我的回答能解决问题吗？
嘿，由于某种原因，这并不能解决问题。这次我也遇到了不同的错误。 "reason" : "内容格式错误，解析后发现多余数据：START_OBJECT"
这个错误听起来像是 json 数据是问题，但我是通过 python 脚本生成的，所以我不明白为什么如果前几行没问题可能会出现问题：S跨度>

标签： python json elasticsearch

【解决方案1】：

尝试将 _bulk 替换为 _update

curl -XPUT 'http://192.168.99.100:9200/_update?pretty' --data-binary "@data.json"

参考：https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html

【讨论】：

更新也不起作用。我认为错误是我的 json 对象的格式。

【解决方案2】：

您遇到的错误与另一个问题相同，即未提供数据文件的完整路径：link to the answer to that question。

在这种情况下更改为：

curl -XPUT 'http://192.168.99.100:9200/_bulk?pretty' --data-binary "@data.json"

到：

curl -XPUT 'http://192.168.99.100:9200/_bulk?pretty' --data-binary "@/full/path/to/data.json"

应该解决问题。

【讨论】：