尝试通过 Python requests.post() 在 Elasticsearch 中添加索引和文档时出错

Question

我正在尝试通过requests.post() 方法将索引和文档添加到 Elasticsearch：

elastic_url = 'http://localhost:9200/'
elastic_headers = {'Content-Type': 'application/json'}
test_json = json.dumps({"id": 1, "foo": "bar"})

requests.post(elastic_url + 'test/_doc/1', 
              json=test_json, 
              headers=elastic_headers)

得到以下错误作为响应：

{'error': {'root_cause': [{'type': 'mapper_parsing_exception',
           'reason': 'failed to parse'}],
 'type': 'mapper_parsing_exception',
 'reason': 'failed to parse',
 'caused_by': {'type': 'not_x_content_exception',
               'reason': 'Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes'}},
 'status': 400}

通过curl 在终端中执行相同操作时：

curl -X POST -H 'Content-Type: application/json' 'http://localhost:9200/test/_doc/1' -d '{"id": 1, "foo": "bar"}'

数据写入成功。 不知道是什么原因，请帮帮我。

Answer 1

当您发布请求时，您再次转储了 json，而您所要做的就是发送 test_json。

看看这是否有效：

requests.post(elastic_url + 'test/_doc/1', 
              json=test_json, 
              headers=headers)

这可能会修复映射错误！如果您仍然收到如下映射解析异常，请更新您的映射或检查您提交的数据。

'caused_by': {'type': 'not_x_content_exception',
               'reason': 'Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes'}}

Mapping 示例：

# mapping tells es what kind of data each field contains

mapping = {
  "mappings": {
      "properties": {
          "id": {
            "type": "long"
          },
          "foo": {
            "type": "text"
          }
         }
        }
       }

你需要对这篇文章使用请求吗？

如果没有，我可以推荐python elasticsearch client吗？你可以像下面那样做。我经常需要创建索引和映射并将文档上传到 elasticsearch，这就是我的做法。 （注意 - 我将所有这些都包装成易于使用的功能，但我想我只是向你展示胆量！）

from elasticsearch import Elasticsearch
from elasticsearch import helpers

# bulk generator function with the es required formatting for successful bulk upload. 
# the required dict is the index and source line below 
def bulk_gen(json_data):

    bulk_action = ({"_index": es_index,"_source": j,} for j in json_data if json_data is not None)
    
    for b in bulk_action:
        yield b
        print(f'processed {es_index}')

# connect to elasticsearch 
es = Elasticsearch(cloud_id=cloud_id,
                   http_auth=('username', 'password')
# name index 
es_index = 'index name here'

# create index, add mapping
es.indices.create(index=es_index, body=mapping, ignore=400)

# process your json data 
json_data = json.loads({"id": 1, "foo": "bar"})

# finally, let's upload everything! 
# we've already created an index and mapping and processed the json data into bulk upload format, now we let 'er rip       

try: 
    helpers.bulk(es, bulk_gen(json_data))
    print('Imported resource succesfully')
except Exception as e: 
    print(f'there was an error: {e}')

希望这会有所帮助。我苦苦挣扎，然后开始使用 es python 客户端，事情变得容易了很多。