当字段包含 ~ 时过滤弹性搜索数据

Question

我有一堆文件，如下所示。我想过滤 projectkey 以 ~ 开头的数据。 我确实读过一些文章，其中说 ~ 是弹性查询中的一个运算符，因此不能真正过滤。 有人可以帮助形成 /branch/_search API 的搜索查询吗？？

{
  "_index": "branch",
  "_type": "_doc",
  "_id": "GAz-inQBJWWbwa_v-l9e",
  "_version": 1,
  "_score": null,
  "_source": {
    "branchID": "refs/heads/feature/12345",
    "displayID": "feature/12345",
    "date": "2020-09-14T05:03:20.137Z",
    "projectKey": "~user",
    "repoKey": "deploy",
    "isDefaultBranch": false,
    "eventStatus": "CREATED",
    "user": "user"
  },
  "fields": {
    "date": [
      "2020-09-14T05:03:20.137Z"
    ]
  },
  "highlight": {
    "projectKey": [
      "~@kibana-highlighted-field@user@/kibana-highlighted-field@"
    ],
    "projectKey.keyword": [
      "@kibana-highlighted-field@~user@/kibana-highlighted-field@"
    ],
    "user": [
      "@kibana-highlighted-field@user@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1600059800137
  ]
}

更新***

我在下面使用 prerana 的答案在我的查询中使用 -prefix

当我使用前缀和范围时仍然有问题 - 我得到以下错误 - 我错过了什么？？

GET /branch/_search
{
  "query": {
    "prefix": {
      "projectKey": "~"
    },
    "range": {
      "date": {
        "gte": "2020-09-14",
        "lte": "2020-09-14"
      }
    }
  }
}



    {
  "error": {
    "root_cause": [
      {
        "type": "parsing_exception",
        "reason": "[prefix] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
        "line": 6,
        "col": 5
      }
    ],
    "type": "parsing_exception",
    "reason": "[prefix] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
    "line": 6,
    "col": 5
  },
  "status": 400
}

Answer 1

如果我很好地理解了您的问题，我建议创建一个自定义分析器来搜索特殊字符 ~。

在将~ 替换为__SPECIAL__ 时，我在本地进行了如下测试：

我创建了一个带有自定义char_filter 的索引，并在projectKey 字段中添加了一个字段。新的 multi_field 的名称是special_characters。

这是映射：

PUT wildcard-index
{
"settings": {
    "analysis": {
    "char_filter": {
        "special-characters-replacement": {
        "type": "mapping",
        "mappings": [
            "~ => __SPECIAL__"
        ]
        }
    },
    "analyzer": {
        "special-characters-analyzer": {
        "tokenizer": "standard",
        "char_filter": [
            "special-characters-replacement"
        ]
        }
    }
    }
},
"mappings": {
    "properties": {
    "projectKey": {
        "type": "text",
        "fields": {
        "special_characters": {
            "type": "text",
            "analyzer": "special-characters-analyzer"
        }
        }
    }
    }
}
}

然后我在索引中摄取了以下内容：

"projectKey": "content1 ~"

"projectKey": "这个~是内容"

"projectKey": "~路上的汽车"

"projectKey": "o ~ngram"

然后，查询是：

GET wildcard-index/_search
{
"query": {
    "match": {
    "projectKey.special_characters": "~"
    }
}
}

回复是：

"hits" : [
  {
    "_index" : "wildcard-index",
    "_type" : "_doc",
    "_id" : "h1hKmHQBowpsxTkFD9IR",
    "_score" : 0.43250346,
    "_source" : {
      "projectKey" : "content1 ~"
    }
  },
  {
    "_index" : "wildcard-index",
    "_type" : "_doc",
    "_id" : "iFhKmHQBowpsxTkFFNL5",
    "_score" : 0.3034693,
    "_source" : {
      "projectKey" : "This ~ is a content"
    }
  },
  {
    "_index" : "wildcard-index",
    "_type" : "_doc",
    "_id" : "-lhKmHQBowpsxTkFG9Kg",
    "_score" : 0.3034693,
    "_source" : {
      "projectKey" : "~ cars on the road"
    }
  }
]

如果您有任何问题，请告诉我，我很乐意为您提供帮助。

注意：如果~ 后面有空格，则此方法有效。从响应中可以看出第四条数据没有显示出来。

Answer 2

虽然@hansley 的回答会起作用，但它需要您创建一个自定义分析器，并且仍然如您所提到的，您只想获得以~ 开头的文档，但在他的结果中我看到所有包含~ 的文档，因此提供我的答案，这需要非常少的配置并按要求工作。

索引映射默认，因此只需在文档下方建立索引，ES 将为所有 text 字段创建一个默认映射，其中包含 .keyword 字段

索引示例文档

{
    "title" : "content1 ~"
}

{
    "title" : "~ staring with"
}

{
    "title" : "in between ~ with"
}

搜索查询应仅从示例文档中获取第二个文档

{
  "query": {
    "prefix" : { "title.keyword" : "~" }
  }
}

以及搜索结果

"hits": [
            {
                "_index": "pre",
                "_type": "_doc",
                "_id": "2",
                "_score": 1.0,
                "_source": {
                    "title": "~ staring with"
                }
            }
        ]

更多信息请参考prefix query

更新 1：

索引映射：

{
  "mappings": {
    "properties": {
      "date": {
        "type": "date" 
      }
    }
  }
}

索引数据：

{
    "date": "2015-02-01",
    "title" : "in between ~ with"
}
{
    "date": "2015-01-01",
    "title": "content1 ~"
}
{
    "date": "2015-02-01",
     "title" : "~ staring with"
}
{
    "date": "2015-02-01",
    "title" : "~ in between with"
}

搜索查询：

{
    "query": {
        "bool": {
            "must": [
                {
                    "prefix": {
                        "title.keyword": "~"
                    }
                },
                {
                    "range": {
                        "date": {
                            "lte": "2015-02-05",
                            "gte": "2015-01-11"
                        }
                    }
                }
            ]
        }
    }
}

搜索结果：

"hits": [
      {
        "_index": "stof_63924930",
        "_type": "_doc",
        "_id": "2",
        "_score": 2.0,
        "_source": {
          "date": "2015-02-01",
          "title": "~ staring with"
        }
      },
      {
        "_index": "stof_63924930",
        "_type": "_doc",
        "_id": "4",
        "_score": 2.0,
        "_source": {
          "date": "2015-02-01",
          "title": "~ in between with"
        }
      }
    ]