elasticsearch match_phrase 查询用于精确的子字符串搜索

Question

我使用 match_phrase 查询进行搜索全文匹配。

但它并没有像我想象的那样工作。

查询：

POST /_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match_phrase": {
            "browsing_url": "/critical-illness"
          }
        }
      ],
      "minimum_should_match": 1
    }
  }
}

结果：

"hits" : [
      {
        "_source" : {
          "browsing_url" : "https://www.google.com/url?q=https://industrytoday.co.uk/market-research-industry-today/global-critical-illness-commercial-insurance-market-to-witness-a-pronounce-growth-during-2020-2025&usg=afqjcneelu0qvjfusnfjjte1wx0gorqv5q"
        }
      },
      {
        "_source" : {
          "browsing_url" : "https://www.google.com/search?q=critical+illness"
        }
      },
      {
        "_source" : {
          "browsing_url" : "https://www.google.com/search?q=critical+illness&tbm=nws"
        }
      },
      {
        "_source" : {
          "browsing_url" : "https://www.google.com/search?q=do+i+have+a+critical+illness+-insurance%3f"
        }
      },
      {
        "_source" : {
          "browsing_url" : "https://www.google.com/search?q=do+i+have+a+critical+illness%3f"
        }
      }
    ]

期望：

To only get results where the given string is an exact sub-string in the field. For example:

https://www.example.com/critical-illness OR
https://www.example.com/critical-illness-insurance

映射：

"browsing_url": {
  "type": "text",
  "norms": false,
  "fields": {
    "keyword": {
      "type": "keyword",
      "ignore_above": 256
    }
  }
}

结果不是我所期望的。我希望得到与搜索 /critical-illness 完全一样的结果作为存储文本的子字符串。

Answer 1

您看到意外结果的原因是，您的搜索查询和字段本身都是通过analyzer 运行的。分析器会将文本分解为可以搜索的单个术语列表。这是一个使用 _analyze 端点的示例：

GET _analyze
{
  "analyzer": "standard",
  "text": "example.com/critical-illness"
}

{
  "tokens" : [
    {
      "token" : "example.com",
      "start_offset" : 0,
      "end_offset" : 11,
      "type" : "<ALPHANUM>",
      "position" : 0
    },
    {
      "token" : "critical",
      "start_offset" : 12,
      "end_offset" : 20,
      "type" : "<ALPHANUM>",
      "position" : 1
    },
    {
      "token" : "illness",
      "start_offset" : 21,
      "end_offset" : 28,
      "type" : "<ALPHANUM>",
      "position" : 2
    }
  ]
}

因此，虽然您的文档的真实值为 example.com/critical-illness，但在幕后 Elasticsearch 只会使用此令牌列表进行匹配。您的搜索查询也是如此，因为您使用的是 match_phrase，它对传入的短语进行标记。最终结果是 Elasticsearch 尝试将标记列表 ["critical", "illness"] 与您的文档标记列表进行匹配。

大多数时候standard analyzer 可以很好地删除不必要的标记，但是在您的情况下，您关心/ 这样的字符，因为您想匹配它们。解决此问题的一种方法是使用不同的分析器，例如 reversed path hierarchy analyzer。以下是如何配置此分析器并将其用于您的 browsing_url 字段的示例：

PUT /browse_history
{
  "settings": {
    "analysis": {
      "analyzer": {
        "url_analyzer": {
          "tokenizer": "url_tokenizer"
        }
      },
      "tokenizer": {
        "url_tokenizer": {
          "type": "path_hierarchy",
          "delimiter": "/",
          "reverse": true
        }
      }
    }
  }, 
  "mappings": {
    "properties": {
      "browsing_url": {
        "type": "text",
        "norms": false,
        "analyzer": "url_analyzer",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

现在，如果您分析一个 URL，您现在会看到 URL 路径保持完整：

GET browse_history/_analyze
{
  "analyzer": "url_analyzer",
  "text": "example.com/critical-illness?src=blah"
}

{
  "tokens" : [
    {
      "token" : "example.com/critical-illness?src=blah",
      "start_offset" : 0,
      "end_offset" : 37,
      "type" : "word",
      "position" : 0
    },
    {
      "token" : "critical-illness?src=blah",
      "start_offset" : 12,
      "end_offset" : 37,
      "type" : "word",
      "position" : 0
    }
  ]
}

这让您可以使用match_phrase_prefix 来查找 URL 包含 critical-illness 路径的所有文档：

POST /browse_history/_search
{
  "query": {
    "match_phrase_prefix": {
      "browsing_url": "critical-illness"
    }
  }
}

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.7896894,
    "hits" : [
      {
        "_index" : "browse_history",
        "_type" : "_doc",
        "_id" : "3",
        "_score" : 1.7896894,
        "_source" : {
          "browsing_url" : "https://www.example.com/critical-illness"
        }
      }
    ]
  }
}

编辑：

修订前的先前答案是使用关键字字段和regexp，但这是一个非常昂贵的查询。

POST /browse_history/_search
{
  "query": {
    "regexp": {
      "browsing_url.keyword": ".*/critical-illness"
    }
  }
}