【问题标题】:Confused about `search_as_you_type` ngram subfields对“search_as_you_type”ngram 子字段感到困惑
【发布时间】:2021-02-27 00:35:38
【问题描述】:

我正在尝试将“键入时搜索”功能添加到 Elasticsearch 中名为 email_address 的字段中。我对from the docs 的理解是,如果我创建一个search_as_you_type 字段,它应该会自动创建为查找部分匹配而优化的ngram 子字段。

但是,它似乎没有按我预期的方式工作,而且我似乎没有从这种特殊字段类型中获得预期的好处。

首先,我创建了一个索引:

$ curl -s -H 'Content-Type: application/json' -XPUT http://localhost:9200/mytestindex -d '
{
  "mappings": {
    "properties": {
      "email_address": {"type": "search_as_you_type"}
    }
  }
}
'

当我请求新创建的电子邮件字段时,我看到的是:

$ curl -s -H 'Content-Type: application/json' http://localhost:9200/mytestindex/_mapping/field/email_address | json_pp
{
   "mytestindex" : {
      "mappings" : {
         "email_address" : {
            "full_name" : "email_address",
            "mapping" : {
               "email_address" : {
                  "max_shingle_size" : 3,
                  "type" : "search_as_you_type"
               }
            }
         }
      }
   }
}

最后,我填充了一些示例数据:

$ curl -s -H 'Content-Type: application/json' http://localhost:9200/mytestindex/_doc -d '
{"email_address": "sam@example.com"}'

$ curl -s -H 'Content-Type: application/json' http://localhost:9200/mytestindex/_doc -d '
{"email_address": "sally@example.com"}'

$ curl -s -H 'Content-Type: application/json' http://localhost:9200/mytestindex/_doc -d '
{"email_address": "jane@example.com"}'

$ curl -s -H 'Content-Type: application/json' http://localhost:9200/mytestindex/_doc -d '
{"email_address": "samantha@example.com"}'

官方文档建议使用 bool_prefix multi_match 搜索以下字段:email_addressemail_address._2gramemail_address._3gram。好奇地尝试子字段,我测试了只包含它们的搜索,但我无法得到任何结果:

$ curl -s -H 'Content-Type: application/json' http://localhost:9200/mytestindex/_search -d '
{
  "query": {
    "multi_match": {
      "query": "sa",
      "type": "bool_prefix",
      "fields": [
        "email_address._2gram",
        "email_address._3gram"
      ]
    }
  }
}
' | json_pp

{
   "hits" : {
      "hits" : [],
      "max_score" : null,
      "total" : {
         "value" : 0,
         "relation" : "eq"
      }
   },
   "took" : 4,
   "_shards" : {
      "skipped" : 0,
      "successful" : 1,
      "total" : 1,
      "failed" : 0
   },
   "timed_out" : false
}

我尝试了各种长度的部分查询(ssasam 等),但从未得到任何结果。

当我执行相同的搜索但只包含 email_address 字段本身时,我会得到我期望的所有结果:

curl -s -H 'Content-Type: application/json' http://localhost:9200/mytestindex/_search -d '
{
  "query": {
    "multi_match": {
      "query": "sa",
      "type": "bool_prefix",
      "fields": [
        "email_address"
      ]
    }
  }
}
' | json_pp
{
   "timed_out" : false,
   "hits" : {
      "max_score" : 1,
      "total" : {
         "relation" : "eq",
         "value" : 3
      },
      "hits" : [
         {
            "_index" : "mytestindex",
            "_id" : "gEbkCXUBC6_J-EeLAygM",
            "_score" : 1,
            "_type" : "_doc",
            "_source" : {
               "email_address" : "sam@example.com"
            }
         },
         {
            "_index" : "mytestindex",
            "_source" : {
               "email_address" : "sally@example.com"
            },
            "_score" : 1,
            "_type" : "_doc",
            "_id" : "gUbkCXUBC6_J-EeLWigu"
         },
         {
            "_index" : "mytestindex",
            "_id" : "jUb5CXUBC6_J-EeL1ij1",
            "_type" : "_doc",
            "_score" : 1,
            "_source" : {
               "email_address" : "samantha@example.com"
            }
         }
      ]
   },
   "took" : 2,
   "_shards" : {
      "failed" : 0,
      "skipped" : 0,
      "successful" : 1,
      "total" : 1
   }
}

因此,我不明白_2gram_3gram 子字段有什么好处。我是否设置错误?还是我对这些字段的实际用途感到困惑?

【问题讨论】:

  • 您有机会浏览我的回答吗,期待您的反馈????

标签: elasticsearch


【解决方案1】:

search_as_you_type 字段类型是一个类似文本的字段,它是 优化以提供对服务于你输入的查询的支持 完成用例

添加一个包含索引数据、映射、搜索查询和搜索结果的工作示例

索引映射:

{
  "mappings": {
    "properties": {
      "title": {
        "type": "search_as_you_type"
      }
    }
  }
}

索引数据:

{"title": "how shingles are actually used"}

分析 API

elasticsearch 中默认的分词器是“标准分词器”,它使用基于语法的分词技术

为文本生成的单个标记是

{
  "tokens": [
    {
      "token": "how",
      "start_offset": 0,
      "end_offset": 3,
      "type": "<ALPHANUM>",
      "position": 0
    },
    {
      "token": "shingles",
      "start_offset": 4,
      "end_offset": 12,
      "type": "<ALPHANUM>",
      "position": 1
    },
    {
      "token": "are",
      "start_offset": 13,
      "end_offset": 16,
      "type": "<ALPHANUM>",
      "position": 2
    },
    {
      "token": "actually",
      "start_offset": 17,
      "end_offset": 25,
      "type": "<ALPHANUM>",
      "position": 3
    },
    {
      "token": "used",
      "start_offset": 26,
      "end_offset": 30,
      "type": "<ALPHANUM>",
      "position": 4
    }
  ]
}

制作三个单词的带状疱疹

POST/_analyze

{
  "tokenizer": "standard",
  "filter": [
    {
      "type": "shingle",
      "min_shingle_size": 3,
      "max_shingle_size": 3,
      "output_unigrams":false
    }
  ],
  "text": "how shingles are actually used"
}

生成的令牌是:

{
  "tokens": [
    {
      "token": "how shingles are",
      "start_offset": 0,
      "end_offset": 16,
      "type": "shingle",
      "position": 0
    },
    {
      "token": "shingles are actually",
      "start_offset": 4,
      "end_offset": 25,
      "type": "shingle",
      "position": 1
    },
    {
      "token": "are actually used",
      "start_offset": 13,
      "end_offset": 30,
      "type": "shingle",
      "position": 2
    }
  ]
}

搜索查询:

title._3gram - 用 shingle 令牌包装 my_field 的分析器 3 号木瓦过滤器

{
  "query": {
    "multi_match": {
      "query": "shingles are actually",
      "type": "bool_prefix",
      "fields": [
        "title._3gram"
      ]
    }
  }
}

搜索结果:

"hits": [
      {
        "_index": "test",
        "_type": "_doc",
        "_id": "1",
        "_score": 1.0,
        "_source": {
          "title": "how shingles are actually used"
        }
      }
    ]

在您的情况下,考虑到 "text": "samantha@example.com",生成的单个令牌是:samanthaexample.com

创建 2 个单词的 shingle 时,生成的标记是:

{
  "tokens": [
    {
      "token": "samantha example.com",
      "start_offset": 0,
      "end_offset": 20,
      "type": "shingle",
      "position": 0
    }
  ]
}

因此,当您使用sa 搜索时,它不会匹配,因为没有生成与之对应的标记。 当使用带有布尔前缀查询的多重匹配时(在email_address 字段上,它匹配是因为" type": "bool prefix"。阅读此内容以了解有关Match bool prefix query 的更多信息。

如果你想用sa查询,并得到所有结果,那么你可以使用Completion suggestor,甚至可以通过UAX URL Email Tokenizer

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2012-01-04
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2019-09-13
    • 2012-07-22
    相关资源
    最近更新 更多