【问题标题】:Why search performance is difference between from&size and search&after为什么搜索性能在 from&size 和 search&after 之间存在差异
【发布时间】:2021-11-03 06:35:47
【问题描述】:

我的索引中有数亿个文档。搜索的时候发现search&after比from&size慢很多。用from&size,搜索很快,花了几毫秒返回,但是用search after,花了20秒。我的搜索结果是按时间和key排序的(一个关键字copy of _id) ,为什么?有什么区别? 搜索命令:

{
    "query":{
        "bool":{
            "filter":[
                {
                    "query_string":{
                        "query":"*"
                    }
                },
                {
                    "range":{
                        "__time__":{
                            "gte":1324958207,
                            "lte":1724958207
                        }
                    }
                }
            ]
        }
    },
    "size":10,
    "sort":[
        {
            "__time__":{
                "order":"desc"
            }
        },
        {
            "__key__":{
                "order":"desc"
            }
        }
    ],
    "search_after":[
        1630594662000,
        "6130e666-2-67e9e3-f5-1"
    ],
    "profile":true
}

简介:

{"searches":[
                    {
                        "query":[
                            {
                                "type":"BoostQuery",
                                "description":"(ConstantScore(DocValuesFieldExistsQuery [field=__time__]))^0.0",
                                "time_in_nanos":45722536283,
                                "breakdown":{
                                    "set_min_competitive_score_count":0,
                                    "match_count":0,
                                    "shallow_advance_count":0,
                                    "set_min_competitive_score":0,
                                    "next_doc":45722241414,
                                    "match":0,
                                    "next_doc_count":410919487,
                                    "score_count":0,
                                    "compute_max_score_count":0,
                                    "compute_max_score":0,
                                    "advance":19517,
                                    "advance_count":39,
                                    "score":0,
                                    "build_scorer_count":78,
                                    "create_weight":14271,
                                    "shallow_advance":0,
                                    "create_weight_count":1,
                                    "build_scorer":261081
                                },
                                "children":[
                                    {
                                        "type":"DocValuesFieldExistsQuery",
                                        "description":"DocValuesFieldExistsQuery [field=__time__]",
                                        "time_in_nanos":16571715415,
                                        "breakdown":{
                                            "set_min_competitive_score_count":0,
                                            "match_count":0,
                                            "shallow_advance_count":0,
                                            "set_min_competitive_score":0,
                                            "next_doc":16571493898,
                                            "match":0,
                                            "next_doc_count":410919487,
                                            "score_count":0,
                                            "compute_max_score_count":0,
                                            "compute_max_score":0,
                                            "advance":15074,
                                            "advance_count":39,
                                            "score":0,
                                            "build_scorer_count":78,
                                            "create_weight":517,
                                            "shallow_advance":0,
                                            "create_weight_count":1,
                                            "build_scorer":205926
                                        }
                                    }
                                ]
                            }
                        ],
                        "rewrite_time":116538,
                        "collector":[
                            {
                                "name":"PagingFieldCollector",
                                "reason":"search_top_hits",
                                "time_in_nanos":30851166561
                            }
                        ]
                    }
                ],
                "aggregations":[

                ]
            }

【问题讨论】:

    标签: elasticsearch search full-text-search


    【解决方案1】:

    因为search_after 使用scroll 方法进行过滤,该方法应该在过滤之前对所有数据进行排序,而不是from/size 仅将元素作为流检索,并且随着您深入,它会变慢并占用内存带有from

    【讨论】:

    • 但在我的测试中,search after 比 from size 慢 10 倍,两者的偏移量为零。
    • 是的,这就是逻辑,就像说的那样,你有一个需要全局视图,而另一个只是顶部响应的流
    猜你喜欢
    • 1970-01-01
    • 2017-12-31
    • 1970-01-01
    • 1970-01-01
    • 2015-02-12
    • 1970-01-01
    • 1970-01-01
    • 2016-03-23
    • 1970-01-01
    相关资源
    最近更新 更多