如何在弹性搜索中获得嵌套字段的不同值？答案

【问题标题】：How can I get distinct values of nested fields in elasticsearch?如何在弹性搜索中获得嵌套字段的不同值？
【发布时间】：2019-09-05 09:45:54
【问题描述】：

我在 elasticsearch 中有以下文档结构：

   root
|-- userid: string (nullable = true)
|-- name: string (nullable = true)
|-- applications: array (nullable = true)
|    |-- element: struct (containsNull = true)
|    |    |-- applicationid: string (nullable = true)
|    |    |-- createdat: string (nullable = true)
|    |    |-- source_name: string (nullable = true)
|    |    |-- accounts: array (nullable = true)
|    |    |    |-- element: struct (containsNull = true)
|    |    |    |    |-- applicationcreditreportaccountid: string 
                                                   (nullable = true)
|    |    |    |    |-- account_type: integer (nullable = true)
|    |    |    |    |-- account_department: string (nullable = true)

下面是我的索引的映射：

{
  "bureau_data" : {
    "mappings" : {
      "dynamic_date_formats" : [
        "yyyy-MM-dd"
      ],
      "dynamic_templates" : [
        {
          "objects" : {
            "match_mapping_type" : "object",
            "mapping" : {
              "type" : "nested"
            }
          }
        }
      ],
      "properties" : {
        "raw_derived" : {
          "type" : "nested",
          "properties" : {
            "applications" : {
              "type" : "nested",
              "properties" : {
                "accounts" : {
                  "type" : "nested",
                  "properties" : {
                    "account_type_name" : {
                      "type" : "text",
                      "fields" : {
                        "keyword" : {
                          "type" : "keyword",
                          "ignore_above" : 256
                        }
                      }
                    },
                    "accounttypeid" : {
                      "type" : "text",
                      "fields" : {
                        "keyword" : {
                          "type" : "keyword",
                          "ignore_above" : 256
                        }
                      }
                    },
                    "applicationcreditreportaccountid" : {
                      "type" : "text",
                      "fields" : {
                        "keyword" : {
                          "type" : "keyword",
                          "ignore_above" : 256
                        }
                      }
                    },
                    "currentbalance" : {
                      "type" : "long"
                    },
                    "dayspastdue" : {
                      "type" : "long"
                    },
                    "institution_name" : {
                      "type" : "text",
                      "fields" : {
                        "keyword" : {
                          "type" : "keyword",
                          "ignore_above" : 256
                        }
                      }
                    },
                    "institutionid" : {
                      "type" : "text",
                      "fields" : {
                        "keyword" : {
                          "type" : "keyword",
                          "ignore_above" : 256
                        }
                      }
                    }
                  }
                },
                "applicationcreditreportid" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "applicationid" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "createdat" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "creditbureautypeid" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "dateofbirth" : {
                  "type" : "date",
                  "format" : "yyyy-MM-dd"
                },
                "firstname" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "lastname" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "score" : {
                  "type" : "long"
                },
                "source_name" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "status" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "updatedat" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                }
              }
            },
            "dob" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "firstname" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "lastname" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "middlename" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "mobilephone" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "source" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            }
          }
        },
        "userid" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          },
          "fielddata" : true
        }
      }
    }
  }
}

我想要作为嵌套字段的 account_type 字段的不同值。我试过查询，它只给我不同的计数。

GET /my_index/_search?size=0
{
  "aggs": {
    "nested_path": {
      "nested": {
        "path": "raw_derived.applications.accounts"
      },
      "aggs": {
        "distinct_values": {
          "cardinality": {
            "field": "raw_derived.applications.accounts.account_type.keyword"
          }
        }
      }
    }
  }
}

我希望输出具有不同的 account_type 值，但输出只是计数。下面是我的输出 sn-p：

  "hits" : {
    "total" : {
      "value" : 50,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "nested_path" : {
      "doc_count" : 828,
      "distinct_values" : {
        "value" : 70
      }
    }
  }
}

以下是我尝试过的查询及其工作：

GET /bureau_data/_search?size=0
{
 "_source": "{aggregations}", 
 "aggs": {
   "unique": {
     "nested": {
       "path": "raw_derived.applications"
     },
         "aggs": {
           "score_unq": {
             "terms": {
               "field": "raw_derived.applications.source_name.keyword"
             }
           }
         }
       }
     }
   }

任何建议都会有帮助

【问题讨论】：

如果加上弹性映射就好了。 GET index_name/_mapping 而不是结构 :)
确定 Richa。我已经用映射更新了我的问题

标签： elasticsearch nested kibana elasticsearch-dsl

【解决方案1】：

来自官方文档 - 基数聚合：- 计算不同值的近似计数的单值指标聚合。值可以从文档中的特定字段中提取或由脚本生成。

不要按“基数”聚合，而是尝试如下聚合：

{
"size":0,
"aggregations": {
    "distinct_values": {
      "terms": {
        "field": "raw_derived.applications.accounts.account_type.keyword",
        "size": 1000,
        "min_doc_count": 1,
        "order": [
          {
            "_count": "desc"
          },
          {
            "_key": "asc"
          }
        ]
      }
    }
}

【讨论】：

是的，我只是尝试使用术语，但我想我必须先给出嵌套路径...它的工作原理。非常感谢您的输入。我已经更新了我的查询