【问题标题】:Aggregating on dictionaries in nest ElasticSearch在嵌套 ElasticSearch 中聚合字典
【发布时间】:2021-04-25 00:14:43
【问题描述】:

所以我有一组索引产品,其中包含一个带有单个键的字典和一个值列表,我正在尝试使用它们构建一个多面搜索。不过我是一个弹性十足的新手。

    Product Product {
    Dictionary<string, List<string>> Properties
    //extra fields removed for simplicity
    }

属性可能类似于

["Color":["Blue","Yellow","Red"],"Size":["Small","Medium","Large"] 

["Material":["Wood"], "Shape":["Circle","Square"], "Size":["Tiny","Medium","Large","Huge"]

我希望编写几个聚合,它们将返回键以及这些键的值。 即如果上面的例子被索引,第一个聚合将返回一个包含"Color","Size","Material","Shape"的桶

第二个聚合将返回 4 个桶,每个桶都有每个键的唯一值。 即Size:["Tiny","small","medium","large","huge"] 等等

我意识到我需要一个嵌套聚合,但是我的尝试都没有带回存储桶中的任何东西。任何指针将不胜感激。这是我目前所拥有的。

    var ProductsQuery = client.Search<Product>(s => s
        .Index("products")
        .Query(q => q.MatchAll())
        .Aggregations(a => a
            .Nested("properties", n => n
                .Path(p => p.Properties.Suffix("keyword"))
                    .Aggregations(a => a
                        .Terms("property-keys", t => t
                            .Field(f => f.Properties.Keys.Suffix("keyword"))))));

编辑一些要求的细节:

当前的属性映射(它似乎正在为每个 Key 创建一个新映射,我不确定这是否是典型的?)我没有把整个对象映射放在这里,因为它相当大。产品有很多领域:

"properties" : {
  "properties" : {
    "Colour" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Equipment" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Football Age" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Football Size" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Frame Weight" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Garment" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Head Shape" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Level" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Product" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Size" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Sport" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Surface" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Type" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Unit" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "Weight" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "comparer" : {
      "type" : "object"
    },
    "count" : {
      "type" : "integer"
    },
    "item" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "keys" : {
      "properties" : {
        "count" : {
          "type" : "integer"
        }
      }
    },
    "values" : {
      "properties" : {
        "count" : {
          "type" : "integer"
        }
      }
    }
  }
}

和一些索引文档

"hits" : [
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134550",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Type" : [
          "Sleds"
        ],
        "Product" : [
          "Sleds"
        ],
        "Colour" : [
          "Black"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134566",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Sport" : [
          "Fitness"
        ],
        "Type" : [
          "Corner",
          "Edge",
          "Middle"
        ],
        "Size" : [
          "10mm",
          "15mm",
          "20mm"
        ],
        "Product" : [
          "Floor Matting"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134576",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Sport" : [
          "Rugby"
        ],
        "Type" : [
          "Skills Training"
        ],
        "Equipment" : [
          "Rugby Balls"
        ],
        "Size" : [
          "4",
          "5"
        ],
        "Level" : [
          "Skills"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134579",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Sport" : [
          "Rugby"
        ],
        "Type" : [
          "Match Union"
        ],
        "Equipment" : [
          "Rugby Balls"
        ],
        "Size" : [
          "4",
          "5"
        ],
        "Level" : [
          "Club",
          " School"
        ],
        "Unit" : [
          "12 Pack",
          "Each"
        ],
        "Colour" : [
          "Blue",
          "Red",
          "White"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134600",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Sport" : [
          "Rugby"
        ],
        "Size" : [
          "Large",
          "Medium",
          "Small",
          "X/Large",
          "X/Small",
          "XX/Small",
          "XXX/Small"
        ],
        "Garment" : [
          "Gloves"
        ],
        "Colour" : [
          "Red"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134601",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Sport" : [
          "Netball"
        ],
        "Size" : [
          "Large",
          "X/Large",
          "X/Small",
          "XX/Small",
          "XXX/Small"
        ],
        "Garment" : [
          "Gloves"
        ],
        "Colour" : [
          "Red"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134609",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Sport" : [
          "Netball"
        ],
        "Size" : [
          "Large",
          "Medium",
          "Small",
          "X/Large",
          "X/Small",
          "XXX/Small"
        ],
        "Garment" : [
          "Gloves"
        ],
        "Colour" : [
          "Black",
          "Green"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134617",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Sport" : [
          "Football"
        ],
        "Type" : [
          "Training"
        ],
        "Football Size" : [
          "2"
        ],
        "Equipment" : [
          "Footballs"
        ],
        "Size" : [
          "4",
          "5"
        ],
        "Unit" : [
          "12 Pack",
          "Each"
        ],
        "Weight" : [
          "290",
          "360"
        ],
        "Surface" : [
          "Grass",
          " Astroturf"
        ],
        "Football Age" : [
          "9-14 years",
          " 14+ years"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134548",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Type" : [
          "Sleds"
        ],
        "Product" : [
          "Sleds"
        ],
        "Colour" : [
          "Black",
          "Grey"
        ]
      }
    }
  },
  {
    "_index" : "products-20-01-2021-13-49-08",
    "_type" : "_doc",
    "_id" : "134558",
    "_score" : 1.0,
    "_source" : {
      "properties" : {
        "Sport" : [
          "Squash"
        ],
        "Equipment" : [
          "Squash Rackets"
        ],
        "Size" : [
          "27\""
        ],
        "Head Shape" : [
          "Bridged Closed Throat"
        ],
        "Frame Weight" : [
          "Over 160g"
        ]
      }
    }
  }
]

【问题讨论】:

  • 介意共享索引映射吗?也许是几个实际的文档,而不仅仅是“类似的东西”:)
  • @JoeSorocin 嘿乔,感谢您的关注,我已经添加了属性字段的映射,并在编辑中添加了一些文档

标签: .net elasticsearch nest


【解决方案1】:

首先,为了获得这些存储桶,您可以使用 Query DSL 执行以下操作:

POST products-*/_search
{
  "size": 0,
  "aggs": {
    "by_Colour": {
      "terms": {
        "field": "properties.Colour.keyword"
      }
    },
    "by_Size": {
      "terms": {
        "field": "properties.Size.keyword"
      }
    }
  }
}

然后需要将其翻译成 NEST 代码——我敢肯定那里有很多示例。


但是您的观察是正确的——ES 自动创建了许多映射。 我建议不要使用当前的数据格式,而是使用:

{
  "properties": [
    {
      "key": "Type",
      "values": ["Sleds"]
    },
    {
      "key": "Product",
      "values": ["Sleds"]
    },
    {
      "key": "Colour",
      "values": ["Black"]
    }
  ]
}

并制作properties 类型为nested

这样您将查询共享路径 properties.key 并聚合到 properties.values.keyword

请记住,nested 字段类型需要它们自己的实际 nested mapping - 如果没有正确的映射,您将无法使用嵌套查询。

查看related questionthis answer 了解更多上下文。

【讨论】:

  • 很高兴我能帮上忙。顺便说一句——我正在整理一本 Elasticsearch 手册,其中一些章节更详细地处理了嵌套字段。当你开始使用 ES 时,我认为整个事情都会对你有益。 Let me know 您到底想了解什么,我会在手册发布时通知您。谢谢
  • 刚刚实施了相关帖子中的建议,它们运行良好。我一定会留意你的书。非常感谢!
  • 太棒了!不要忘记在上面的链接注册?
猜你喜欢
  • 2015-07-30
  • 2021-12-14
  • 2015-11-18
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2017-09-13
  • 2023-04-03
  • 1970-01-01
相关资源
最近更新 更多