如何为父子索引创建映射并搜索由父级过滤的子级

Question

目标：

我想创建一个包含 2 个实体的父子索引。一个profile 和一个comment。 profile（为简单起见）具有自定义 ID（UUID 转换为字符串）、年龄和位置 (GeoPoint)。 comment（为简单起见）具有自定义 id（UUID 转换为字符串）。有了这些信息，我希望能够搜索给定一些针对配置文件的过滤数据的所有 cmets。例如，我想通过年龄在 26 到 36 岁之间并且位于纬度 100 公里以内的所有 cmets：3.0，long 5.0。

类：

// Profile.kt
import org.elasticsearch.common.geo.GeoPoint
import org.springframework.data.annotation.Id
import org.springframework.data.elasticsearch.annotations.Document
import org.springframework.data.elasticsearch.annotations.Field
import org.springframework.data.elasticsearch.annotations.FieldType
import org.springframework.data.elasticsearch.annotations.GeoPointField

@Document(indexName = "message_board", createIndex = false, type = "profile")
data class Profile(
    @Id
    val profileId: String,
    @Field(type = FieldType.Short, store = true)
    val age: Short,
    @GeoPointField
    val location: GeoPoint
)

// Comment.kt
import org.springframework.data.annotation.Id
import org.springframework.data.elasticsearch.annotations.Document
import org.springframework.data.elasticsearch.annotations.Field
import org.springframework.data.elasticsearch.annotations.FieldType
import org.springframework.data.elasticsearch.annotations.Parent

@Document(indexName = "message_board", createIndex = false, type = "comment")
data class Comment(
    @Id
    val commentId: String,
    @Field(type = FieldType.Text, store = true)
    @Parent(type = "profile")
    val parentId: String
)

// RestClientConfig.kt
import org.elasticsearch.client.RestHighLevelClient
import org.springframework.context.annotation.Configuration
import org.springframework.data.elasticsearch.client.ClientConfiguration
import org.springframework.data.elasticsearch.client.RestClients
import org.springframework.data.elasticsearch.config.AbstractElasticsearchConfiguration

@Configuration
class RestClientConfig(
    private val elasticSearchConfig: ElasticSearchConfig
) : AbstractElasticsearchConfiguration() {
    override fun elasticsearchClient(): RestHighLevelClient {
        val clientConfiguration: ClientConfiguration = ClientConfiguration.builder()
            .connectedTo("${elasticSearchConfig.endpoint}:${elasticSearchConfig.port}")
            .build()
        return RestClients.create(clientConfiguration).rest()
    }
}

// Controller.kt
import org.springframework.web.bind.annotation.RestController
import org.springframework.data.elasticsearch.core.ElasticsearchOperations
import org.springframework.data.elasticsearch.core.mapping.IndexCoordinates
import org.springframework.http.MediaType
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RequestMapping

@RestController
@RequestMapping("/", produces = [MediaType.APPLICATION_JSON_VALUE])
class Controller constructor(
    private val elasticsearchOperations: ElasticsearchOperations
) {
    init {
        elasticsearchOperations.indexOps(IndexCoordinates.of("message_board")).let { indexOp ->
            if (!indexOp.exists() && indexOp.create()) {
                val profileMapping = indexOp.createMapping(Profile::class.java)
                println("Profile Mapping: $profileMapping")
                indexOp.putMapping(profileMapping)
                val commentMapping = indexOp.createMapping(Comment::class.java)
                println("Comment Mapping: $commentMapping")
                indexOp.putMapping(commentMapping)
                indexOp.refresh()
            }
        }
    }

    @GetMapping("comments")
    fun getComments(): List<Comment> {
        val searchQuery = NativeSearchQueryBuilder()
            .withFilter(
                HasParentQueryBuilder(
                    "profile",
                    QueryBuilders
                        .boolQuery()
                        .must(
                            QueryBuilders
                                .geoDistanceQuery("location")
                                .distance(100, DistanceUnit.KILOMETERS)
                                .point(3.0, 5.0)
                        )
                        .must(
                            QueryBuilders
                                .rangeQuery("age")
                                .gte(26)
                                .lte(36)
                        ),
                    false
                )
            )
            .build()
        return elasticsearchOperations.search(searchQuery, Comment::class.java, IndexCoordinates.of("message_board")).toList().map(SearchHit<Comment>::getContent)
    }
}

我的设置：

我通过以下方式在 docker 中运行 elasticsearch：

docker run --name es -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -d -v es_data:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:7.4.2

Spring Boot：“2.3.2.RELEASE”

Spring Data Elasticsearch：“4.0.2.RELEASE”

问题：

我无法通过控制器的初始化块，但有以下异常：

Profile Mapping: MapDocument@?#? {"properties":{"age":{"store":true,"type":"short"},"location":{"type":"geo_point"}}}
Comment Mapping: MapDocument@?#? {"_parent":{"type":"profile"},"properties":{"parentId":{"store":true,"type":"text"}}}

Suppressed: org.elasticsearch.client.ResponseException: method [PUT], host [http://localhost:9200], URI [/message_board/_mapping?master_timeout=30s&timeout=30s], status line [HTTP/1.1 400 Bad Request]

Caused by: org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=mapper_parsing_exception, reason=Root mapping definition has unsupported parameters:  [_parent : {type=profile}]]

我需要一个不涉及向 ES 发出直接 POST 请求的解决方案。理想情况下，这可以通过 Elasticsearch 客户端 API 解决。感觉好像我在数据类上的注释缺少一些东西，但是我找不到任何关于此的文档。

Answer 1

这不是使用 REST API 的问题。这些调用由 Elasticsearch RestHighlevelClient 创建。

Elasticsearch since version 7.0.0 不再支持在一个索引中包含多种类型。因此，您不能以这种方式对数据进行建模。

Elasticsearch 为此支持join data type。我们目前正在开发一个 PR，它将为下一版本的 Spring Data Elasticsearch (4.1) 添加对此的支持。

Answer 2

我能够通过以下更改找到短期解决方案。

// Comment.kt
@Document(indexName = "message_board")
data class Comment(
    @Id
    val commentId: String,
    val relationField: Map<String, String>
)

// Profile.kt
@Document(indexName = "message_board")
data class Profile(
    @Id
    val profileId: String,
    @Field(type = FieldType.Short)
    val age: Short,
    @GeoPointField
    val location: GeoPoint,
    @Field(type = FieldType.Text)
    val relationField: String = "profile"
)

// Controller.kt
class Controller constructor(
    private val elasticsearchOperations: ElasticsearchOperations
) {
    init {
        elasticsearchOperations.indexOps(IndexCoordinates.of("message_board")).let { indexOp ->
            if (!indexOp.exists() && indexOp.create()) {
                val relationMap = Document.from(
                    mapOf(
                        "properties" to mapOf(
                            "relationField" to mapOf(
                                "type" to "join",
                                "relations" to mapOf(
                                    "profile" to "comment"
                                )
                            ),
                            "location" to mapOf(
                                "type" to "geo_point"
                            )
                        )
                    )
                )
                indexOp.putMapping(relationMap)
                indexOp.refresh()
            }
        }
    }
}

注意添加到两个数据类以及手动生成的映射文档中的relationField。现在，ES 在初始化上有一个适当的映射：

{
  "message_board" : {
    "mappings" : {
      "properties" : {
        "location" : {
          "type" : "geo_point"
        },
        "relationField" : {
          "type" : "join",
          "eager_global_ordinals" : true,
          "relations" : {
            "profile" : "comment"
          }
        }
      }
    }
  }
}

现在，创建profile 很简单：

val profile = Profile(
    profileId = UUID.randomUUID().toString(),
    age = 27,
    location = GeoPoint(3.0, 5.0)
)
val indexQuery = IndexQueryBuilder()
    .withId(profile.profileId)
    .withObject(profile)
    .build()
elasticsearchOperations.index(indexQuery, IndexCoordinates.of("message_board"))

但是，创建comment 有点麻烦，因为需要路由 ID：

val comment = Comment(
    commentId = UUID.randomUUID().toString(),
    relationField = mapOf(
        "name" to "comment",
        "parent" to profileId
    )
)
val bulkOptions = BulkOptions.builder()
    .withRoutingId(profileId)
    .build()
val indexQuery = IndexQueryBuilder()
    .withId(comment.commentId)
    .withObject(comment)
    .withParentId(profileId)
    .build()
elasticsearchOperations.bulkIndex(listOf(indexQuery), bulkOptions, IndexCoordinates.of("message_board"))

这就是我能够使用新的 JOIN 关系类型获得父子关系的方法。