mongodb迁移导致key重复

Question

在一个旧项目中，我们正在尝试从基本 spring 迁移到 spring boot 2.3.1。为此，因为我们有一个 mongo 数据库，我们必须从编写此代码的 spring-data-mongodb:1.10.18 迁移：

DBCollection contextCollection = this.mongoTemplate.getCollection("productStock");
BulkWriteOperation builder = contextCollection.initializeUnorderedBulkOperation();
StockType stockItem = stockMessage.getStockItem();

final BasicDBObject id = new BasicDBObject("storeID", stockItem.getStoreID()).append("productID", stockItem.getProductID());
BulkWriteRequestBuilder bulkWriteRequestBuilder = builder.find(new BasicDBObject("_id", id));
HashMap<String, Object> stock = new HashMap<>();
Date currentDate = Calendar.getInstance().getTime();

stock.put("value", stockItem.getValue());
if (stockItem.getAssociateDate() != null) {
    stock.put("associateDate", stockItem.getAssociateDate());
}

if (stockItem.getLastAccessDateSource() != null) {
    stock.put("lastAccessDateSource", stockItem.getLastAccessDateSource());

    // check
    BasicDBObject ltLast = new BasicDBObject("$lt", stockItem.getLastAccessDateSource());
    BasicDBList dbList = new BasicDBList();
    dbList.add(new BasicDBObject(stockItem.getStockCategory() + ".lastAccessDateSource", ltLast));
    dbList.add(new BasicDBObject(stockItem.getStockCategory() + ".lastAccessDateSource", null));
    bulkWriteRequestBuilder = builder.find(new BasicDBObject("_id", id).append("$or", dbList));
} else {
    stock.put("lastAccessDateSource", currentDate);
}

stock.put("lastUpdateDate", currentDate);
BasicDBObject set = new BasicDBObject(stockItem.getStockCategory(), new Document(stock));
bulkWriteRequestBuilder.upsert().updateOne(new BasicDBObject("$set", set));
builder.execute();

到 spring-data-mongodb:3.0.1.RELEASE 用这个更新的代码

Map<String, List<StockType>> mapMultiUpdate = new HashMap<>();
StockType stockItem = stockMessage.getStockItem();

final Document id = new Document("storeID", stockItem.getStoreID()).append("productID", stockItem.getProductID());
HashMap<String, Object> stock = new HashMap<>();
Date currentDate = Calendar.getInstance().getTime();
Document searchQuery = new Document("_id", id).append("$or", dbList);
stock.put("value", stockItem.getValue());
if (stockItem.getAssociateDate() != null) {
    stock.put("associateDate", stockItem.getAssociateDate());
}

if (stockItem.getLastAccessDateSource() != null) {
    stock.put("lastAccessDateSource", stockItem.getLastAccessDateSource());

    // check
    Document ltLast = new Document("$lt", stockItem.getLastAccessDateSource());
    List<Document> dbList = Lists.newArrayList();
    dbList.add(new Document(stockItem.getStockCategory() + ".lastAccessDateSource", ltLast));
    dbList.add(new Document(stockItem.getStockCategory() + ".lastAccessDateSource", null));
} else {
    stock.put("lastAccessDateSource", currentDate);
}


//Bulk write options
BulkWriteOptions bulkWriteOptions = new BulkWriteOptions();
bulkWriteOptions.ordered(false);
bulkWriteOptions.bypassDocumentValidation(true);

MongoCollection<Document> mongoCollection = this.mongoTemplate.getCollection("productStoreStock");
mongoCollection.bulkWrite(updateDocuments, bulkWriteOptions);

但是当新代码在一个已经存在的对象上执行时，我们会得到一个重复键错误

com.mongodb.MongoBulkWriteException: Bulk write operation error on server localhost:27017. Write errors: [BulkWriteError{index=0, code=11000, message='E11000 duplicate key error collection: test.productStoreStock index: _id_ dup key: { : { storeID: 400, productID: 100000 } }', details={}}]. 

我们也从 mongo-java-driver:3.6.4 切换到 mongodb-driver-sync:4.0.4

编辑：

在测试阶段的第三步，在空数据库/集合上抛出此错误。步骤：

• 在特定日期用一种产品的库存开始收集
• 检查基数的值
• 在java中修改股票的价值而不是日期并尝试更新它
• 检查值仍然是第一个，因为 mongo 查询上的 LT 过滤器

我们从来没有达到检查值，在迁移之前，这次测试一切都很好

Answer 1

Mongo 没有更新它是 upserting（插入一个新行）当没有找到匹配项时。

Answer 2

test.productStoreStock index: _id_ dup key: { : { storeID: 400, productID: 100000 } }', details={}}

看起来您正在用您自己的“构造”ID 替换内部 id 字段。这不是一件好事。让 mongo 创建自己的 _id，这些在使用过程中保证是唯一的.. 永远不会重复。 将自己的 companyId 作为附加字段并没有错，但是替换 mongodbs 自动生成的 _id 字段很危险......正如您所发现的那样。