【问题标题】:Oplog does not record operations within a transaction - MongoDbOplog 不记录事务中的操作 - MongoDb
【发布时间】:2021-09-28 04:43:21
【问题描述】:

如果您运行以下代码,outside a transaction 对象将作为插入操作出现在 oplog 中,但 oplog 将没有 inside a transaction 的记录,但它们都会正确保存到集合中。

我尝试查找问题没有运气,虽然我能够得到a confirmation that in my mongodb server version, oplog will create a separate entry for each operation,虽然我没有得到4.0 样式,也没有得到4.2

代码

    const { MongoClient } = require('mongodb');

    const dbName = "db-name";
    const collectionName = "collection-name";
    const dbUri = "mongodb+srv://<user>:<pass>@<cluster-url>/<db-name>?retryWrites=true&w=majority";
 
    const client = await new MongoClient(dbUri, { useUnifiedTopology: true }).connect();
    const db = client.db(dbName);
    const collection = db.collection(collectionName);

    const session = client.startSession();
    const transactionOptions = {
        readConcern: { level: 'snapshot' },
        writeConcern: { w: 'majority' }
    };

    await collection.insertOne({ name: "outside a transaction" });
    await session.withTransaction(async () => {
        await collection.insertOne({ name: "inside a transaction" }, { session: session});
    }, transactionOptions);

    await session.endSession();
    await client.close();

Oplog 结果

查询:db.oplog.rs.find({}).sort({$natural: -1}) 实际数量:1 预期数量:2

{
   "lsid":{
      "id":{
         "$binary":"IwdZfNnGTvGDoABb0gjxXQ==",
         "$type":"4"
      },
      "uid":{
         "$binary":"ddqot68vLpBUPnOuqFj2cdClv/dv/vF9ZVSwknYHEsE=",
         "$type":"0"
      }
   },
   "txnNumber":{
      "$numberLong":"1"
   },
   "op":"i",
   "ns":"testDb.tests",
   "ui":{
      "$binary":"2a1ZJWDySfGuvtgMK2G2gQ==",
      "$type":"4"
   },
   "o":{
      "_id":{
         "$oid":"60f6feb12a662ec3fc985400"
      },
      "name":"outside a transaction"
   },
   "ts":{
      "$timestamp":{
         
      }
   },
   "t":{
      "$numberLong":"73"
   },
   "wall":{
      "$date":"2021-07-20T16:49:53.226Z"
   },
   "v":{
      "$numberLong":"2"
   },
   "stmtId":0,
   "prevOpTime":{
      "ts":{
         "$timestamp":{
            
         }
      },
      "t":{
         "$numberLong":"-1"
      }
   }
}

【问题讨论】:

  • 据我了解,您是说此代码await collection.insertOne({ name: "outside a transaction" }); 未按事务处理。我不熟悉node,但我相信节点行为与其他驱动程序相同。因此,您看到的行为是预期的,因为仅当您将相关会话对象传递给此操作时,该操作才被视为事务下的操作
  • @dododo 正确,await collection.insertOne({ name: "outside a transaction" }); 预计将在事务/会话之外执行,问题或我的问题是,为什么 oplog 捕获 outside a transaction 而不是 inside a transaction
  • 因为在您将会话传递给insertOne之前它不在事务内部
  • 从底部开始的第 4-6 行,确实有一个通过会话的 insertOne,await collection.insertOne({ name: "inside a transaction" }, { session: session}); 基于上面的代码,oplog 中应该有 2 个条目,但我只得到一个条目(未链接到会话的那个)。另外我认为您误解了我的问题,或者我的问题可能不够清楚,但问题不在于交易或它们如何工作,问题是为什么第二个insertOne 存在于交易中没有出现在oplog
  • 是的,我的错,稍后可以再看一遍

标签: node.js mongodb transactions replicaset mongodb-oplog


【解决方案1】:

当我在 MongoDB 5.0(3 节点副本集)上运行此测试时,我看到两个 oplog 条目...

use local

db.oplog.rs.find({ns: {$ne: ''}}).sort({ts: -1})

 {
    lsid: {
      id: UUID("74450c89-04ca-441c-93b4-24472c833e98"),
      uid: Binary(Buffer.from("92e7f3b655ee46a4a10212cc1146f3a783e6f4266bef6b5699dbc5528be5177c", "hex"), 0)
    },
    txnNumber: Long("1"),
    op: 'c',
    ns: 'admin.$cmd',
    o: {
      applyOps: [
        {
          op: 'i',
          ns: 'nodetest.mycollection',
          ui: UUID("b6079273-b2de-49a4-b148-837b4ac2c95f"),
          o: {
            _id: ObjectId("60f74ca3601963e6c5ca02f2"),
            name: 'inside a transaction'
          }
        }
      ]
    },
    ts: Timestamp(3, 1626819747),
    t: Long("95"),
    v: Long("2"),
    wall: ISODate("2021-07-20T22:22:27.714Z"),
    prevOpTime: { ts: Timestamp(0, 0), t: Long("-1") }
  },
  {
    lsid: {
      id: UUID("92372b8e-89fd-48ff-ab2a-780520e97c39"),
      uid: Binary(Buffer.from("92e7f3b655ee46a4a10212cc1146f3a783e6f4266bef6b5699dbc5528be5177c", "hex"), 0)
    },
    txnNumber: Long("1"),
    op: 'i',
    ns: 'nodetest.mycollection',
    ui: UUID("b6079273-b2de-49a4-b148-837b4ac2c95f"),
    o: {
      _id: ObjectId("60f74ca3601963e6c5ca02f1"),
      name: 'outside a transaction'
    },
    ts: Timestamp(2, 1626819747),
    t: Long("95"),
    v: Long("2"),
    wall: ISODate("2021-07-20T22:22:27.367Z"),
    stmtId: 0,
    prevOpTime: { ts: Timestamp(0, 0), t: Long("-1") }
  }

MongoDB 4.4 中也是一样的......

{
    "lsid" : {
        "id" : UUID("8d72b35a-a8d7-4ffd-a56a-1f2f02b944a5"),
        "uid" : BinData(0,"kufztlXuRqShAhLMEUbzp4Pm9CZr72tWmdvFUovlF3w=")
    },
    "txnNumber" : NumberLong(1),
    "op" : "c",
    "ns" : "admin.$cmd",
    "o" : {
        "applyOps" : [
            {
                "op" : "i",
                "ns" : "nodetest.mycollection",
                "ui" : UUID("52fcd28a-9de8-45b0-966a-d1b869976d11"),
                "o" : {
                    "_id" : ObjectId("60f74ead58cd50e7b5ec7e1c"),
                    "name" : "inside a transaction"
                }
            }
        ]
    },
    "ts" : Timestamp(1626820269, 1),
    "t" : NumberLong(185),
    "wall" : ISODate("2021-07-20T22:31:09.111Z"),
    "v" : NumberLong(2),
    "prevOpTime" : {
        "ts" : Timestamp(0, 0),
        "t" : NumberLong(-1)
    }
}
{
    "lsid" : {
        "id" : UUID("3985c2b0-1863-4718-8851-747ff8160045"),
        "uid" : BinData(0,"kufztlXuRqShAhLMEUbzp4Pm9CZr72tWmdvFUovlF3w=")
    },
    "txnNumber" : NumberLong(1),
    "op" : "i",
    "ns" : "nodetest.mycollection",
    "ui" : UUID("52fcd28a-9de8-45b0-966a-d1b869976d11"),
    "o" : {
        "_id" : ObjectId("60f74eac58cd50e7b5ec7e1b"),
        "name" : "outside a transaction"
    },
    "ts" : Timestamp(1626820268, 2),
    "t" : NumberLong(185),
    "wall" : ISODate("2021-07-20T22:31:08.912Z"),
    "v" : NumberLong(2),
    "stmtId" : 0,
    "prevOpTime" : {
        "ts" : Timestamp(0, 0),
        "t" : NumberLong(-1)
    }
}

MongoDB 4.2

{
    "ts" : Timestamp(1626820723, 2),
    "t" : NumberLong(501),
    "h" : NumberLong(0),
    "v" : 2,
    "op" : "c",
    "ns" : "admin.$cmd",
    "wall" : ISODate("2021-07-20T22:38:43.304Z"),
    "lsid" : {
        "id" : UUID("0df8170a-6700-4751-9c14-88651ec54f9b"),
        "uid" : BinData(0,"kufztlXuRqShAhLMEUbzp4Pm9CZr72tWmdvFUovlF3w=")
    },
    "txnNumber" : NumberLong(1),
    "prevOpTime" : {
        "ts" : Timestamp(0, 0),
        "t" : NumberLong(-1)
    },
    "o" : {
        "applyOps" : [
            {
                "op" : "i",
                "ns" : "nodetest.mycollection",
                "ui" : UUID("4c0507b2-8018-48d4-9f80-f2bb6bd227dd"),
                "o" : {
                    "_id" : ObjectId("60f75073abc759e7f6d53ba6"),
                    "name" : "inside a transaction"
                }
            }
        ]
    }
}
{
    "ts" : Timestamp(1626820723, 1),
    "t" : NumberLong(501),
    "h" : NumberLong(0),
    "v" : 2,
    "op" : "i",
    "ns" : "nodetest.mycollection",
    "ui" : UUID("4c0507b2-8018-48d4-9f80-f2bb6bd227dd"),
    "wall" : ISODate("2021-07-20T22:38:43.016Z"),
    "lsid" : {
        "id" : UUID("4983da87-0c9c-448b-8d54-50f3885ce9ce"),
        "uid" : BinData(0,"kufztlXuRqShAhLMEUbzp4Pm9CZr72tWmdvFUovlF3w=")
    },
    "txnNumber" : NumberLong(1),
    "stmtId" : 0,
    "prevOpTime" : {
        "ts" : Timestamp(0, 0),
        "t" : NumberLong(-1)
    },
    "o" : {
        "_id" : ObjectId("60f75072abc759e7f6d53ba5"),
        "name" : "outside a transaction"
    }
}

MongoDB 4.0

{
    "ts" : Timestamp(1626821049, 3),
    "t" : NumberLong(2),
    "h" : NumberLong("-7247099881451421420"),
    "v" : 2,
    "op" : "c",
    "ns" : "admin.$cmd",
    "wall" : ISODate("2021-07-20T22:44:09.560Z"),
    "lsid" : {
        "id" : UUID("984168e5-c1fd-47cd-9981-acae8585e3e9"),
        "uid" : BinData(0,"kufztlXuRqShAhLMEUbzp4Pm9CZr72tWmdvFUovlF3w=")
    },
    "txnNumber" : NumberLong(1),
    "stmtId" : 0,
    "prevOpTime" : {
        "ts" : Timestamp(0, 0),
        "t" : NumberLong(-1)
    },
    "o" : {
        "applyOps" : [
            {
                "op" : "i",
                "ns" : "nodetest.mycollection",
                "ui" : UUID("60acd457-0d98-4e06-8921-bf10d1bb63c7"),
                "o" : {
                    "_id" : ObjectId("60f751b926f38ce856f9b08d"),
                    "name" : "inside a transaction"
                }
            }
        ]
    }
}
{
    "ts" : Timestamp(1626821049, 2),
    "t" : NumberLong(2),
    "h" : NumberLong("6159503664626025226"),
    "v" : 2,
    "op" : "i",
    "ns" : "nodetest.mycollection",
    "ui" : UUID("60acd457-0d98-4e06-8921-bf10d1bb63c7"),
    "wall" : ISODate("2021-07-20T22:44:09.266Z"),
    "lsid" : {
        "id" : UUID("8f7d6590-8c7c-4819-9ce7-f69b92725c26"),
        "uid" : BinData(0,"kufztlXuRqShAhLMEUbzp4Pm9CZr72tWmdvFUovlF3w=")
    },
    "txnNumber" : NumberLong(1),
    "stmtId" : 0,
    "prevOpTime" : {
        "ts" : Timestamp(0, 0),
        "t" : NumberLong(-1)
    },
    "o" : {
        "_id" : ObjectId("60f751b926f38ce856f9b08c"),
        "name" : "outside a transaction"
    }
}

不使用 3.6 进行测试,因为它不再是受支持的版本,并且无论如何也不支持事务。

我必须修改您的示例程序以使其正常工作,这是我运行的测试...

修改后的测试程序

const { MongoClient } = require('mongodb');

const dbName = "nodetest";
const collectionName = "mycollection";
const dbUri = "mongodb://barry:????????@localhost:44011,localhost:44012,localhost:44013/nodetest?replicaSet=replSet&authSource=admin&retryWrites=true&w=majority";

async function run() {
    const client = await new MongoClient(dbUri, { useUnifiedTopology: true }).connect();
    const db = client.db(dbName);
    const collection = db.collection(collectionName);
    
    const session = client.startSession();
    const transactionOptions = {
        readConcern: { level: 'snapshot' },
        writeConcern: { w: 'majority' }
    };
    
    await collection.insertOne({ name: "outside a transaction" });
    await session.withTransaction(async () => {
        await collection.insertOne({ name: "inside a transaction" }, { session: session});
    }, transactionOptions);
    
    await session.endSession();
    await client.close();
}

run().catch(console.dir);

【讨论】:

  • 太棒了!感谢您花时间运行这些测试,我终于与 mongodb atlas 支持人员进行了交谈,他们表示我正在测试的实例类型存在限制,一旦我们升级 oplog 按预期工作,我也认为我误解了文档正如我认为从 mongo 4.2 开始,事务操作将由它自己的文档/条目表示,但事实并非如此 - docs.atlas.mongodb.com/reference/unsupported-commands/…(M0、M2 和 M5 不支持 applyOps)
  • 我会接受这个作为答案,如果您想更新答案以使用 Mongo Atlas 捕获特定用例,我认为这将使未来读者的答案更具决定性,我也是认为更准确的查询是db.oplog.rs.find({ns: "admin.$cmd", op: "c"}).sort({ts: -1})
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2016-02-19
  • 1970-01-01
  • 1970-01-01
  • 2010-11-28
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多