Mongo 辅助自动进入 Recovering 状态并卡住

Question

我正在使用 mongodb 2.6.5 并且有一个 3 节点副本集。很多时候我看到辅助节点进入 Recovering 状态，尽管我没有尝试做任何同步或任何事情。不确定 mongo 是否在后端执行此操作。状态永远不会出现在恢复中。我看到许多线程告诉我如何从恢复状态恢复到辅助状态，但我正在寻找它以找到关于它为什么要恢复状态的问题。是否有任何配置可能导致这种情况..

来自我的辅助服务器的几行日志（我觉得这可能很有用）用于恢复。 192.168.12.155:5000 是主节点，192.168.12.154:5000 是辅助节点。以下日志取自 192.168.12.154..

2015-03-08T20:02:20.963+0530 [conn223] end connection 192.168.31.152:43503 (4 connections now open)
2015-03-08T20:02:20.965+0530 [initandlisten] connection accepted from 192.168.31.152:43505 #225 (5 connections now open)
2015-03-08T20:02:21.065+0530 [conn224] end connection 192.168.31.152:43504 (4 connections now open)
2015-03-08T20:02:21.076+0530 [initandlisten] connection accepted from 192.168.31.152:43506 #226 (5 connections now open)
2015-03-08T20:02:21.207+0530 [conn225] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, optimes: [ { _id: ObjectId('54fc5b764db3d7d780142e5b'), optime: Timestamp 1425825044000|445, config: { _id: 3, host: "192.168.31.152:5000" } } ] } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 112ms
2015-03-08T20:02:21.209+0530 [conn225] end connection 192.168.31.152:43505 (4 connections now open)
2015-03-08T20:02:21.211+0530 [initandlisten] connection accepted from 192.168.31.152:43507 #227 (5 connections now open)
2015-03-08T20:02:21.345+0530 [conn227] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, handshake: { handshake: ObjectId('54fc5b764db3d7d780142e5b'), member: 3, config: { _id: 3, host: "192.168.31.152:5000" } } } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 133ms
2015-03-08T20:02:21.441+0530 [conn226] end connection 192.168.31.152:43506 (4 connections now open)
2015-03-08T20:02:21.453+0530 [initandlisten] connection accepted from 192.168.31.152:43508 #228 (5 connections now open)
2015-03-08T20:02:21.586+0530 [conn227] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, optimes: [ { _id: ObjectId('54fc5b764db3d7d780142e5b'), optime: Timestamp 1425825044000|448, config: { _id: 3, host: "192.168.31.152:5000" } } ] } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 113ms
2015-03-08T20:02:21.588+0530 [conn227] end connection 192.168.31.152:43507 (4 connections now open)
2015-03-08T20:02:21.590+0530 [initandlisten] connection accepted from 192.168.31.152:43509 #229 (5 connections now open)
2015-03-08T20:02:21.707+0530 [conn229] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, handshake: { handshake: ObjectId('54fc5b764db3d7d780142e5b'), member: 3, config: { _id: 3, host: "192.168.31.152:5000" } } } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 116ms
2015-03-08T20:02:21.808+0530 [conn228] end connection 192.168.31.152:43508 (4 connections now open)
2015-03-08T20:02:21.821+0530 [initandlisten] connection accepted from 192.168.31.152:43510 #230 (5 connections now open)
2015-03-08T20:02:21.833+0530 [conn229] end connection 192.168.31.152:43509 (4 connections now open)
2015-03-08T20:02:21.834+0530 [initandlisten] connection accepted from 192.168.31.152:43511 #231 (5 connections now open)
2015-03-08T20:02:22.069+0530 [conn62] end connection 192.168.12.155:42354 (4 connections now open)
2015-03-08T20:02:22.069+0530 [initandlisten] connection accepted from 192.168.12.155:42811 #232 (6 connections now open)
2015-03-08T20:02:22.069+0530 [conn231] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, handshake: { handshake: ObjectId('54fc5b764db3d7d780142e5b'), member: 3, config: { _id: 3, host: "192.168.31.152:5000" } } } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 234ms
2015-03-08T20:02:22.167+0530 [conn230] end connection 192.168.31.152:43510 (4 connections now open)
2015-03-08T20:02:22.177+0530 [initandlisten] connection accepted from 192.168.31.152:43512 #233 (5 connections now open)
2015-03-08T20:02:22.315+0530 [conn231] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, optimes: [ { _id: ObjectId('54fc5b764db3d7d780142e5b'), optime: Timestamp 1425825044000|454, config: { _id: 3, host: "192.168.31.152:5000" } } ] } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 116ms
2015-03-08T20:02:22.317+0530 [conn231] end connection 192.168.31.152:43511 (4 connections now open)
2015-03-08T20:02:22.319+0530 [initandlisten] connection accepted from 192.168.31.152:43513 #234 (5 connections now open)
2015-03-08T20:02:22.432+0530 [conn234] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, handshake: { handshake: ObjectId('54fc5b764db3d7d780142e5b'), member: 3, config: { _id: 3, host: "192.168.31.152:5000" } } } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 112ms
2015-03-08T20:02:22.529+0530 [conn233] end connection 192.168.31.152:43512 (4 connections now open)
2015-03-08T20:02:22.540+0530 [initandlisten] connection accepted from 192.168.31.152:43514 #235 (5 connections now open)
2015-03-08T20:02:22.681+0530 [conn234] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, optimes: [ { _id: ObjectId('54fc5b764db3d7d780142e5b'), optime: Timestamp 1425825044000|457, config: { _id: 3, host: "192.168.31.152:5000" } } ] } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 118ms
2015-03-08T20:02:22.682+0530 [conn234] end connection 192.168.31.152:43513 (4 connections now open)
2015-03-08T20:02:22.684+0530 [initandlisten] connection accepted from 192.168.31.152:43515 #236 (5 connections now open)
2015-03-08T20:02:22.794+0530 [conn236] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, handshake: { handshake: ObjectId('54fc5b764db3d7d780142e5b'), member: 3, config: { _id: 3, host: "192.168.31.152:5000" } } } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 108ms
2015-03-08T20:02:22.891+0530 [conn235] end connection 192.168.31.152:43514 (4 connections now open)
2015-03-08T20:02:22.902+0530 [initandlisten] connection accepted from 192.168.31.152:43516 #237 (5 connections now open)
2015-03-08T20:02:22.927+0530 [conn236] end connection 192.168.31.152:43515 (4 connections now open)
2015-03-08T20:02:22.929+0530 [initandlisten] connection accepted from 192.168.31.152:43517 #238 (5 connections now open)
2015-03-08T20:02:23.156+0530 [conn238] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, handshake: { handshake: ObjectId('54fc5b764db3d7d780142e5b'), member: 3, config: { _id: 3, host: "192.168.31.152:5000" } } } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 226ms
2015-03-08T20:02:23.252+0530 [conn237] end connection 192.168.31.152:43516 (4 connections now open)
2015-03-08T20:02:23.263+0530 [initandlisten] connection accepted from 192.168.31.152:43518 #239 (5 connections now open)
2015-03-08T20:02:23.414+0530 [conn238] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, optimes: [ { _id: ObjectId('54fc5b764db3d7d780142e5b'), optime: Timestamp 1425825044000|463, config: { _id: 3, host: "192.168.31.152:5000" } } ] } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 122ms
2015-03-08T20:02:23.415+0530 [conn238] end connection 192.168.31.152:43517 (4 connections now open)
2015-03-08T20:02:23.417+0530 [initandlisten] connection accepted from 192.168.31.152:43519 #240 (5 connections now open)
2015-03-08T20:02:23.619+0530 [conn239] end connection 192.168.31.152:43518 (4 connections now open)
2015-03-08T20:02:23.628+0530 [initandlisten] connection accepted from 192.168.31.152:43520 #241 (5 connections now open)
2015-03-08T20:02:23.781+0530 [conn240] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, optimes: [ { _id: ObjectId('54fc5b764db3d7d780142e5b'), optime: Timestamp 1425825044000|466, config: { _id: 3, host: "192.168.31.152:5000" } } ] } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 124ms
2015-03-08T20:02:23.782+0530 [conn240] end connection 192.168.31.152:43519 (4 connections now open)
2015-03-08T20:02:23.784+0530 [initandlisten] connection accepted from 192.168.31.152:43521 #242 (5 connections now open)
2015-03-08T20:02:23.979+0530 [conn241] end connection 192.168.31.152:43520 (4 connections now open)
2015-03-08T20:02:23.986+0530 [initandlisten] connection accepted from 192.168.31.152:43522 #243 (5 connections now open)
2015-03-08T20:02:24.148+0530 [conn242] command admin.$cmd command: replSetUpdatePosition { replSetUpdatePosition: 1, optimes: [ { _id: ObjectId('54fc5b764db3d7d780142e5b'), optime: Timestamp 1425825044000|469, config: { _id: 3, host: "192.168.31.152:5000" } } ] } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 127ms
2015-03-08T20:02:24.149+0530 [conn242] end connection 192.168.31.152:43521 (4 connections now open)
2015-03-08T20:02:24.152+0530 [initandlisten] connection accepted from 192.168.31.152:43523 #244 (5 connections now open)
2015-03-08T20:02:24.341+0530 [conn243] end connection 192.168.31.152:43522 (4 connections now open)
2015-03-08T20:02:24.350+0530 [initandlisten] connection accepted from 192.168.31.152:43524 #245 (5 connections now open)
2015-03-08T20:02:24.504+0530 [conn244] end connection 192.168.31.152:43523 (4 connections now open)
2015-03-08T20:02:24.506+0530 [initandlisten] connection accepted from 192.168.31.152:43525 #246 (5 connections now open)
2015-03-08T20:02:24.707+0530 [conn245] end connection 192.168.31.152:43524 (4 connections now open)
2015-03-08T20:02:24.714+0530 [initandlisten] connection accepted from 192.168.31.152:43526 #247 (5 connections now open)
2015-03-08T20:02:24.899+0530 [conn246] end connection 192.168.31.152:43525 (4 connections now open)
2015-03-08T20:02:24.901+0530 [initandlisten] connection accepted from 192.168.31.152:43527 #248 (5 connections now open)
2015-03-08T20:02:24.969+0530 [rsBackgroundSync] repl: old cursor isDead, will initiate a new one
2015-03-08T20:02:25.051+0530 [rsBackgroundSync] replSet syncing to: 192.168.12.155:5000
2015-03-08T20:02:25.081+0530 [rsBackgroundSync] replSet not trying to sync from 192.168.12.155:5000, it is vetoed for 600 more seconds
2015-03-08T20:02:25.081+0530 [rsBackgroundSync] replSet not trying to sync from 192.168.12.155:5000, it is vetoed for 600 more seconds
2015-03-08T20:02:25.081+0530 [rsBackgroundSync] replSet error RS102 too stale to catch up, at least from 192.168.12.155:5000
2015-03-08T20:02:25.081+0530 [rsBackgroundSync] replSet our last optime : Mar  8 20:00:53 54fc5d1d:3c
2015-03-08T20:02:25.081+0530 [rsBackgroundSync] replSet oldest at 192.168.12.155:5000 : Mar  8 20:00:54 54fc5d1e:5d
2015-03-08T20:02:25.081+0530 [rsBackgroundSync] replSet See http://dochub.mongodb.org/core/resyncingaverystalereplicasetmember
2015-03-08T20:02:25.081+0530 [rsBackgroundSync] replSet error RS102 too stale to catch up
2015-03-08T20:02:25.081+0530 [rsBackgroundSync] replSet RECOVERING
2015-03-08T20:02:25.137+0530 [conn247] end connection 192.168.31.152:43526 (4 connections now open)
2015-03-08T20:02:25.146+0530 [initandlisten] connection accepted from 192.168.31.152:43528 #249 (5 connections now open)
2015-03-08T20:02:25.147+0530 [conn248] end connection 192.168.31.152:43527 (4 connections now open)
2015-03-08T20:02:25.148+0530 [initandlisten] connection accepted from 192.168.31.152:43529 #250 (5 connections now open)
2015-03-08T20:02:25.245+0530 [conn249] end connection 192.168.31.152:43528 (4 connections now open)
2015-03-08T20:02:25.253+0530 [initandlisten] connection accepted from 192.168.31.152:43530 #251 (5 connections now open)
2015-03-08T20:02:25.254+0530 [conn250] end connection 192.168.31.152:43529 (4 connections now open)
2015-03-08T20:02:25.254+0530 [initandlisten] connection accepted from 192.168.31.152:43531 #252 (5 connections now open)
2015-03-08T20:02:25.349+0530 [conn251] end connection 192.168.31.152:43530 (4 connections now open)
2015-03-08T20:02:25.356+0530 [initandlisten] connection accepted from 192.168.31.152:43532 #253 (5 connections now open)
2015-03-08T20:02:25.357+0530 [conn252] end connection 192.168.31.152:43531 (4 connections now open)
2015-03-08T20:02:25.357+0530 [initandlisten] connection accepted from 192.168.31.152:43533 #254 (5 connections now open)

Answer 1

Mongodb 文档对此有一个部分 https://docs.mongodb.com/manual/tutorial/resync-replica-set-member/#replica-set-auto-resync-stale-member