Mongodb 3.4.24 复制集构建

 

​​​​​​

1. 准备

3个节点，我这里的IP及hostname分别是：

192.168.31.164   mongodb-master
192.168.31.165   mongodb-slave
192.168.31.167   mongodb-arbiter

版本：mongodb-linux-x86_64-rhel70-3.4.24.tgz

同时节点需要完成：

关闭防火墙、SSH互信

1. 节点规划

mongodb-master
mongodb-slave
mongodb-arbiter

 

1. 安装步骤

略

 [mongod@mongodb-master ~]$ cat /home/mongod/.bash_profile 
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs
export MONGO_HOME=/usr/local/mongodb/single/mongodb/
PATH=$PATH:$HOME/.local/bin:$HOME/bin:$MONGO_HOME/bin

export PATH

 

1. 创建MongoDB启动配置文件

配置文件： 

/usr/local/mongodb/single/mongodb/bin/mongo.conf

​

 

配置文件内容：

[mongod@mongodb-master bin]$ cat mongodb.conf 

#端口号
port=27017

#数据文件存储路径

dbpath=/usr/local/mongodb/single/mongodb/data

#日志文件存储路径

logpath=/usr/local/mongodb/single/mongodb/log/mongodb.log

#使用追加的方式写日志

logappend=true

#以守护进程的方式启动，即在后台运行

fork=true

#最大连接数

maxConns=5000

#是否启用身份验证

auth=false

#复制集名称

replSet=repl1

 

 

1. 同样的配置在其他两个节点完成。

可以通过scp命令，把配置好的mongodb文件夹拷贝到其他两个节点。

scp -r mongodb/ mongodb-slave:/usr/local/

scp -r mongodb/ mongodb-slave:/usr/local/

 

启动

在mongodb-master上运行命令： mongod --config /usr/local/mongodb/single/mongodb/bin/mongodb.conf

在mongodb-slave上运行命令： mongod --config /usr/local/mongodb/single/mongodb/bin/mongodb.conf

在mongodb-arbiter上运行命令： mongod --config /usr/local/mongodb/single/mongodb/bin/mongodb.conf、

或者在bin下编写启动脚本

[mongod@mongodb-master bin]$ cat start-mongodb.sh 
mongod --config /usr/local/mongodb/single/mongodb/bin/mongodb.conf

[mongod@mongodb-master bin]$ cat shut-mongodb.sh 
nohup /usr/local/mongodb/single/mongodb/bin/mongod --shutdown -f /usr/local/mongodb/single/mongodb/bin/mongodb.conf &

 

 

启动成功是可以看到如下日志：

a[mongod@mongodb-master data]$ mongod --config /usr/local/mongodb/single/mongodb/bin/mongodb.conf
about to fork child process, waiting until server is ready for connections.
forked process: 3864
child process started successfully, parent exiting
[mongod@mongodb-master data]$ ps -ef |grep mongo
root      3822  3602  0 13:02 pts/0    00:00:00 su - mongod
mongod    3823  3822  0 13:02 pts/0    00:00:00 -bash
mongod    3864     1  6 13:09 ?        00:00:00 mongod --config /usr/local/mongodb/single/mongodb/bin/mongodb.conf
mongod    3893  3823  0 13:09 pts/0    00:00:00 ps -ef
mongod    3894  3823  0 13:09 pts/0    00:00:00 grep --color=auto mongo

 

1. 初始化复制集

启动三个节点的mongodb服务后，开始把他们初始化为副本集。

在想设置primary的节点上运行客户端mongo

[mongod@mongodb-master ~]$ mongo
MongoDB shell version v3.4.24
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.24
Server has startup warnings: 
2020-02-28T13:09:21.322+0800 I CONTROL  [initandlisten] 
2020-02-28T13:09:21.322+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2020-02-28T13:09:21.322+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2020-02-28T13:09:21.322+0800 I CONTROL  [initandlisten] 
2020-02-28T13:09:21.322+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2020-02-28T13:09:21.322+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2020-02-28T13:09:21.322+0800 I CONTROL  [initandlisten] 

复制集初始化命令：

编写等级

config_repl1={_id:"repl1", members:[{_id:0, host:"mongodb-master:27017", priority:1}, {_id:1, host:"mongodb-slave:27017"}, {_id:2, host:"mongodb-arbiter:27017", arbiterOnly:true}]}
{
        "_id" : "repl1",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "mongodb-master:27017",
                        "priority" : 1
                },
                {
                        "_id" : 1,
                        "host" : "mongodb-slave:27017"
                },
                {
                        "_id" : 2,
                        "host" : "mongodb-arbiter:27017",
                        "arbiterOnly" : true
                }
        ]
}

> rs.initiate(config_repl1);

{ "ok" : 1 }

FAQ：2020-02-28T13:14:30.729+0800 E QUERY    [thread1] ReferenceError: config_repl1 is not defined :

如果输出以上，可能代表防火墙没关闭。

PRIMARY要等一会。

 

正常初始化复制集之后，可以看到

第一个节点变更为PRIMARY。

第二个节点变更为SECONDARY。

第三个节点变更为ARBITER。（仲裁节点）

​

​

​

 

1. 数据同步测试

首先向PRIMRY（主节点）写入一条数据，

repl1:PRIMARY> use qch
switched to db qch
repl1:PRIMARY> db.say.insert({"text":"Hello World"})
WriteResult({ "nInserted" : 1 })
repl1:PRIMARY> db.say.find()
{ "_id" : ObjectId("5e58a23938f373e67ab59351"), "text" : "Hello World" }
repl1:PRIMARY> db.say.find()
{ "_id" : ObjectId("5e58a23938f373e67ab59351"), "text" : "Hello World" }

 

进入SECONDARY（副节点）查看数据是否同步。

默认情况下SECONDARY节点不能读写，要设定slaveOK为true才可以从SECONDARY节点读取数据。

（replSet里只能有一个Primary节点，只能在Primary写数据，不能在SECONDARY写数据）

如果为设置slaveOK为true，会看到报错信息：

repl1:SECONDARY> rs.slaveOk()
repl1:SECONDARY> db.say.find()
repl1:SECONDARY> db.say.find()
repl1:SECONDARY> use qch
switched to db qch
repl1:SECONDARY> db.say.find()
{ "_id" : ObjectId("5e58a23938f373e67ab59351"), "text" : "Hello World" }
repl1:SECONDARY> 

1. 故障切换测试

副本集还有个重要的功能就是故障切换，

如果把主节点关闭，看看副节点是否能接替主节点进行工作。

 

1. 1. 关闭主节点

命令：

epl1:PRIMARY> use admin
switched to db admin
repl1:PRIMARY> db.shutdownServer()
server should be down...
2020-02-28T13:19:34.807+0800 I NETWORK  [thread1] trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed
2020-02-28T13:19:35.044+0800 I NETWORK  [thread1] Socket recv() Connection reset by peer 127.0.0.1:27017
2020-02-28T13:19:35.044+0800 I NETWORK  [thread1] SocketException: remote: (NONE):0 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27017] 
2020-02-28T13:19:35.044+0800 I NETWORK  [thread1] reconnect 127.0.0.1:27017 (127.0.0.1) failed failed 
> exit
bye

1. 1. 去副节点查看状态

repl1:SECONDARY> rs.status()
{
        "set" : "repl1",
        "date" : ISODate("2020-02-28T05:19:45.118Z"),
        "myState" : 1,
        "term" : NumberLong(2),
        "syncingTo" : "",
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1582867171, 1),
                        "t" : NumberLong(1)
                },
                "appliedOpTime" : {
                        "ts" : Timestamp(1582867171, 1),
                        "t" : NumberLong(1)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1582867171, 1),
                        "t" : NumberLong(1)
                }
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "mongodb-master:27017",
                        "health" : 0,
                        "state" : 8,
                        "stateStr" : "(not reachable/healthy)",
                        "uptime" : 0,
                        "optime" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2020-02-28T05:19:44.599Z"),
                        "lastHeartbeatRecv" : ISODate("2020-02-28T05:19:34.148Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "Connection refused",
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : -1
                },
                {
                        "_id" : 1,
                        "name" : "mongodb-slave:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 558,
                        "optime" : {
                                "ts" : Timestamp(1582867171, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-02-28T05:19:31Z"),
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "could not find member to sync from",
                        "electionTime" : Timestamp(1582867184, 1),
                        "electionDate" : ISODate("2020-02-28T05:19:44Z"),
                        "configVersion" : 1,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 2,
                        "name" : "mongodb-arbiter:27017",
                        "health" : 1,
                        "state" : 7,
                        "stateStr" : "ARBITER",
                        "uptime" : 273,
                        "lastHeartbeat" : ISODate("2020-02-28T05:19:44.598Z"),
                        "lastHeartbeatRecv" : ISODate("2020-02-28T05:19:41.156Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 1
                }
        ],
        "ok" : 1
}

1. 故障反向回切测试
   1. 开启主节点

[mongod@mongodb-master ~]$ mongod --config /usr/local/mongodb/single/mongodb/bin/mongodb.conf
about to fork child process, waiting until server is ready for connections.
forked process: 4058
child process started successfully, parent exiting

节点状态

repl1:SECONDARY> rs.status()
{
        "set" : "repl1",
        "date" : ISODate("2020-02-28T05:21:18.606Z"),
        "myState" : 2,
        "term" : NumberLong(2),
        "syncingTo" : "mongodb-slave:27017",
        "syncSourceHost" : "mongodb-slave:27017",
        "syncSourceId" : 1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1582867276, 1),
                        "t" : NumberLong(2)
                },
                "appliedOpTime" : {
                        "ts" : Timestamp(1582867276, 1),
                        "t" : NumberLong(2)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1582867276, 1),
                        "t" : NumberLong(2)
                }
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "mongodb-master:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 33,
                        "optime" : {
                                "ts" : Timestamp(1582867276, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2020-02-28T05:21:16Z"),
                        "syncingTo" : "mongodb-slave:27017",
                        "syncSourceHost" : "mongodb-slave:27017",
                        "syncSourceId" : 1,
                        "infoMessage" : "",
                        "configVersion" : 1,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "mongodb-slave:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 32,
                        "optime" : {
                                "ts" : Timestamp(1582867276, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1582867276, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2020-02-28T05:21:16Z"),
                        "optimeDurableDate" : ISODate("2020-02-28T05:21:16Z"),
                        "lastHeartbeat" : ISODate("2020-02-28T05:21:17.967Z"),
                        "lastHeartbeatRecv" : ISODate("2020-02-28T05:21:16.688Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1582867184, 1),
                        "electionDate" : ISODate("2020-02-28T05:19:44Z"),
                        "configVersion" : 1
                },
                {
                        "_id" : 2,
                        "name" : "mongodb-arbiter:27017",
                        "health" : 1,
                        "state" : 7,
                        "stateStr" : "ARBITER",
                        "uptime" : 32,
                        "lastHeartbeat" : ISODate("2020-02-28T05:21:17.967Z"),
                        "lastHeartbeatRecv" : ISODate("2020-02-28T05:21:16.203Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 1
                }
        ],
        "ok" : 1
}

 

1. 1. 备用节点插入数据

repl1:PRIMARY> use qch

switched to db qch

repl1:PRIMARY> db.say.insert({"text":"Hello World"})

WriteResult({ "nInserted" : 1 })

repl1:PRIMARY>

repl1:PRIMARY> db.say.find()

{ "_id" : ObjectId("5e58a23938f373e67ab59351"), "text" : "Hello World" }

{ "_id" : ObjectId("5e58a36cf66ed7cea9d2b50b"), "text" : "Hello World" }

1. 1. 主节点检查数据同步

repl1:SECONDARY> use qch

switched to db qch

repl1:SECONDARY> db.say.find()

{ "_id" : ObjectId("5e58a23938f373e67ab59351"), "text" : "Hello World" }

{ "_id" : ObjectId("5e58a36cf66ed7cea9d2b50b"), "text" : "Hello World" }

1. 1. 关闭备用节点

repl1:PRIMARY> use admin

switched to db admin

repl1:PRIMARY> db.shutdownServer()

server should be down...

2020-02-28T13:23:14.250+0800 I NETWORK  [thread1] trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed

2020-02-28T13:23:14.387+0800 I NETWORK  [thread1] Socket recv() Connection reset by peer 127.0.0.1:27017

2020-02-28T13:23:14.387+0800 I NETWORK  [thread1] SocketException: remote: (NONE):0 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27017]

2020-02-28T13:23:14.387+0800 I NETWORK  [thread1] reconnect 127.0.0.1:27017 (127.0.0.1) failed failed

2020-02-28T13:23:14.390+0800 I NETWORK  [thread1] trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed

2020-02-28T13:23:14.390+0800 W NETWORK  [thread1] Failed to connect to 127.0.0.1:27017, in(checking socket for error after poll), reason: Connection refused

2020-02-28T13:23:14.390+0800 I NETWORK  [thread1] reconnect 127.0.0.1:27017 (127.0.0.1) failed failed

> exit

Bye

1. 1. 检查主节点状态

repl1:SECONDARY> rs.status()

{

        "set" : "repl1",

        "date" : ISODate("2020-02-28T05:23:30.808Z"),

        "myState" : 1,

        "term" : NumberLong(3),

        "syncingTo" : "",

        "syncSourceHost" : "",

        "syncSourceId" : -1,

        "heartbeatIntervalMillis" : NumberLong(2000),

        "optimes" : {

                "lastCommittedOpTime" : {

                        "ts" : Timestamp(1582867386, 1),

                        "t" : NumberLong(2)

                },

                "appliedOpTime" : {

                        "ts" : Timestamp(1582867407, 1),

                        "t" : NumberLong(3)

                },

                "durableOpTime" : {

                        "ts" : Timestamp(1582867407, 1),

                        "t" : NumberLong(3)

                }

        },

        "members" : [

                {

                        "_id" : 0,

                        "name" : "mongodb-master:27017",

                        "health" : 1,

                        "state" : 1,

                        "stateStr" : "PRIMARY",

                        "uptime" : 165,

                        "optime" : {

                                "ts" : Timestamp(1582867407, 1),

                                "t" : NumberLong(3)

                        },

                        "optimeDate" : ISODate("2020-02-28T05:23:27Z"),

                        "syncingTo" : "",

                        "syncSourceHost" : "",

                        "syncSourceId" : -1,

                        "infoMessage" : "could not find member to sync from",

                        "electionTime" : Timestamp(1582867405, 1),

                        "electionDate" : ISODate("2020-02-28T05:23:25Z"),

                        "configVersion" : 1,

                        "self" : true,

                        "lastHeartbeatMessage" : ""

                },

                {

                        "_id" : 1,

                        "name" : "mongodb-slave:27017",

                        "health" : 0,

                        "state" : 8,

                        "stateStr" : "(not reachable/healthy)",

                        "uptime" : 0,

                        "optime" : {

                                "ts" : Timestamp(0, 0),

                                "t" : NumberLong(-1)

                        },

                        "optimeDurable" : {

                                "ts" : Timestamp(0, 0),

                                "t" : NumberLong(-1)

                        },

                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),

                        "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),

                        "lastHeartbeat" : ISODate("2020-02-28T05:23:29.162Z"),

                        "lastHeartbeatRecv" : ISODate("2020-02-28T05:23:14.739Z"),

                        "pingMs" : NumberLong(0),

                        "lastHeartbeatMessage" : "Connection refused",

                        "syncingTo" : "",

                        "syncSourceHost" : "",

                        "syncSourceId" : -1,

                        "infoMessage" : "",

                        "configVersion" : -1

                },

                {

                        "_id" : 2,

                        "name" : "mongodb-arbiter:27017",

                        "health" : 1,

                        "state" : 7,

                        "stateStr" : "ARBITER",

                        "uptime" : 164,

                        "lastHeartbeat" : ISODate("2020-02-28T05:23:29.158Z"),

                        "lastHeartbeatRecv" : ISODate("2020-02-28T05:23:26.226Z"),

                        "pingMs" : NumberLong(0),

                        "lastHeartbeatMessage" : "",

                        "syncingTo" : "",

                        "syncSourceHost" : "",

                        "syncSourceId" : -1,

                        "infoMessage" : "",

                        "configVersion" : 1

                }

        ],

        "ok" : 1