通过 numactl 作为守护进程启动 mongodb

Question

我正在尝试在 NUMA 机器上启动 mongodb 作为守护进程。当我跑步时

numactl --interleave=all mongod &

Mongo 启动并正常运行，但所有输出仍然显示。 （例如，Fri Jun 22 12:10:29 [initandlisten] 接受来自 127.0.1.1:51837 的连接）

但是，当我自己启动 mongo（如下所示）时，它会失败（如下所示）：

service mongodb start

我在日志中得到以下信息

Fri Jun 22 12:08:41 [initandlisten] MongoDB starting : pid=3348 port=27017 dbpath=/var/lib/mongodb 64-bit host=beckett
Fri Jun 22 12:08:41 [initandlisten]
Fri Jun 22 12:08:41 [initandlisten] ** WARNING: You are running on a NUMA machine.
Fri Jun 22 12:08:41 [initandlisten] **          We suggest launching mongod like this to avoid performance problems:
Fri Jun 22 12:08:41 [initandlisten] **              numactl --interleave=all mongod [other options]
Fri Jun 22 12:08:41 [initandlisten]
Fri Jun 22 12:08:41 [initandlisten] db version v2.0.6, pdfile version 4.5
Fri Jun 22 12:08:41 [initandlisten] git version: e1c0cbc25863f6356aa4e31375add7bb49fb05bc
Fri Jun 22 12:08:41 [initandlisten] build info: Linux ip-10-110-9-236 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Fri Jun 22 12:08:41 [initandlisten] options: { auth: "true", command: [ "run" ], config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", logappend: "true", logpath: "/var/log/mongodb/mongodb.log" }
Fri Jun 22 12:08:41 [initandlisten] journal dir=/var/lib/mongodb/journal
Fri Jun 22 12:08:41 [initandlisten] recover : no journal files present, no recovery needed
Fri Jun 22 12:08:42 [initandlisten] couldn't open /var/lib/mongodb/admin.ns errno:13 Permission denied
Fri Jun 22 12:08:42 [initandlisten] error couldn't open file /var/lib/mongodb/admin.ns terminating
Fri Jun 22 12:08:42 dbexit:
Fri Jun 22 12:08:42 [initandlisten] shutdown: going to close listening sockets...
Fri Jun 22 12:08:42 [initandlisten] shutdown: going to flush diaglog...
Fri Jun 22 12:08:42 [initandlisten] shutdown: going to close sockets...
Fri Jun 22 12:08:42 [initandlisten] shutdown: waiting for fs preallocator...
Fri Jun 22 12:08:42 [initandlisten] shutdown: lock for final commit...
Fri Jun 22 12:08:42 [initandlisten] shutdown: final commit...
Fri Jun 22 12:08:42 [initandlisten] shutdown: closing all files...
Fri Jun 22 12:08:42 [initandlisten] closeAllFiles() finished
Fri Jun 22 12:08:42 [initandlisten] journalCleanup...
Fri Jun 22 12:08:42 [initandlisten] removeJournalFiles
Fri Jun 22 12:08:42 [initandlisten] shutdown: removing fs lock...
Fri Jun 22 12:08:42 dbexit: really exiting now

我不知道当我以 root 身份运行时 admin.ns 会如何出现权限问题，或者为什么用 numactl 包装时它启动正常。理想情况下，我想在 start_server() 函数中使用 numactl，如下所示：

start_server(){
    /usr/bin/numactl --interleave=all -- \
    start-stop-daemon --background --start --quiet --pidfile $PIDFILE \
        --make-pidfile --chuid $DAEMONUSER \
        --exec $DAEMON -- $DAEMON_OPTS
    errcode=$?
    return $errcode
}

底线，如何让 mongo 作为 NUMA 机器上的守护进程启动？

Answer 1

适用于 ubuntu 16.04

[Unit]
Description=High-performance, schema-free document-oriented database
After=time-sync.target network.target

[Service]
Type=forking
User=mongod
Group=mongod
LimitNOFILE=65000
PermissionsStartOnly=true
EnvironmentFile=/etc/default/mongod
ExecStartPre=/usr/bin/percona-server-mongodb-helper.sh
ExecStart=/usr/bin/env bash -c "numactl --interleave=all /usr/bin/mongod $OPTIONS > ${STDOUT} 2> ${STDERR}"
#ExecStart=/usr/bin/env bash -c "/usr/bin/mongod $OPTIONS > ${STDOUT} 2> ${STDERR}"
PIDFile=/var/run/mongod.pid

[Install]
WantedBy=multi-user.target

Answer 2

适用于 ubuntu 14

# apt-get -y install numactl

添加到 /etc/sysctl.conf 行 vm.zone_reclaim_mode = 0

# sysctl -p

# service mongod restart

Answer 3

上面的两个解决方案对我没有用，所以下面是让我的 mongodb 运行的原因：

将/etc/init.d/mongodb中的start_server()函数替换为下面的代码

start_server() {
    test -e "$RUNDIR" || install -m 755 -o mongodb -g mongodb -d "$RUNDIR"

    NUMACTL=$(which numactl)

    if [ ! "$NUMACTL" ]; then
        # start original
        start-stop-daemon --background --start --quiet --pidfile $PIDFILE --make-pidfile --chuid $DAEMONUSER --exec $DAEMON -- $DAEMON_OPTS
        errcode=$?
        return $errcode
    else
        # Start the process using the wrapper
        $NUMACTL --interleave=all -- start-stop-daemon --background --start --quiet --pidfile $PIDFILE --make-pidfile --chuid $DAEMONUSER --exec $DAEMON -- $DAEMON_OPTS
        errcode=$?
        return $errcode
    fi
}

Answer 4

原来我的问题是 numa 和权限问题的结合。谢谢，@Mark 的帮助。要将 mongodb 作为 NUMA 设置上的守护进程启动，请将 /etc/init.d/mongodb 中的 start_server() 函数替换为以下内容：

start_server() {
# check for numactl
NUMACTL=$(which numactl)
if [ -n "$NUMACTL" ]; then
    DAEMON_OPTS="--interleave=all ${DAEMON} ${DAEMON_OPTS}"
    DAEMON="$NUMACTL"
fi

# Start the process using the wrapper
            /usr/bin/numactl --interleave=all -- \
            start-stop-daemon --background --start --quiet --pidfile $PIDFILE \
                        --make-pidfile --chuid $DAEMONUSER \
                        --exec $DAEMON -- $DAEMON_OPTS
            errcode=$?
        return $errcode
}

Answer 5

我想你知道关于 'mongo & numa' 的常见警告 (http://www.mongodb.org/display/DOCS/NUMA)，所以我不会继续讨论它们。

这是一个带有 numa 的 mongodb 的示例 upstart 配置文件 - https://gist.github.com/1364716。

基于这个Google Group thread，在init脚本的start_server函数中添加了以下几行，成功了-

start_server() { 
# check for numactl 
NUMACTL=$(which numactl) 
if [ -n "$NUMACTL" ]; then 
    DAEMON_OPTS="--interleave=all ${DAEMON} ${DAEMON_OPTS}" 
    DAEMON="$NUMACTL" 
fi 

# Start the process using the wrapper