【问题标题】:Change max-job size of beanstalkd更改 beanstalkd 的最大作业大小
【发布时间】:2016-03-03 20:38:05
【问题描述】:

有人如何更改 beanstalkd 的最大作业大小? 我收到消息 JOB_TOO_BIGAdding Job to beanstalkd 的问题是,它说默认大小是 65k。 有人知道如何改变吗?

编辑: 我在文件夹 /etc/init.d 中的 beanstalkd 初始化脚本如下所示(我添加了 -z 选项以增加作业大小):

#!/bin/sh
#
# Copyright (c) 2007 Javier Fernandez-Sanguino <jfs@debian.org>
#
# This is free software; you may redistribute it and/or modify
# it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2,
# or (at your option) any later version.
#
# This is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License with
# the Debian operating system, in /usr/share/common-licenses/GPL;  if
# not, write to the Free Software Foundation, Inc., 59 Temple Place,
# Suite 330, Boston, MA 02111-1307 USA
#
### BEGIN INIT INFO
# Provides:          beanstalkd
# Required-Start:    $remote_fs $network $local_fs
# Required-Stop:     $remote_fs $network $local_fs
# Should-Start:      $named
# Should-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: in-memory workqueue service
# Description:       beanstalk is a simple, fast, queueing server. Its
#                    interface is generic, but was originally designed
#                    for reducing the latency of page views in high-volume
#                    web applications by running time-consuming tasks
#                    asynchronously.
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

DAEMON=/usr/bin/beanstalkd # Introduce the server's location here
NAME=beanstalkd             # Introduce the short server's name here
DESC="in-memory queueing server" # Introduce a short description here
LOGDIR=/var/log/beanstalkd  # Log directory to use
BEANSTALKD_LISTEN_ADDR=0.0.0.0
BEANSTALKD_LISTEN_PORT=11300

PIDFILE=/var/run/$NAME.pid

test -x $DAEMON || exit 0

. /lib/lsb/init-functions

# Default options, these can be overriden by the information
# at /etc/default/$NAME
DAEMON_OPTS="-l $BEANSTALKD_LISTEN_ADDR -p $BEANSTALKD_LISTEN_PORT -z 550000000000"          # Additional options given to the server

#DIETIME=10              # Time to wait for the server to die, in seconds
                        # If this value is set too low you might not
                        # let some servers to die gracefully and
                        # 'restart' will not work

STARTTIME=1             # Time to wait for the server to start, in seconds
                        # If this value is set each time the server is
                        # started (on start or restart) the script will
                        # stall to try to determine if it is running
                        # If it is not set and the server takes time
                        # to setup a pid file the log message might
                        # be a false positive (says it did not start
                        # when it actually did)

#LOGFILE=$LOGDIR/$NAME.log  # Server logfile
DAEMONUSER=root #beanstalkd   # Users to run the daemons as. If this value
                        # is set start-stop-daemon will chuid the server

# Include defaults if available
if [ -f /etc/default/$NAME ] ; then
    . /etc/default/$NAME
fi

# Check that the user exists (if we set a user)
# Does the user exist?
if [ -n "$DAEMONUSER" ] ; then
    if getent passwd | grep -q "^$DAEMONUSER:"; then
        # Obtain the uid and gid
        DAEMONUID=`getent passwd |grep "^$DAEMONUSER:" | awk -F : '{print $3}'`
        DAEMONGID=`getent passwd |grep "^$DAEMONUSER:" | awk -F : '{print $4}'`
    else
        log_failure_msg "The user $DAEMONUSER, required to run $NAME does not exist."
        exit 0
    fi
fi


set -e

running_pid() {
# Check if a given process pid's cmdline matches a given name
    pid=$1
    name=$2
    [ -z "$pid" ] && return 1
    [ ! -d /proc/$pid ] &&  return 1
    cmd=`cat /proc/$pid/cmdline | tr "\000" "\n"|head -n 1 |cut -d : -f 1`
    # Is this the expected server
    [ "$cmd" != "$name" ] &&  return 1
    return 0
}

running() {
# Check if the process is running looking at /proc
# (works for all users)

    # No pidfile, probably no daemon present
    [ ! -f "$PIDFILE" ] && return 1
    pid=`cat $PIDFILE`
    running_pid $pid $DAEMON || return 1
    return 0
}

start_server() {
# Start the process using the wrapper
        if [ "x$START" != "xyes" -a "x$START" != "xtrue"  ]; then
            echo ""
            echo "beanstalkd not configured to start, please edit /etc/default/beanstalkd to enable"
            exit 0
        fi
        if [ -z "$DAEMONUSER" ] ; then
            start_daemon -p $PIDFILE $DAEMON $DAEMON_OPTS
            errcode=$?
        else
# if we are using a daemonuser then change the user id
            start-stop-daemon --start --quiet --pidfile $PIDFILE \
                        --chuid $DAEMONUSER --make-pidfile --oknodo \
                        --background --exec $DAEMON -- $DAEMON_OPTS
            errcode=$?
        fi
        return $errcode
}

stop_server() {
# Stop the process using the wrapper
        if [ -z "$DAEMONUSER" ] ; then
            killproc -p $PIDFILE $DAEMON
            errcode=$?
        else
# if we are using a daemonuser then look for process that match
            start-stop-daemon --stop --quiet --pidfile $PIDFILE \
                        --user $DAEMONUSER \
                        --exec $DAEMON
            errcode=$?
        fi
        rm -f $PIDFILE

        return $errcode
}

reload_server() {
    [ ! -f "$PIDFILE" ] && return 1
    pid=pidofproc $PIDFILE # This is the daemon's pid
    # Send a SIGHUP
    kill -1 $pid
    return $?
}

force_stop() {
# Force the process to die killing it manually
    [ ! -e "$PIDFILE" ] && return
    if running ; then
        kill -15 $pid
        # Is it really dead?
        sleep "$DIETIME"s
        if running ; then
            kill -9 $pid
            sleep "$DIETIME"s
            if running ; then
                echo "Cannot kill $NAME (pid=$pid)!"
                exit 0
            fi
        fi
    fi
    rm -f $PIDFILE
}


case "$1" in
  start)
        log_daemon_msg "Starting $DESC " "$NAME"
        # Check if it's running first
        if running ;  then
            log_progress_msg "apparently already running"
            log_end_msg 0
            exit 0
        fi
        if start_server ; then
            # NOTE: Some servers might die some time after they start,
            # this code will detect this issue if STARTTIME is set
            # to a reasonable value
            [ -n "$STARTTIME" ] && sleep $STARTTIME # Wait some time 
            if  running ;  then
                # It's ok, the server started and is running
                log_end_msg 0
            else
                # It is not running after we did start
                log_end_msg 1
            fi
        else
            # Either we could not start it
            log_end_msg 1
        fi
        ;;
  stop)
        log_daemon_msg "Stopping $DESC" "$NAME"
        if running ; then
            # Only stop the server if we see it running
            errcode=0
            stop_server || errcode=$?
            log_end_msg $errcode
        else
            # If it's not running don't do anything
            log_progress_msg "apparently not running"
            log_end_msg 0
            exit 0
        fi
        ;;
  force-stop)
        # First try to stop gracefully the program
        $0 stop
        if running; then
            # If it's still running try to kill it more forcefully
            log_daemon_msg "Stopping (force) $DESC" "$NAME"
            errcode=0
            force_stop || errcode=$?
            log_end_msg $errcode
        fi
        ;;
  restart|force-reload)
        log_daemon_msg "Restarting $DESC" "$NAME"
        errcode=0
        stop_server || errcode=$?
        # Wait some sensible amount, some server need this
        [ -n "$DIETIME" ] && sleep $DIETIME
        start_server || errcode=$?
        [ -n "$STARTTIME" ] && sleep $STARTTIME
        running || errcode=$?
        log_end_msg $errcode
        ;;
  status)

        log_daemon_msg "Checking status of $DESC" "$NAME"
        if running ;  then
            log_progress_msg "running"
            log_end_msg 0
        else
            log_progress_msg "apparently not running ... "
            log_end_msg 1
            exit 0
        fi
        ;;
  reload)
        log_warning_msg "Reloading $NAME daemon: not implemented (use restart)."
        ;;
  *)
        N=/etc/init.d/$NAME
        echo "Usage: $N {start|stop|force-stop|restart|force-reload|status}" >&2
        exit 1
        ;;
esac

exit 0

我的 beanstalkd.conf 文件位于文件夹 /etc/default 中,如下所示:

## Defaults for the beanstalkd init script, /etc/init.d/beanstalkd on
## Debian systems. Append "-b /var/lib/beanstalkd" for persistent
## storage.
BEANSTALKD_LISTEN_ADDR=0.0.0.0
BEANSTALKD_LISTEN_PORT=11300
# create the journal path before use !!!
BEANSTALKD_JOURNAL_PATH="/var/lib/beanstalkd" 
DAEMON_OPTS="-l $BEANSTALKD_LISTEN_ADDR -p $BEANSTALKD_LISTEN_PORT -z 550000000000 -b $BEANSTALKD_JOURNAL_PATH -V"

## Uncomment to enable startup during boot.
START=yes

最好的问候,

【问题讨论】:

标签: beanstalkd


【解决方案1】:

slickorange 在以下链接中的回答可能会对您有所帮助: JOB_TOO_BIG Pheanstalk - what can be done?

可以通过添加以下行来增加 beanstalkd 作业的大小(或取消注释现有的 BEANSTALKD_EXTRA 行并进行编辑):

BEANSTALKD_EXTRA="-z 524280"

大小以字节为单位,默认大小为 65535 字节。
进行更改后重新启动 beanstalkd:

sudo service beanstalkd restart

【讨论】:

    【解决方案2】:

    在 Debian 8.7 x64 上编辑 /etc/default/beanstalkd 中的设置有效。可能是一个错误。

    【讨论】:

      【解决方案3】:

      我在 Debian 8 中有systemctl,我的配置看起来与其他答案中建议的略有不同,以增加限制。这就是我所做的:

      1. 编辑beanstalkd 文件:

        nano /etc/sysconfig/beanstalkd
        
      2. 然后将MAX_JOB_SIZE 的限制从默认的 65535 增加到 524280

        MAX_JOB_SIZE=-z 524280
        
      3. 重启beanstalkd并查看状态:

        service beanstalkd restart
        systemctl status beanstalkd
        

      【讨论】:

        【解决方案4】:

        有守护进程的设置

        -b DIR   wal directory
         -f MS    fsync at most once every MS milliseconds (use -f0 for "always fsync")
         -F       never fsync (default)
         -l ADDR  listen on address (default is 0.0.0.0)
         -p PORT  listen on port (default is 11300)
         -u USER  become user and group
         -z BYTES set the maximum job size in bytes (default is 65535)
         -s BYTES set the size of each wal file (default is 10485760)
                    (will be rounded up to a multiple of 512 bytes)
         -c       compact the binlog (default)
         -n       do not compact the binlog
         -v       show version information
         -V       increase verbosity
         -h       show this help
        

        因此,根据您的 Linux,您应该找出保存位置并更改它。一般在beanstalkd.conf

        【讨论】:

        • 好的。当我输入“service beanstalkd start”时,我还会收到消息:“beanstalkd:无法识别的服务”。你知道在这种情况下我该怎么办吗?
        • 不确定如何在服务器上设置 beanstalkd。
        • 这样,“无法识别的服务”消息就解决了。但是我对最大值的大小仍然有同样的问题。工作规模。我将以下命令插入到 beanstalkd.conf 文件和 /etc/init.d/beanstalkd 的 init-script 中: -z 55000000 。但我仍然收到 JOB_TOO_BIG 错误消息。
        • 看起来不错,不知道哪里出了问题,需要重新启动,您甚至应该在进程列表中看到选项 -z
        猜你喜欢
        • 2014-06-18
        • 1970-01-01
        • 1970-01-01
        • 1970-01-01
        • 1970-01-01
        • 2020-07-07
        • 2015-05-17
        相关资源
        最近更新 更多