【发布时间】：2014-10-23 11:01:56
【问题描述】：
在远程节点上部署一些 Apache Kafka 实例后，我发现作为 Kafka 存档一部分的 kafka-server-stop.sh 脚本存在问题。
默认包含：
#!/bin/sh
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
ps ax | grep -i 'kafka\.Kafka' | grep java | grep -v grep | awk '{print $1}' | xargs kill -SIGTERM
如果我将 apache kafka 作为非后台进程执行，则此脚本效果很好，例如：
/var/lib/kafka/bin/kafka-server-start.sh /var/lib/kafka/config/server.properties
当我将它作为后台进程执行时它也可以工作：
/var/lib/kafka/bin/kafka-server-start.sh /var/lib/kafka/config/server.properties &
但是在我的远程节点上，我使用这个 python 脚本执行它（使用 Ansible）：
#!/usr/bin/env python
import argparse
import os
import subprocess
KAFKA_PATH = "/var/lib/kafka/"
def execute_command_pipe_output(command_to_call):
return subprocess.Popen(command_to_call, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
def execute_command_no_output(command_to_call):
with open(os.devnull, "w") as null_file:
return subprocess.Popen(command_to_call, stdout=null_file, stderr=subprocess.STDOUT)
def start_kafka(args):
command_to_call = ["nohup"]
command_to_call += [KAFKA_PATH + "bin/zookeeper-server-start.sh"]
command_to_call += [KAFKA_PATH + "config/zookeeper.properties"]
proc = execute_command_no_output(command_to_call)
command_to_call = ["nohup"]
command_to_call += [KAFKA_PATH + "bin/kafka-server-start.sh"]
command_to_call += [KAFKA_PATH + "config/server.properties"]
proc = execute_command_no_output(command_to_call)
def stop_kafka(args):
command_to_call = [KAFKA_PATH + "bin/kafka-server-stop.sh"]
proc = execute_command_pipe_output(command_to_call)
for line in iter(proc.stdout.readline, b''):
print line,
command_to_call = [KAFKA_PATH + "bin/zookeeper-server-stop.sh"]
proc = execute_command_pipe_output(command_to_call)
for line in iter(proc.stdout.readline, b''):
print line,
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Starting Zookeeper and Kafka instances")
parser.add_argument('action', choices=['start', 'stop'], help="action to take")
args = parser.parse_args()
if args.action == 'start':
start_kafka(args)
elif args.action == 'stop':
stop_kafka(args)
else:
parser.print_help()
执行后
manage-kafka.py start
manage-kafka.py stop
Zookeeper 已关闭（应该如此），但 Kafka 仍在运行。
更有趣的是，当我（手动）调用时
nohup /var/lib/kafka/bin/kafka-server-stop.sh
或
nohup /var/lib/kafka/bin/kafka-server-stop.sh &
kafka-server-stop.sh 正确关闭 Kafka 实例。我怀疑这个问题可能是由一些 Linux/Python 引起的。
【问题讨论】：
-
只是出于好奇：为什么要使用 python？为什么不使用 upstart 并进行类似“service kafka-broker start/stop”之类的 python 调用？
-
它是作为第一个解决方案创建的，我只是对为什么观察所描述的行为感兴趣。我将把它转移到主管进行管理，因为我们已经将它用于不同的应用程序。
-
你找到答案了吗？
标签： python linux apache-kafka