深入理解子进程/守护进程/进程高可用实现

2019独角兽企业重金招聘Python工程师标准>>> 深入理解子进程/守护进程/进程高可用实现

单个进程:

说明: 进程是程序执行的实例,运行过程中,内核会将程序代码载入内存,为程序变量分配内存建立数据结构,记录进程相关信息,可以将其理解为容器,容器内的资源可以动态调整,但容器内的程序只能使用容器内的资源.

生成进程:

说明: 类Unix系统提供fork()系统调用,它非常特殊,普通函数调用1次返回1次,但它调用1次返回2次,因为操作系统自动把当前进程(父进程)复制出一份(子进程),然后在父进程和子进程内返回,子进程永远返回0,父进程返回子进程的pid,这样一个父进程可以fork多个子进程,父进程要记下每个子进程的pid,而子进程只需要os.getppid()就可以拿到父进程的pid

#!/usr/bin/env python

# -*- coding: utf-8 -*-

"""

#

# Authors: limanman

# OsChina: http://xmdevops.blog.51cto.com/

# Purpose:

#

"""

# 说明: 导入公共模块

import os

# 说明: 导入其它模块

if __name__ == '__main__':

print 'master process pid(%s) start ...' % (os.getpid())

pid = os.fork()

if pid == 0:

print 'I am a sub process (%s) created by %s.' % (os.getpid(), os.getppid())

else:

print 'I (%s) created a sub process (%s)' % (os.getpid(), pid)

说明: 由于os.fork()会返回2次,分别在子进程内返回然后在主进程内返回,所以如上代码if语句的两个判断都会被打印,第一次返回主进程创建了子进程,第二次进入子进程返回被主进程创建.

守护进程:

1. 守护进程也称后台进程,要实现后台进程必须使其与其与原运行环境隔离,包括未关闭的文件描述符,控制终端,会话,进程组,工作目录,以及文件创建掩码等等,有时还必须保证单实例运行

#!/usr/bin/env python

# -*- coding: utf-8 -*-

"""

#

# Authors: limanman

# OsChina: http://xmdevops.blog.51cto.com/

# Purpose:

#

"""

# 说明: 导入公共模块

import os

import sys

import time

import atexit

from signal import SIGTERM

# 说明: 导入其它模块

class Daemon(object):

def __init__(self, pidfile='daemon.pid', stdin=os.devnull,

stdout='daemon.log', stderr='daemon.log'):

self.stdin = stdin

self.stdout = stdout

self.stderr = stderr

self.pidfile = pidfile

def daemonize(self):

sys.stdout.write('current process => %s, leader process => %s%s' %

(os.getpid(), os.getsid(), os.linesep))

try:

pid = os.fork()

if pid != 0:

sys.exit(0)

except OSError, e:

sys.exit(1)

os.chdir('.')

os.umask(0)

os.setsid()

sys.stdout.write('current process => %s, leader process => %s%s' %

(os.getpid(), os.getsid(), os.linesep))

try:

pid = os.fork()

if pid != 0:

sys.exit(0)

except OSError, e:

sys.exit(1)

sys.stdout.write('current process => %s, leader process => %s' %

(os.getpid(), os.getsid(), os.linesep))

sys.stdout.flush()

sys.stderr.flush()

si = file(self.stdin, 'r')

so = file(self.stdout, 'a+')

se = file(self.stderr, 'a+')

os.dup2(si.fileno(), sys.stdin.fileno())

os.dup2(so.fileno(), sys.stdout.fileno())

os.dup2(se.fileno(), sys.stderr.fileno())

atexit.register(self.delpid)

strpid = ''.join([str(os.getpid()), os.linesep])

file(self.pidfile, 'w+b').write(strpid)

def delpid(self):

os.remove(self.pidfile)

def start(self):

try:

with open(self.pidfile, 'r+b') as f:

pid = int(f.read().strip())

except IOError, e:

pid = None

if pid:

msg = 'pidfile %s already exist. Daemon already running?%s'

sys.stdout.write(msg % (self.pidfile, os.linesep))

sys.exit(1)

self.daemonize()

self.run()

def stop(self):

try:

with open(self.pidfile, 'r+b') as f:

pid = int(f.read().strip())

except IOError, e:

pid = None

if not pid:

msg = 'pidfile %s does not exist. Daemon not running?%s'

sys.stdout.write(msg % (self.pidfile, os.linesep))

sys.exit(1)

try:

while True:

os.kill(pid, SIGTERM)

time.sleep(0.1)

except OSError, e:

if os.path.exists(self.pidfile):

self.delpid()

sys.exit(1)

def restart(self):

self.stop()

self.start()

def run(self):

pass

#!/usr/bin/env python

# -*- coding: utf-8 -*-

"""

#

# Authors: limanman

# OsChina: http://xmdevops.blog.51cto.com/

# Purpose:

#

"""

# 说明: 导入公共模块

import os

import time

from daemon import Daemon

# 说明: 导入其它模块

class TaskRun(Daemon):

def __init__(self, *args, **kwargs):

super(TaskRun, self).__init__(*args, **kwargs)

def run(self):

with open('running.log', 'a+b')

while True:

timestamp = str(time.time())

f.write(''.join([timestamp, os.linesep]))

f.flush()

time.sleep(1)

if __name__ == '__main__':

t = TaskRun()

t.start()

进程管控:

说明: 在运维开发中很多时候程序BUG导致程序异常退出,crond时间精度上不可控,常常导致一段时间服务不可用,为了增强程序的可用性,可让子进程处理业务,主进程接收子进程SIGCHLD信号,此信号为系统默认信号子进程退出时会主动发送给主进程,主进程只需要捕捉此信号并启动新的子进程接管业务即可实现业务的高可用

#!/usr/bin/env python

# -*- coding: utf-8 -*-

"""

#

# Authors: limanman

# OsChina: http://xmdevops.blog.51cto.com/

# Purpose:

#

"""

# 说明: 导入公共模块

import time

import signal

import multiprocessing

# 说明: 导入其它模块

def task(interval):

for _ in xrange(5):

print _

time.sleep(interval)

print 'found notice: sub process(%s) exit with error!' % (multiprocessing.current_process())

def signal_handler(sig_num, frame):

p = multiprocessing.Process(target=task, args=(1,))

p.daemon = True

p.start()

if __name__ == '__main__':

p = multiprocessing.Process(target=task, args=(1,))

p.daemon = True

p.start()

s = signal.signal(signal.SIGCHLD, signal_handler)

while True:

signal.pause()

登录乐搏学院官网http://www.learnbo.com/

或关注我们的官方微博微信，还有更多惊喜哦~

深入理解子进程/守护进程/进程高可用实现

本文出自 “满满李 - 运维开发之路” 博客，请务必保留此出处http://xmdevops.blog.51cto.com/11144840/1861815

转载于:https://my.oschina.net/learnbo/blog/846900