【发布时间】:2019-02-04 14:28:16
【问题描述】:
我使用 multiprocessing.managers 的 BaseManager 开发了 3 个脚本(一个服务器、一个写入器和一个读取器)。所有这些都在 AWS 或本地运行良好。但现在我想把事情搞混。对于更复杂的脚本,使用 GPU,我想使用我的本地机器,并且仍然在 AWS 之外运行它......
如何连接到在 AWS 上运行的创建 IPC、队列等的“服务器”?
(编辑:我找到了答案,请看下面,我删除了我之前留下的技术细节,并为有同样问题的人创建了一个更通用的问题和答案)
请查看以下 3 个实体的代码:
服务器:
from multiprocessing.managers import BaseManager
from queue import Queue
class Server():
def __init__(self, ip, port, authkey):
super(Server, self).__init__()
self.ip = ip
self.port = port
self.authkey = authkey
def make_server_manager(self):
""" Create a manager for the server, listening on the given port.
Return a manager object with all the required queues.
"""
q = Queue() # input queue
BaseManager.register('q', callable=lambda: q)
manager = BaseManager(address=(self.ip, self.port), authkey=self.authkey)
print('Server started at port ' + str(self.port))
return manager
if __name__ == "__main__":
ip = '172.31.2.45'
port = 11111
authkey = 'abc'.encode('utf-8')
s = Server(ip, port, authkey)
mgr = s.make_server_manager()
s = mgr.get_server()
s.serve_forever()
Writer(在 AWS 上运行,但如果我需要它也可以在我的机器上本地运行):
from multiprocessing.managers import BaseManager
class Client():
def __init__(self, ip, port, authkey):
super(Client, self).__init__()
self.ip = ip
self.port = port
self.authkey = authkey
# Create the queus and variables
BaseManager.register('q')
m = BaseManager(address=(ip, port), authkey=authkey)
m.connect()
self.q = m.q()
if __name__ == "__main__":
ip = '172.31.2.45'
port = 11111
authkey = 'abc'.encode('utf-8')
Client = Client(ip, port, authkey)
while True:
X = input('What is your secret message?')
Client.q.put(X)
if X == 'Close':
break
阅读器(在我的本地机器上运行以完成本练习)
from multiprocessing.managers import BaseManager
class Client():
def __init__(self, ip, port, authkey):
super(Client, self).__init__()
self.ip = ip
self.port = port
self.authkey = authkey
# Create the queus and variables
BaseManager.register('q')
m = BaseManager(address=(ip, port), authkey=authkey)
print('connecting...')
m.connect()
print('connected')
self.q = m.q()
if __name__ == "__main__":
ip = '18.191.25.217'
port = 11111
authkey = 'abc'.encode('utf-8')
Client = Client(ip, port, authkey)
while True:
rcvd = Client.q.get()
print(rcvd)
if rcvd == 'Close':
break
【问题讨论】:
标签: python amazon-ec2 multiprocessing ipc