【问题标题】:How to use tornado coroutines in this case?在这种情况下如何使用龙卷风协程?
【发布时间】:2016-12-15 08:37:39
【问题描述】:

我创建了 Tornado 服务器,它接受 python 和 matlab 代码并执行它。这是服务器代码。

from zmq.eventloop.zmqstream import ZMQStream
from zmq.eventloop import ioloop
ioloop.install()

from functools import partial
from tornado import web, gen, escape
from tornado import options, httpserver
from tornado.concurrent import Future

settings = dict()
settings['autoreload'] = True 
settings['debug'] = True

from jupyter_client import MultiKernelManager

reply_futures = {}
kids = []

class MainHandler(web.RequestHandler):
    def get(self):
        self.write("Hello")

class ExecuteHandler(web.RequestHandler):
    def get(self):
        self.write("approaching execute")

    def post(self):
        data = escape.json_decode(self.request.body)
        print data
        self.application.execute_code(data)

class Application(web.Application):

    def __init__(self):

        handlers = []
        handlers.append((r"/", MainHandler))
        handlers.append((r"/execute", ExecuteHandler))
        web.Application.__init__(self, handlers, **settings)
        self.km = MultiKernelManager()
        self.setup_kernels()

    def setup_kernels(self):

        matlab_kernel_id = self.km.start_kernel(kernel_name="matlab")
        python_kernel_id = self.km.start_kernel(kernel_name="python")

        self.matlab_kernel_id = matlab_kernel_id
        self.python_kernel_id = python_kernel_id

        matkab_kernel_client = self.km.get_kernel(matlab_kernel_id).client()
        matkab_kernel_client.start_channels()

        python_kernel_client = self.km.get_kernel(python_kernel_id).client()
        python_kernel_client.start_channels()

        self.matkab_kernel_client = matkab_kernel_client
        self.python_kernel_client = python_kernel_client

        matlab_iopub_stream = ZMQStream(matkab_kernel_client.iopub_channel.socket)
        matlab_shell_stream = ZMQStream(matkab_kernel_client.shell_channel.socket)

        python_iopub_stream = ZMQStream(python_kernel_client.iopub_channel.socket)
        python_shell_stream = ZMQStream(python_kernel_client.shell_channel.socket)

        matlab_iopub_stream.on_recv_stream(partial(self.reply_callback, matkab_kernel_client.session))
        matlab_shell_stream.on_recv_stream(partial(self.reply_callback, matkab_kernel_client.session))

        python_iopub_stream.on_recv_stream(partial(self.reply_callback, python_kernel_client.session))
        python_shell_stream.on_recv_stream(partial(self.reply_callback, python_kernel_client.session))

    def reply_callback(self, session, stream, msg_list):
        idents, msg_parts = session.feed_identities(msg_list)
        reply = session.deserialize(msg_parts)

        if "stream" == reply["msg_type"]:
            print reply["content"]["text"]
        parent_id = reply['parent_header'].get('msg_id')
        reply_future = reply_futures.get(parent_id)
        if reply_future:
            reply_future.set_result(reply)

    def execute_code(self, data):

        matlab_code = data['matlab_code']
        python_code = data['python_code']

        self.execute_matlab_then_python(matlab_code, python_code)

    @gen.coroutine
    def execute_matlab_then_python(self, matlab_code, python_code):

        print "executing matlab code"
        parent_id1 = self.matkab_kernel_client.execute(matlab_code)
        f1 = reply_futures[parent_id1] = Future()
        yield f1    

        print "executing python code"
        parent_id2 = self.python_kernel_client.execute(python_code)
        f2 = reply_futures[parent_id2] = Future()
        yield f2

    def shutdown_kernels(self):
        self.km.get_kernel(self.matlab_kernel_id).cleanup_connection_file()
        self.km.shutdown_kernel(self.matlab_kernel_id)

        self.km.get_kernel(self.python_kernel_id).cleanup_connection_file()
        self.km.shutdown_kernel(self.python_kernel_id)

if __name__ == '__main__':

    options.parse_command_line()
    app = Application()
    server = httpserver.HTTPServer(app)
    server.listen(8888)

    try:
        ioloop.IOLoop.current().start()
    except KeyboardInterrupt:
        print 'going down'
    finally:    
        app.shutdown_kernels()

我用来访问的客户端代码在这里

import json
import requests

matlab_code = "a=magic(3);disp(a)"
python_code = "print 'Hello World!!'"

data = {}
data['matlab_code'] = matlab_code
data['python_code'] = python_code

r = requests.post('http://0.0.0.0:8888/execute', json.dumps(data))

我关心的是保持执行顺序,这样 python 代码只有在 matlab 完成后才会执行。我正在使用 jupyter_client 执行 matlab/python 代码。我在这里使用python27。问题是当我提交代码时它会抛出TypeError: 'NoneType' object is not iterable。这是它的堆栈跟踪。

[E 160809 15:17:51 ioloop:633] 回调中的异常无 回溯(最近一次通话最后): 文件“/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py”,第 887 行,开始 handler_func(fd_obj,事件) 文件“/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py”,第 275 行,在 null_wrapper 返回 fn(*args, **kwargs) _handle_events 中的文件“/usr/local/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py”,第 440 行 self._handle_recv() _handle_recv 中的文件“/usr/local/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py”,第 472 行 self._run_callback(回调,味精) _run_callback 中的文件“/usr/local/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py”,第 414 行 回调(*args,**kwargs) 文件“/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py”,第 275 行,在 null_wrapper 返回 fn(*args, **kwargs) 文件“/usr/local/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py”,第 191 行,在 self.on_recv(lambda msg:回调(self,msg),复制=复制) 文件“tornado_test.py”,第 86 行,reply_callback reply_future.set_result(回复) 文件“/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py”,第 276 行,在 set_result self._set_done() _set_done 中的文件“/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py”,第 320 行 对于 self._callbacks 中的 cb: TypeError:“NoneType”对象不可迭代

我不明白这里有什么问题?

【问题讨论】:

  • 最好将代码的所有相关部分都放在这里,以便让您的问题对其他人有所帮助,因为链接和存储库可能会失效。

标签: python asynchronous tornado jupyter coroutine


【解决方案1】:

不幸的神秘错误消息“'NoneType' object is not iterable”意味着您在同一个Future对象上多次调用set_result。我对 zmq 或 jupyter 内核接口了解不多,但我的猜测是两个不同内核的 execute 方法返回的 ID 是重叠的,或者每次执行调用都会得到多个响应(来自两个不同的流?)。

【讨论】:

    猜你喜欢
    • 2018-10-10
    • 1970-01-01
    • 1970-01-01
    • 2022-01-23
    • 2015-11-30
    • 2017-05-16
    • 2012-10-06
    • 2011-05-18
    • 2019-11-29
    相关资源
    最近更新 更多