tornado使用了单进程(当然也可以多进程) + 协程 + I/O多路复用的机制,解决了C10K中因为过多的线程(进程)的上下文切换 而导致的cpu资源的浪费。

tornado中的I/O多路复用前面已经讲过了。本文不做详细解释。

来看一下tornado中的协程模块:tornado.gen:

tornado.gen是根据生成器(generator)实现的,用来更加简单的实现异步。

先来说一下tornado.gen.coroutine的实现思路:

  我们知道generator中的yield语句可以使函数暂停执行,而send()方法则可以恢复函数的执行。

  tornado将那些异步操作放置到yield语句后,当这些异步操作完成后,tornado会将结果send()至generator中恢复函数执行。

在tornado的官方文档中有这么一句话:

Most asynchronous functions in Tornado return a Future; yielding this object returns its result.

就是说:

  在tornado中大多数的异步操作返回一个Future对象

  yield Future对象 会返回该异步操作的结果,这句话的意思就是说 假如 ret = yield some_future_obj 当some_future_obj所对应的异步操作完成后会自动的将该异步操作的结果赋值给 ret

那么,Future对象到底是什么?

一  Future对象

先来说说Future对象:

Future对象可以概括为: 一个异步操作的占位符,当然这个占位符有些特殊,它特殊在:

  1 这个占位符是一个对象

  2 这个对象包含了很多属性,包括_result 以及 _callbacks,分别用来存储异步操作的结果以及回调函数

  3 这个对象包含了很多方法,比如添加回调函数,设置异步操作结果等。

  4 当这个对象对应的异步操作完成后,该对象会被set_done,然后遍历并运行_callbacks中的回调函数

来看一下Future的简化版

class Future(object):
    '''
        Future对象主要保存一个回调函数列表_callbacks与一个执行结果_result,当我们set_result时,就会执行_callbacks中的函数
        如果set_result或者set_done,就会遍历_callbacks列表并执行callback(self)函数
    '''
    def __init__(self):
        self._result = None    # 执行的结果
        self._callbacks = []    # 用来保存该future对象的回调函数

    def result(self, timeout=None):
        # 如果操作成功,返回结果。如果失败则抛出异常
        self._clear_tb_log()
        if self._result is not None:
            return self._result
        if self._exc_info is not None:
            raise_exc_info(self._exc_info)
        self._check_done()
        return self._result

    def add_done_callback(self, fn):
        if self._done:
            fn(self)
        else:
            self._callbacks.append(fn)

    def set_result(self, result):
        self._result = result
        self._set_done()

    def _set_done(self):
        # 执行结束(成功)后的操作。
        self._done = True
        for cb in self._callbacks:
            try:
                cb(self)
            except Exception:
                app_log.exception('Exception in callback %r for %r', cb, self)
        self._callbacks = None

完整源码:

class Future(object):
    '''
        Future对象主要保存一个回调函数列表_callbacks与一个执行结果_result,当我们set_result时,就会执行_callbacks中的函数
    '''
    def __init__(self):
        self._done = False  # 是否执行完成
        self._result = None    # 执行的结果
        self._exc_info = None    # 执行的异常信息

        self._log_traceback = False   # Used for Python >= 3.4
        self._tb_logger = None        # Used for Python <= 3.3

        self._callbacks = []    # 用来保存该future对象的回调函数

    # Implement the Python 3.5 Awaitable protocol if possible
    # (we can't use return and yield together until py33).
    if sys.version_info >= (3, 3):
        exec(textwrap.dedent("""
        def __await__(self):
            return (yield self)
        """))
    else:
        # Py2-compatible version for use with cython.
        def __await__(self):
            result = yield self
            # StopIteration doesn't take args before py33,
            # but Cython recognizes the args tuple.
            e = StopIteration()
            e.args = (result,)
            raise e

    def cancel(self):
        """Cancel the operation, if possible. 如果可能的话取消操作
        tornado对象不支持取消操作,所以总是返回False
        """
        return False

    def cancelled(self):
        # 同上
        return False

    def running(self):
        """Returns True if this operation is currently running."""
        return not self._done

    def done(self):
        """Returns True if the future has finished running."""
        return self._done

    def _clear_tb_log(self):
        self._log_traceback = False
        if self._tb_logger is not None:
            self._tb_logger.clear()
            self._tb_logger = None

    def result(self, timeout=None):
        """If the operation succeeded, return its result.  If it failed,
        re-raise its exception. 如果操作成功,返回结果。如果失败则抛出异常

        This method takes a ``timeout`` argument for compatibility with
        `concurrent.futures.Future` but it is an error to call it
        before the `Future` is done, so the ``timeout`` is never used.
        """
        self._clear_tb_log()
        if self._result is not None:
            return self._result
        if self._exc_info is not None:
            raise_exc_info(self._exc_info)
        self._check_done()
        return self._result

    def exception(self, timeout=None):
        """If the operation raised an exception, return the `Exception`
        object.  Otherwise returns None.

        This method takes a ``timeout`` argument for compatibility with
        `concurrent.futures.Future` but it is an error to call it
        before the `Future` is done, so the ``timeout`` is never used.
        """
        self._clear_tb_log()
        if self._exc_info is not None:
            return self._exc_info[1]
        else:
            self._check_done()
            return None

    def add_done_callback(self, fn):
        """Attaches the given callback to the `Future`. 将callback附加到

        It will be invoked with the `Future` as its argument when the Future
        has finished running and its result is available.  In Tornado
        consider using `.IOLoop.add_future` instead of calling
        `add_done_callback` directly.
        """
        if self._done:
            fn(self)
        else:
            self._callbacks.append(fn)

    def set_result(self, result):
        """Sets the result of a ``Future``. 将 result 设置为该future对象的结果

        It is undefined to call any of the ``set`` methods more than once
        on the same object.
        """
        self._result = result
        self._set_done()

    def set_exception(self, exception):
        """Sets the exception of a ``Future.``"""
        self.set_exc_info(
            (exception.__class__,
             exception,
             getattr(exception, '__traceback__', None)))

    def exc_info(self):
        """Returns a tuple in the same format as `sys.exc_info` or None.

        .. versionadded:: 4.0
        """
        self._clear_tb_log()
        return self._exc_info

    def set_exc_info(self, exc_info):
        """Sets the exception information of a ``Future.``

        Preserves tracebacks on Python 2.

        .. versionadded:: 4.0
        """
        self._exc_info = exc_info
        self._log_traceback = True
        if not _GC_CYCLE_FINALIZERS:
            self._tb_logger = _TracebackLogger(exc_info)

        try:
            self._set_done()
        finally:
            # Activate the logger after all callbacks have had a
            # chance to call result() or exception().
            if self._log_traceback and self._tb_logger is not None:
                self._tb_logger.activate()
        self._exc_info = exc_info

    def _check_done(self):
        if not self._done:
            raise Exception("DummyFuture does not support blocking for results")

    def _set_done(self):
        # 执行结束(成功)后的操作。
        self._done = True
        for cb in self._callbacks:
            try:
                cb(self)
            except Exception:
                app_log.exception('Exception in callback %r for %r', cb, self)
        self._callbacks = None

    # On Python 3.3 or older, objects with a destructor part of a reference
    # cycle are never destroyed. It's no longer the case on Python 3.4 thanks to
    # the PEP 442.
    if _GC_CYCLE_FINALIZERS:
        def __del__(self):
            if not self._log_traceback:
                # set_exception() was not called, or result() or exception()
                # has consumed the exception
                return

            tb = traceback.format_exception(*self._exc_info)

            app_log.error('Future %r exception was never retrieved: %s',
                          self, ''.join(tb).rstrip())
Future源码

相关文章: