在 App Engine 开发服务器上使用 SQLAlchemy

Question

我看到 some questions 关于在 App Engine 上使用 SQLAlchemy 连接到 Google Cloud SQL。但我不确定是否可以使用本地 MySQL 数据库和现有的 SQLAlchemy 方言进行开发。在我第一次尝试时，我将 SQLAlchemy 0.8.0 添加到应用程序并定义了一个架构：

from sqlalchemy import create_engine, Column, Integer, Table
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()    

foo_table = Table('foo', Base.metadata,
  Column('id', Integer, primary_key=True, autoincrement=True),
)

当我尝试使用以下方法在开发服务器上创建表时：

url = 'mysql+gaerdbms:///%s?instance=%s' % ('database_name', 'instance_name')
engine = create_engine(url)
Base.metadata.create_all(engine)

...我收到错误DBAPIError: (ImportError) No module named pwd None None，这意味着 SQLAlchemy 正在导入一个被开发服务器列入黑名单的模块。

我做错了吗？或者，如果没有，我应该怎么做才能在开发服务器上使用 SQLAlchemy？或者第一个问题可能是：我可以使用 SQLAlchemy 的gaerdbms 方言在本地 MySql 数据库中使用开发服务器进行开发吗？

编辑：此错误不仅在尝试创建表时发生。我手动创建了表并尝试查询它们，并且发生了同样的错误。

完整的回溯是：

Traceback (most recent call last):
  File "[...]/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "[...]/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "[...]/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "[...]/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "[...]/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "[...]/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "[...]/webapp/admin.py", line 12, in get
    db.Base.metadata.create_all(engine)
  File "[...]/webapp/sqlalchemy/schema.py", line 2784, in create_all
    tables=tables)
  File "[...]/webapp/sqlalchemy/engine/base.py", line 1486, in _run_visitor
    with self._optional_conn_ctx_manager(connection) as conn:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "[...]/webapp/sqlalchemy/engine/base.py", line 1479, in _optional_conn_ctx_manager
    with self.contextual_connect() as conn:
  File "[...]/webapp/sqlalchemy/engine/base.py", line 1669, in contextual_connect
    self.pool.connect(),
  File "[...]/webapp/sqlalchemy/pool.py", line 272, in connect
    return _ConnectionFairy(self).checkout()
  File "[...]/webapp/sqlalchemy/pool.py", line 425, in __init__
    rec = self._connection_record = pool._do_get()
  File "[...]/webapp/sqlalchemy/pool.py", line 855, in _do_get
    return self._create_connection()
  File "[...]/webapp/sqlalchemy/pool.py", line 225, in _create_connection
    return _ConnectionRecord(self)
  File "[...]/webapp/sqlalchemy/pool.py", line 318, in __init__
    self.connection = self.__connect()
  File "[...]/webapp/sqlalchemy/pool.py", line 368, in __connect
    connection = self.__pool._creator()
  File "[...]/webapp/sqlalchemy/engine/strategies.py", line 80, in connect
    return dialect.connect(*cargs, **cparams)
  File "[...]/webapp/sqlalchemy/engine/default.py", line 279, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "[...]/google_appengine/google/storage/speckle/python/api/rdbms_googleapi.py", line 183, in __init__
    super(GoogleApiConnection, self).__init__(*args, **kwargs)
  File "[...]/google_appengine/google/storage/speckle/python/api/rdbms.py", line 810, in __init__
    self.OpenConnection()
  File "[...]/google_appengine/google/storage/speckle/python/api/rdbms.py", line 832, in OpenConnection
    self.SetupClient()
  File "[...]/google_appengine/google/storage/speckle/python/api/rdbms_googleapi.py", line 193, in SetupClient
    self._client = RdbmsGoogleApiClient(**kwargs)
  File "[...]/google_appengine/google/storage/speckle/python/api/rdbms_googleapi.py", line 106, in __init__
    rdbms.OAUTH_CREDENTIALS_PATH)
  File "/usr/lib/python2.7/posixpath.py", line 259, in expanduser
    import pwd
  File "[...]/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 822, in load_module
    raise ImportError('No module named %s' % fullname)
DBAPIError: (ImportError) No module named pwd None None

Answer 1

我找到了解决方法。事实上，SQLAlchemy 的gaerdbms 方言无法连接到本地数据库。但是用下面的方言就可以了。按照this answer 的说明进行操作，但请改用此方言：

# mysql/gaerdbms.py
# Copyright (C) 2005-2013 the SQLAlchemy authors and contributors <see AUTHORS file>
#
# This module is part of SQLAlchemy and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
"""
.. dialect:: mysql+gaerdbms
    :name: Google Cloud SQL
    :dbapi: rdbms
    :connectstring: mysql+gaerdbms:///<dbname>?instance=<instancename>
    :url: https://developers.google.com/appengine/docs/python/cloud-sql/developers-guide

    This dialect is based primarily on the :mod:`.mysql.mysqldb` dialect with minimal
    changes.

    .. versionadded:: 0.7.8


Pooling
-------

Google App Engine connections appear to be randomly recycled,
so the dialect does not pool connections.  The :class:`.NullPool`
implementation is installed within the :class:`.Engine` by
default.

"""

import os
import re

from sqlalchemy.dialects.mysql.mysqldb import MySQLDialect_mysqldb
from sqlalchemy.pool import NullPool


class MySQLDialect_gaerdbms(MySQLDialect_mysqldb):

    @classmethod
    def dbapi(cls):
        # from django:
        # http://code.google.com/p/googleappengine/source/
        #     browse/trunk/python/google/storage/speckle/
        #     python/django/backend/base.py#118
        # see also [ticket:2649]
        # see also https://stackoverflow.com/q/14224679/34549
        if is_production():
            # Production mode.
            from google.storage.speckle.python.api import rdbms_apiproxy
            return rdbms_apiproxy
        elif is_remote_mode():
            # Development mode with remote database.
            from google.storage.speckle.python.api import rdbms_googleapi
            return rdbms_googleapi
        else:
            # Development mode with local database.
            from google.appengine.api import rdbms_mysqldb
            return rdbms_mysqldb

    @classmethod
    def get_pool_class(cls, url):
        # Cloud SQL connections die at any moment
        return NullPool

    def create_connect_args(self, url):
        opts = url.translate_connect_args()
        if is_production() or is_remote_mode():
            # 'dsn' and 'instance' are because we are skipping
            # the traditional google.api.rdbms wrapper.
            # they are not needed in local mode; 'dns' even causes an error.
            opts['dsn'] = ''
            opts['instance'] = url.query['instance']
        return [], opts

    def _extract_error_code(self, exception):
        match = re.compile(r"^(\d+):|^\((\d+),").match(str(exception))
        # The rdbms api will wrap then re-raise some types of errors
        # making this regex return no matches.
        code = match.group(1) or match.group(2) if match else None
        if code:
            return int(code)

dialect = MySQLDialect_gaerdbms

def is_production():
  return os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine')

def is_remote_mode():
  return os.getenv('SETTINGS_MODE') == 'prod'

此方言在开发服务器上运行时默认使用本地数据库。要在开发期间使用对 Google Cloud SQL 的远程访问，必须在环境中设置一个变量，遵循the pattern used by Django：

os.environ['SETTINGS_MODE'] = 'prod'