没有交易时 GAE 上的 TransactionFailedError

Question

我收到了这个错误：

TransactionFailedError: too much contention on these datastore entities. please try again.

即使我没有进行任何交易。导致错误的代码行是

ndb.put_multi(entity_list) # entity_list is a list of 100 entities

这个错误不会经常发生，所以没什么大不了的，但我很好奇为什么会收到这个错误。有什么想法吗？

这是大部分的回溯：

Traceback (most recent call last):
  ...
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 318, in post
    self.run_from_request()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 313, in run_from_request
    run(self.request.body)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 155, in run
    return func(*args, **kwds)
  File "/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/tasks.py", line 70, in start_election
    models.Voter.create(e.eid, chunk)
  File "/base/data/home/apps/s~opavote/2017-09-15.404125237783169549/models.py", line 2426, in create
    ndb.put_multi(voters + vbs)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3958, in put_multi
    for future in put_multi_async(entities, **ctx_options)]
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 824, in put
    key = yield self._put_batcher.add(entity, options)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 358, in _put_tasklet
    keys = yield self._conn.async_put(options, datastore_entities)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
    result = rpc.get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 928, in get_result
    result = rpc.get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1893, in __put_hook
    self.check_rpc_success(rpc)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1385, in check_rpc_success
    raise _ToDatastoreError(err)
TransactionFailedError: too much contention on these datastore entities. please try again.

Answer 1

请注意，错误实际上是从数据存储本身收到的，在 RPC 响应中：self.check_rpc_success(rpc)。

这让我怀疑，在数据存储方面，为了确保支持它的冗余基础设施的操作一致性/可靠性，每个写入操作实际上都使用与事务操作相同/相似的机制。不同之处在于，那些在客户端也有一些事务检查，在 RPC 交换之前/之后，可能还有数据存储的显式 RPC 事务开始/结束触发器。

来自Life of a Datastore Write 的引述表明，无论操作是否为事务性操作，都在使用一些通用机制（强调我的）：

如果提交阶段成功但应用阶段失败，则 数据存储区将前滚以将更改应用于两个以下的索引 情况：

1. 下次您对该实体组执行读取或写入或启动事务时，数据存储将首先滚动 转发并完全应用此已提交但未应用的写入，基于 日志中的数据。

失败的可能原因之一是对相同实体的并行访问太多，即使它们只是只读的。请参阅Contention problems in Google App Engine，尽管在这种情况下它们用于客户端的事务。

请注意，这只是一个理论；）

Answer 2

可能值得重新查看transactions and entity groups，注意各种定义和限制。

将“创建、更新或删除实体的每一次尝试都发生在事务的上下文中”以及“在单个实体组中的写入吞吐量限制约为每秒一个事务”，可能会这样说与您所看到的一样，特别是如果 entity_list 包含属于同一实体组的实体。