kombu 升级后获取 <> 不是用于 task_id 的 JSON 序列化

Question

升级后我会收到ERROR EncodeError(TypeError('6JQAKHNMG9 is not JSON serializable',),)。

安装成功：

• amqp-2.1.4
• 台球-3.5.0.2
• celery-4.0.2
• kombu-4.0.2
• pytz-2017.2

（Flower 0.9.1 安装最初需要）

正在运行：

• amqp==1.4.9
• 台球==3.3.0.23
• 芹菜==3.1.23
• kombu==3.0.35
• pytz==2016.10

现在我明白了：

05/10/2017 06:37:55.045 ERROR EncodeError(TypeError('6JQAKHNMG9 is not JSON serializable',),)
Traceback (most recent call last):
  File "/usr/local/src/gonzo/api/version1_0/application/api_main.py", line 191, in post
    retries=3)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/base.py", line 737, in send_task
    amqp.send_task_message(P, name, message, **options)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/amqp.py", line 558, in send_task_message
    **properties
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 169, in publish
    compression, headers)
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 252, in _prepare
    body) = dumps(body, serializer=serializer)
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 54, in _reraise_errors
    reraise(wrapper, wrapper(exc), sys.exc_info()[2])
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 50, in _reraise_errors
    yield
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/json.py", line 72, in dumps
    **dict(default_kwargs, **kwargs))
  File "/usr/lib/python2.7/json/__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/json.py", line 62, in default
    return super(JSONEncoder, self).default(o)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
EncodeError: 6JQAKHNMG9 is not JSON serializable

我的代码：

我传递了一个自定义“task_id”，它是一个字符串，（不是 JSON 对象）它在升级之前曾经工作过（如果我恢复版本它再次工作），传递自定义 task_id 的正确格式可能是什么？

celery.send_task('process_campaign',
                             exchange='gonzo',
                             queue='gold',
                             routing_key='gonzo.gold',
                             kwargs={'campaign_instance': campaign_instance},
                             task_id=campaign_instance.reference,
                             retries=3)

我试过了：

task_id=json.dumps(campaign_instance.reference) or
task_id={'task_id': campaign_instance.reference}

同样的错误，有什么想法，我找不到这个问题的文档吗？

Answer 1

由于version 4.0，celery 中的默认序列化程序是'json'。之前的默认值为'pickle'。 （我不知道kombu包版本的详细信息）

发送消息时，您传递了 campaign_instance 参数，该参数可能不是 JSON 可序列化的，但 PICKLE 可序列化，这就是它以前工作的原因。

您可以在设置中更改默认序列化程序，指定'pickle'，这是最短的方法：http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_serializer

或者您可以修改代码以仅将 JSON 可序列化参数传递给您的任务。