【问题标题】:Python grpc fail : PicklingError: Can't pickle <class 'demo_pb2.msg'>: import of module demo_pb2 failedPython grpc 失败:PicklingError: Can\'t pickle <class \'demo_pb2.msg\'>: import of module demo_pb2 failed
【发布时间】:2022-08-04 01:57:13
【问题描述】:

无法导入 protobuf 生成的 python 模块。这是代码。

  • error message

这是运行python main.py 后的错误消息。

# python main.py 
Traceback (most recent call last):
  File \"/usr/lib64/python2.7/multiprocessing/queues.py\", line 266, in _feed
    send(obj)
PicklingError: Can\'t pickle <class \'demo_pb2.msg\'>: import of module demo_pb2 failed
^CTraceback (most recent call last):
  File \"main.py\", line 31, in <module>
    main()
  File \"main.py\", line 24, in main
    for msg in iter(q.get, None):
  File \"/usr/lib64/python2.7/multiprocessing/queues.py\", line 117, in get
    res = self._recv()
KeyboardInterrupt
  • 代码树
.
├── __init__.py
├── main.py
└── pb
    ├── demo_pb2_grpc.py
    ├── demo_pb2.py
    ├── demo.proto
    ├── __init__.py
    └── run_codegen.py
  • main.py
from multiprocessing import Queue
import os
from threading import Thread
import time
from pb import demo_pb2

q = Queue()


def generate_file_path(path):
    for root, dirs, files in os.walk(path):
        for dir_ in dirs:
            q.put(demo_pb2.msg(path=os.path.join(root, dir_)))
            time.sleep(0.1)
        for file_ in files:
            q.put(demo_pb2.msg(path=os.path.join(root, file_)))
            time.sleep(0.1)
    q.put(None)


def main():
    t = Thread(target=generate_file_path, args=(\'/root/pip\',))
    t.start()
    for msg in iter(q.get, None):
        print(msg)
    q.close()
    t.join()


if __name__ == \'__main__\':
    main()
  • pb/demo.proto
syntax = \"proto3\";

package demo;

message msg {
    string path = 1;
}
  • pb/run_codegen.py
from grpc_tools import protoc

protoc.main((
    \'-I./\',
    \'--python_out=./\',
    \'--grpc_python_out=./\',
    \'demo.proto\',
))
  • 你好@iskylite,欢迎来到 StackOverflow!请与我们分享demo_pb2.msg 类的定义。此外,如果您将代码设为minimal reproducible example,我们将不胜感激。
  • 您好@David,感谢您的回复。但是我没有找到文件demo_pb2.msg,文件在哪里?以上是最小的可重现代码。实际上只需要main.pypb/demo.proto两个文件。
  • 请分享文件pb/demo_pb2.py 的内容。里面有应该成为class msg: 定义(除非msg 是从某处导入的)。

标签: python grpc grpc-python


【解决方案1】:

Protobuf 消息不可提取。 multiprocessing.Queue 要求入队的对象是 picleable。也许尝试在一端序列化为 JSON,然后在另一端反序列化。

【讨论】:

    猜你喜欢
    • 2023-01-11
    • 2022-08-09
    • 2022-12-28
    • 2022-10-21
    • 2018-11-19
    • 2011-12-13
    • 2010-11-27
    • 1970-01-01
    • 2022-11-20
    相关资源
    最近更新 更多