如何在 Windows 10 中修复 python 中的多处理问题

Question

我尝试使用本教程来训练我自己的汽车模型识别模型：https://github.com/Helias/Car-Model-Recognition。我想使用 coda 和我的 gpu 性能来提高训练速度（预处理步骤已完成，没有任何错误）。但是当我尝试训练我的模型时，出现以下错误：

######### ERROR #######

        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.


######### batch #######
Traceback (most recent call last):
  File "D:\Car-Model-Recognition\main.py", line 78, in train_model


######### ERROR #######
[Errno 32] Broken pipe
    for i, batch in enumerate(loaders[mode]):


######### batch #######  File "C:\Program Files\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 279, in __iter__

    return _MultiProcessingDataLoaderIter(self)
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 719, in __init__
  File "main.py", line 78, in train_model
    w.start()
  File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 112, in start
    for i, batch in enumerate(loaders[mode]):
  File "C:\Program Files\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 279, in __iter__
    self._popen = self._Popen(self)
  File "C:\Program Files\Python37\lib\multiprocessing\context.py", line 223, in _Popen
    return _MultiProcessingDataLoaderIter(self)
  File "C:\Program Files\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 719, in __init__
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Program Files\Python37\lib\multiprocessing\context.py", line 322, in _Popen
    w.start()
    return Popen(process_obj)
  File "C:\Program Files\Python37\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
  File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 112, in start
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Program Files\Python37\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    self._popen = self._Popen(self)
  File "C:\Program Files\Python37\lib\multiprocessing\context.py", line 223, in _Popen
    _check_not_importing_main()
  File "C:\Program Files\Python37\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Program Files\Python37\lib\multiprocessing\context.py", line 322, in _Popen
    is not going to be frozen to produce an executable.''')
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
    return Popen(process_obj)

我使用了给定链接中的确切代码，如果我使用 wsl 启动我的代码，一切正常，但我不能使用来自 wsl 的 gpu。我应该在哪里插入此 name == 'ma​​in' 检查以防止此类错误或如何禁用此多处理

Answer 1

查看main.py，您在模块级别运行了大量代码。在 Windows 上，python 的 multiprocessing 模块将启动一个新的 python 解释器，导入您的模块，取消选中父上下文的快照，然后调用您的工作函数。问题是所有模块级代码仅通过导入执行，您实际上是在运行程序的新副本，而不是为您的工作人员构建上下文。

解决方案有两个。首先，将所有模块级代码移动到函数中。您希望能够在没有副作用的情况下导入您的模块。其次，调用从条件启动程序的函数

def main():
    the stuff you were doing a module level

if __name__ == "__main__":
    main()

这工作的原因在于模块名称。当您运行 python 的顶级脚本（例如，python main.py）时，它是一个名为"__main__" 的脚本，而不是一个模块。如果一个不同的程序导入main，它是一个名为"main" 的模块（或任何你命名的脚本）。如果主代码由其他一些 python 代码（例如多处理模块）导入，则“if”会阻止您的主代码执行。

在模块级别有一些可执行代码是可以的，尤其是在您设置默认值等时。但是不要在模块级别做任何你不想在其他代码导入你的脚本时做的事情。