【问题标题】:What's the difference between importing a module inside `if __name__ == '__main__'` or outside with OpenMPI?在 `if __name__ == '__main__'` 内或在 OpenMPI 外导入模块有什么区别?
【发布时间】:2022-01-12 23:31:20
【问题描述】:

我有两个可执行的 Python3 脚本,如下所示。

脚本1:

#/usr/bin/env python
import sys
from mymodule import launch

if __name__ == '__main__':
    sys.exit(launch())

和

脚本2:

#/usr/bin/env python
import sys

if __name__ == '__main__':
    from mymodule import launch
    sys.exit(launch())

我对 CPython 的理解是,这两个脚本应该是一样的。

但是,如果我尝试使用 OpenMPI 的 mpirun 运行 ./script1 脚本,如下所示:

$ mpirun -np 2 ./script1

它失败(在 Debian linux 上)并出现以下错误:

--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  getting local rank failed
  --> Returned value No permission (-17) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  orte_ess_init failed
  --> Returned value No permission (-17) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "No permission" (-17) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[3018108860e1:05983] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

但如果我跑:

$ mpirun -np 2 ./script2

它运行没有错误。

我希望能给您一个独立的简短示例,说明 mymodule 模块和重现此错误的 go 函数。我试图将我的大型生产代码减少到重现问题的小代码,但每次我尝试将生产代码缩减为小代码时,错误是不可重现的。所以,也就是说,我不是要求大家告诉我我的错误在哪里。

相反,我想知道你们中是否有人有任何想法关于为什么两个启动脚本可能表现不同?

【问题讨论】:

    标签: python openmpi


    【解决方案1】:

    在if __name__ == "__main__" 保护下导入一个模块表示该模块中的任何其他代码都没有使用它。也就是说,如果您将模块作为库导入到其他脚本,它会在没有导入的情况下按预期工作。

    某些模块可能需要很长时间才能导入,因此除非必要,否则最好避免导入它们。

    【讨论】:

    • 感谢@kindal 的回复。我想我理解你的意思,但我对这个特定用例的理解是,如果从命令行执行,两个脚本(是否在 if __name__ == '__main__' 保护下导入模块)应该产生相同的结果。所以,我仍然不明白为什么他们甚至可以产生不同的结果。
    • 守卫下的代码使用了导入的库。例如,它可能使用argparse 解析命令行参数,为调用其他函数做准备。如果导入到其他脚本,它不会这样做,因此不需要argparse。
    • 是的。谢谢@kindal,但这仍然不能解释为什么上面的两个示例在从命令行执行时表现不同。在这两种情况下,if __name__ == '__main__' 块的内容都会被执行。因此,两个脚本中执行的行是相同的。唯一的区别是 import 语句发生在if-block 的范围之内或之外,但我不明白为什么会改变行为。
    猜你喜欢
    • 2018-11-22
    相关资源
    最近更新 更多