【问题标题】:How to list .so files that are loaded on python script run?如何列出在 python 脚本运行时加载的 .so 文件?
【发布时间】:2019-09-27 12:10:27
【问题描述】:

如何列出所有在 python 脚本运行时加载的 .so 文件(完整路径)?

是否可以获得(例如)libcudart.so.10.1 的完整路径? 是否可以获得已加载的 .so(完整路径)列表?

例如我正在跑步:

python -c "import tensorflow as tf; tf.test.is_gpu_available()"

输出:

2019-09-27 15:02:27.186029: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2019-09-27 15:02:27.657901: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: TITAN RTX major: 7 minor: 5 memoryClockRate(GHz): 1.77
pciBusID: 0000:17:00.0
2019-09-27 15:02:27.658580: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 1 with properties:
name: TITAN RTX major: 7 minor: 5 memoryClockRate(GHz): 1.77
pciBusID: 0000:65:00.0
2019-09-27 15:02:27.658766: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.1
2019-09-27 15:02:27.659868: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10
2019-09-27 15:02:27.661073: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10
2019-09-27 15:02:27.661305: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10
2019-09-27 15:02:27.662477: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10
2019-09-27 15:02:27.663054: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10
2019-09-27 15:02:27.665455: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-09-27 15:02:27.667986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0, 1
2019-09-27 15:02:27.668031: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.1
2019-09-27 15:02:27.669523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-09-27 15:02:27.669535: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0 1
2019-09-27 15:02:27.669542: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N N
2019-09-27 15:02:27.669546: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 1:   N N
2019-09-27 15:02:27.674204: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:0 with 22845 MB memory) -> physical GPU (device: 0, name: TITAN RTX, pci bus id: 0000:17:00.0, compute capability: 7.5)
2019-09-27 15:02:27.675838: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:1 with 22823 MB memory) -> physical GPU (device: 1, name: TITAN RTX, pci bus id: 0000:65:00.0, compute capability: 7.5)

【问题讨论】:

    标签: python linux tensorflow shared-libraries


    【解决方案1】:

    在 Linux 下运行时,根据您的环境,您可以使用以下替代方法之一获取信息:

    1. ld.so 调试标志
    2. 追踪

    使用 ld.so 调试标志,您可以通过在运行 python 代码之前将 LD_DEBUG 环境变量设置为“libs”(甚至是“all”)来激活调试模式(取决于您的 shell:export LD_DEBUG=libs,或setenv LD_DEBUG 库)。动态加载的库搜索路径将显示到 stderr。

    查看 ld.so 手册页了解其他选项。

    示例输出 - 来自 LD_DEBUG=libs(使用 python 'import json')

    owner@vm1:~/Project/stackoverflow/58152351$ python
         28796: find library=libc.so.6 [0]; searching
         28796:  search cache=/etc/ld.so.cache
         28796:   trying file=/lib/x86_64-linux-gnu/libc.so.6
         28796: 
         28796: find library=libpthread.so.0 [0]; searching
         28796:  search cache=/etc/ld.so.cache
         28796:   trying file=/lib/x86_64-linux-gnu/libpthread.so.0
         28796: 
         28796: find library=libdl.so.2 [0]; searching
         28796:  search cache=/etc/ld.so.cache
         28796:   trying file=/lib/x86_64-linux-gnu/libdl.so.2
    ...
    >>> import json
         28796: 
         28796: calling init: /usr/lib/python2.7/lib-dynload/_json.x86_64-linux-gnu.so
         28796: 
    

    作为替代方案,可以使用 strace,但解析它的输出需要付出更多的努力。 'stat' 行可以提供有关每个库的搜索路径的信息。

    【讨论】:

    • .so 的实际负载是否与calling init 一致?
    • 似乎我也不能只 grep 与 cuda 相关的库,例如:LD_DEBUG=libs python -c "import tensorflow as tf; tf.test.is_gpu_available()" | grep cuda
    • 请注意,LD_DEBUG 的输出会出现标准错误。 grep 的简单方法是将 stderr 放入文件(python ... 2>/tmp/myerr.out),然后在 /tmp/myerr.out 中 grep 所需的库
    • 类似:CUDA_VISIBLE_DEVICES='1' LD_DEBUG=libs python -c "import tensorflow as tf; tf.test.is_gpu_available()" 2>&1 | grep cuda
    猜你喜欢
    • 2018-08-01
    • 1970-01-01
    • 1970-01-01
    • 2019-06-09
    • 2020-10-05
    • 1970-01-01
    • 2014-03-23
    • 2018-09-08
    • 1970-01-01
    相关资源
    最近更新 更多