Nsight Compute 无法分析 Waveglow（PyTorch 应用程序）答案

【问题标题】：Nsight Compute can't profile Waveglow (PyTorch application)Nsight Compute 无法分析 Waveglow（PyTorch 应用程序）
【发布时间】：2021-03-10 00:33:08
【问题描述】：

我试图通过这个命令分析 https://github.com/NVIDIA/waveglow ：

nv-nsight-cu-cli --export ./nsight_output ~/.virtualenvs/waveglow/bin/python3 inference.py -f <(ls mel_spectrograms/*.pt) -w waveglow_256channels.pt -o . --is_fp16 -s 0.6

Python 命令来自 https://github.com/NVIDIA/waveglow#generate-audio-with-our-pre-existing-model 的指令，它适用于 Nsight 系统，而不是 Nsight Compute。

分析不会结束打印此日志；所以我按下了Ctrl+C。此外，它只配置一个内核，但我有更多内核。（由 Nsight 系统检查）

...
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 286: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 287: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 288: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 289: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 290: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 291: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 292: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 293: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 294: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 295: 0%....50%....100% - 48 passes
==PROF== Profiling "weight_norm_fwd_first_dim_ker..." - 296: 0%....50%...^C
==PROF== Received signal, trying to shutdown target application
 - 43 passes
==ERROR== Failed to profile kernel "weight_norm_fwd_first_dim_ker..." in process
==ERROR== An error occurred while trying to profile.
==ERROR== An error occurred while trying to profile
==PROF== Report: nsight_compute_result.nsight-cuprof-report

操作系统：CentOS Linux 7，Nsight Compute（2019.3.1，Build 26317742），显卡：Tesla V100-PCIE-32GB

我该如何解决这个问题？

【问题讨论】：

标签： pytorch nsight-compute

【解决方案1】：

我认为这里没有任何错误，该工具按预期运行。它不仅分析了一个内核，还分析了您的日志输出中已经启动的 296 个内核（它们似乎都来自一个内核函数）。

您可以控制使用例如配置文件的内核的数量或类型。 --launch-count 或 --kernel-regex 选项。您还可以使用 --metrics 和 --section 控制为每个内核收集的指标，因为收集较少的指标会降低工具的开销。

有关更多可用命令行选项，请参阅 https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html#command-line-options。

【讨论】：