Pyplot savefig 内存泄漏 - 替代方案？答案

【问题标题】：Pyplot savefig memory leak - alternative?Pyplot savefig 内存泄漏 - 替代方案？
【发布时间】：2019-06-01 15:04:56
【问题描述】：

有没有 pyplot 的替代方法可以在 python 中绘制一些图形？我有一些内存泄漏，导致我的项目无法完成。

我已经尝试了使用 plt.close()、plt.clf() 等的通常推荐的“解决方案”，无论是否使用单个 fig = plt.figure() 对象，但它完全没有做任何事情。

我也尝试过使用线程，但这是我第一次在 python 中使用它，所以我可能做错了：

import gc
import psutil
from threading import Thread
import matplotlib.pyplot as plt


def plot_func():
    plt.plot([1,2,3], [1,2,3])
    plt.savefig('plot_test.png')
    plt.close()

gc.collect()
init_mem_tot = psutil.Process().memory_info().rss / 2**20
for i in range(20):
    init_mem = psutil.Process().memory_info().rss / 2**20
    thread = Thread(target=plot_func)
    thread.start()
    thread.join()

    # tried with and without those the two following lines
    del thread
    gc.collect()
    new_mem = psutil.Process().memory_info().rss / 2**20
    print("Memory : %f, previous : %f. (%f difference)" % (new_mem, init_mem, init_mem - new_mem))

final_mem = psutil.Process().memory_info().rss / 2**20
print("Final : %f, Init : %f. (%f tot difference)" % (final_mem, init_mem_tot, final_mem - init_mem_tot))

结果给出：

Memory : 107.441406, previous : 102.687500. (4.753906 difference)
Memory : 108.578125, previous : 107.445312. (1.132812 difference)
Memory : 109.941406, previous : 108.578125. (1.363281 difference)
Memory : 111.281250, previous : 109.941406. (1.339844 difference)
Memory : 112.195312, previous : 111.281250. (0.914062 difference)
Memory : 113.281250, previous : 112.195312. (1.085938 difference)
Memory : 114.535156, previous : 113.281250. (1.253906 difference)
Memory : 115.640625, previous : 114.535156. (1.105469 difference)
Memory : 116.707031, previous : 115.640625. (1.066406 difference)
Memory : 117.843750, previous : 116.707031. (1.136719 difference)
Memory : 119.175781, previous : 117.843750. (1.332031 difference)
Memory : 120.367188, previous : 119.175781. (1.191406 difference)
Memory : 121.507812, previous : 120.367188. (1.140625 difference)
Memory : 122.660156, previous : 121.507812. (1.152344 difference)
Memory : 123.937500, previous : 122.660156. (1.277344 difference)
Memory : 124.828125, previous : 123.937500. (0.890625 difference)
Memory : 126.406250, previous : 124.832031. (1.574219 difference)
Memory : 127.671875, previous : 126.406250. (1.265625 difference)
Memory : 128.585938, previous : 127.671875. (0.914062 difference)
Memory : 129.699219, previous : 128.589844. (1.109375 difference)
Final : 129.699219, Init : 102.687500. (27.011719 tot difference)

如果这些都是解决方案，我还有其他方法可以绘制内容吗？

【问题讨论】：

所以我不确定如果你在主线程之外运行 matplotlib 会发生什么。但如果不涉及 GUI，它可能没问题。在这种情况下，您可以使用"Agg" 后端来节省内存。然后我看不出使用线程与不使用线程之间的区别，并且内存消耗仅略微增加。如果这种轻微的增加是不可避免的，我不知道。可能需要更好的内存分析来查看发生了什么。
实际上是 savefig 导致了问题。而且 1mo/plot 对我的应用程序来说太大了，因为我很快就会耗尽内存（即使增加一点也会令人沮丧，因为它根本没有理由增加）。我需要它不增加任何东西。

标签： python matplotlib memory-leaks

【解决方案1】：

如果不使用线程并且不使用 pyplot，则只会观察到非常轻微的内存消耗增加。我不知道它来自哪里，但它可能是可以忍受的。

import gc
import psutil
from matplotlib.figure import Figure


def plot_func():
    fig = Figure()
    ax = fig.add_subplot()
    ax.plot([1,2,3], [1,2,3])
    fig.savefig('plot_test.png')

init_mem_tot = psutil.Process().memory_info().rss / 2**20
for i in range(100):
    init_mem = psutil.Process().memory_info().rss / 2**20

    plot_func()
    gc.collect()
    new_mem = psutil.Process().memory_info().rss / 2**20
    print("Memory : %f, previous : %f. (%f difference)" % (new_mem, init_mem, init_mem - new_mem))

final_mem = psutil.Process().memory_info().rss / 2**20
print("Final : %f, Init : %f. (%f tot difference)" % (final_mem, init_mem_tot, final_mem - init_mem_tot))

输出：（注意第一张图的 95%）

Memory : 48.113281, previous : 44.140625. (-3.972656 difference)
Memory : 48.152344, previous : 48.113281. (-0.039062 difference)
Memory : 48.195312, previous : 48.152344. (-0.042969 difference)
Memory : 48.199219, previous : 48.195312. (-0.003906 difference)
Memory : 48.199219, previous : 48.199219. (0.000000 difference)
Memory : 48.199219, previous : 48.199219. (0.000000 difference)
Memory : 48.257812, previous : 48.199219. (-0.058594 difference)
Memory : 48.257812, previous : 48.257812. (0.000000 difference)
Memory : 48.265625, previous : 48.257812. (-0.007812 difference)
Memory : 48.265625, previous : 48.265625. (0.000000 difference)
Memory : 48.269531, previous : 48.265625. (-0.003906 difference)
Memory : 48.273438, previous : 48.269531. (-0.003906 difference)
Memory : 48.273438, previous : 48.273438. (0.000000 difference)
Memory : 48.273438, previous : 48.273438. (0.000000 difference)
Memory : 48.273438, previous : 48.273438. (0.000000 difference)
Memory : 48.273438, previous : 48.273438. (0.000000 difference)
Memory : 48.281250, previous : 48.273438. (-0.007812 difference)
Memory : 48.285156, previous : 48.281250. (-0.003906 difference)
Memory : 48.285156, previous : 48.285156. (0.000000 difference)
Memory : 48.312500, previous : 48.285156. (-0.027344 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.312500, previous : 48.312500. (0.000000 difference)
Memory : 48.316406, previous : 48.312500. (-0.003906 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.316406, previous : 48.316406. (0.000000 difference)
Memory : 48.320312, previous : 48.316406. (-0.003906 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Memory : 48.320312, previous : 48.320312. (0.000000 difference)
Final : 48.320312, Init : 44.140625. (4.179688 tot difference)

【讨论】：

您认为这种没有pyplot 的方法普遍可行吗？
@gboffi 我不知道。需要使用更好的内存分析工具来找出剩余部分。
很抱歉，我的问题并不清楚：我对内存使用不太感兴趣，而是我想了解，从您知情的 POV 来看，使用 pyplot 命名空间是否会增加真实的方便用户/程序员。我找到了（高级）教程，其中 plt 的唯一用途是 plt.subplots() 和 plt.show()...
@gboffi 好的，这似乎是一个完全不同的问题。当然 pyplot 很有用，因为它允许创建一个图形，该图形稍后将在交互式 GUI 中精确地显示在一行中 (plt.figure())。但是如果你只想保存一个人物的图片，它会增加很多开销。
首先，谢谢。让我们看看我是否理解：fig = matplotlib.figure.Figure() 本质上是非交互式的，pyplot.figure() 引入的开销（大部分）与提供用户交互所需的复杂机制有关。