matplotlib.animation 的 Python 内存问题

Question

我正在创建一个复杂系统中的扩散模拟，该系统将任意图像作为基底，允许任意创建扩散前沿，并允许表面反应以及新材料在起始基底上的沉积。到目前为止，我对结果感到非常自豪，您可以在这里查看我用它制作的关于 CVD 和 SFD 沉积在粒子上的电影。

CVD Movie

SFD Movie

不幸的是，我无法生成超过 50 个左右的帧，因为它内存不足。我已经尝试在整个模拟过程中尽可能多地清除东西，但我认为我一定遗漏了一些东西。总结一下：

我从创建一个空列表开始

ims = []

然后，每次我的“模拟”运行时，如果frame number % frame "rate" == 0，它会生成一个帧：

• 使用plt.ion() 到plt.draw() 和显示
• 使用ims.append() 将渲染图添加到动画帧数组中。

在每一帧渲染之前，我都会运行 plt.clf() 以防止该图仅具有越来越多的重叠图。

如果没有 ims.append() 步骤，代码会消耗 140 到 170MB 的 RAM。通过该步骤，50 帧会消耗近 1.4GB 的 RAM。显然，这是非常有限的。 50 帧很好，但我真的希望至少 350。这条路线可能不可能，但这表明内存使用纯粹由 ims 数组大约每帧 24MB。

一种解决方法是创建框架并将其渲染到循环内的.svg 或.png 文件并将其保存到磁盘。我发现这个渲染过程非常占用 CPU，所以这样做通常会使代码变得很慢。此外，创建 350 个 PNG 文件然后手动将它们转换为视频非常麻烦，所以我很想以某种方式将它们全部放入程序本身。

有没有人知道如何减少此示例代码的内存使用量而不诉诸渲染并将每个帧写入磁盘？

在这个玩具代码中，我只是使用随机数生成器来填充 cmets 中描述的两个数据集以加快速度。

代码：

import matplotlib.pyplot as plt
import matplotlib.animation as anim
from numpy import *
from matplotlib import *
import time

# Defines the number of frames of animation to render.
outputframes = 50

# Defines the size of the canned simulation.
nx = 800
ny = 800

# Defines the number of actual simulation timesteps
nt = 100

# This gets the number of timesteps between outputframes.
framestep = 2

# For reporting.
framenum = 0

# Creates two steps, one for the stepped simulated step,
# and one for the prior state. There are two independently
# changing materials, each of which will have half the simulation
# space containing random values here, plus 10% overlap in the
# middle.

p1 = zeros((nx, ny, 2))
p1[360:800,:,0] = random.rand(440, ny)
p2 = zeros((nx, ny, 2))
p2[0:440,:,0] = random.rand(440, ny)

# Animation colormap setup
norm = colors.Normalize(vmin=0, vmax = 1)
# And sets up two corresponding colormaps, one blue and one
# red for p1 and p2 respectively (goal is overlaid).
cmap1 = cm.Blues
cmap2 = cm.Reds

# Sets up an empty array to hold animation frames.
ims = []

# Sets up and uses ion to draw the figure without blocking.
plt.ion()
fig = plt.figure()
plt.draw()

# Run the simulation.

for t in range(nt):
    # This looks to see how far we are, and if we're at a point
    # where t is an even multiple of framestep, we should render
    # a new frame.

    if (t%framestep == 0):
        print('Frame ' + str(framenum))
        framenum = framenum + 1
        plt.clf()
        # In here I did a bunch of stuff to get special colors in
        # the colormap to get substrates and surfaces and other
        # features clearly identified. I am creating a new frame1
        # and frame2 object because in reality I will be doing a
        # log plot math to convert to the graphic frame.
        frame1 = p1[:,:,0]

        # This part is necessary in my real program because
        # I manually modify the colormap after it's created
        # to include the above mentioned special colors.
        frame1_colors = cmap1(norm(frame1))

        # This is my (not quite right) attempt to do overlaid plots.
        plt.imshow(frame1_colors, alpha = 0.5)

        # Do the same for the second set of data.
        frame2 = p2[:,:,0]
        frame2_colors = cmap2(norm(frame2))

        # The goal here was to take the combined output and make
        # it into an animation frame to append to ims, the image
        # array.

        # This is where I start to run into problems. Without the
        # ims.append, the program has constant memory usage. With
        # it, I am using 1340MB by the 50th frame. This is the
        # biggest issue. Even throwing away all other simulation
        # data, this image array for animation is *enormous*.

        # With the ims.append line replaced with the plt.imshow
        # line alone, memory usage is much smaller, ranging from
        # 140-170MB depending on execution point, but relatively
        # constant.

        ims.append([plt.imshow(frame2_colors, alpha = 0.5)])
#        plt.imshow(frame2_colors, alpha = 0.5)

        # Then try to draw updating animation to show progress
        # using draw(). As best I can tell, this basically works,
        # in that the plot is displaying with all components.
        plt.draw()


    # I'll put in a timer so that this doesn't go too fast, since
    # the actual calculation is very complex.
    time.sleep(0.01)

    # Proxy for the actual calculation. Just overwrite with new
    # random data in the overlapping ranges to show some change
    # visually.
    p1[360:800,:,1] = random.rand(440, ny)
    p2[0:440,:,1] = random.rand(440, ny)

    # In this version, it is trivial, but in the real simulation
    # p1[:,:,1] does not end up equal to p1[:,:,0], so the following
    # resets the simulation for the next timestep, overwriting the
    # old values to avoid memory overflow from the p1 and p2 arrays
    # being enormous.

    # Copy new values into old values.
    p1[:,:,0] = p1[:,:,1]
    p2[:,:,0] = p2[:,:,1]

# This is just a repeat for the final frame.
plt.clf()
frame1 = p1[:,:,0]
frame1_colors = cmap1(norm(frame1))
plt.imshow(frame1_colors, alpha = 0.5)
frame2 = p2[:,:,0]
frame2_colors = cmap2(norm(frame2))

# As above, the ims.append uses tons of memory, the imshow alone works well.
ims.append([plt.imshow(frame2_colors, alpha = 0.5)])
# plt.imshow(frame2_colors, alpha = 0.5)

plt.draw()

anim = anim.ArtistAnimation(fig, ims, blit=True)
anim.save('test.mp4', fps=10, writer='avconv')

Answer 1

最后，我决定这样做的唯一合理方法是将我需要的每一帧渲染为 .png，然后使用 avconv 从图像生成电影。

感谢所有建议，但由于未压缩图像的 RAM 使用情况，这似乎只是一个限制。