【问题标题】:Bug? Increasing memory usage per iteration when using findContours()漏洞?使用 findContours() 时增加每次迭代的内存使用量
【发布时间】:2019-09-06 09:37:19
【问题描述】:

我已经尝试调试了一个月,确定这是我的编程习惯不好,但我认为这可能是一个错误,所以我在报告之前先在这里询问。

考虑以下代码:

#include <sys/resource.h> // memory management.
#include <stdio.h>
#include <iostream>
#include <iomanip>

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/background_segm.hpp"

using namespace std;
using namespace cv;

// Load frame from disk.
void readFrame(int frameNum, Mat &frame) {
    // Construct filenames
    Mat image;
    stringstream number, filename;

    number << setw(7) << setfill('0') << frameNum; // expecting over 1e10 images over the installation period.
    filename << "../images/store-" << number.str() << ".jpg"; // assumes jpegs!//
    cout << "Loading filename: " << filename.str() << endl;

    image = imread( filename.str() );

    if (image.empty() or !image.data) {
        cout << "Input image empty:\n";
    }

    frame = image.clone();
}

// Class to hold the perceptual chunks.
class percepUnit {

    public:
        cv::Mat image; // percept itself
        cv::Mat mask; // alpha channel

        // constructor method
        percepUnit(cv::Mat &ROI, cv::Mat &alpha, int ix, int iy, int iw, int ih, int area)  {
            image = ROI.clone();
            mask = alpha.clone();
        }
};

// Segment foreground from background
void segmentForeground(list<percepUnit*> &percepUnitsForeground, Mat &foreground, Mat &frame) {
    Mat contourImage = Mat(foreground.rows, foreground.cols, CV_8UC1, Scalar::all(0));
    vector<vector<Point>> contours;
    int area;

    // The following causes strange spikes in memory usage:
    // find contours
    findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);

    for (int idx = 0; idx < contours.size(); idx++) {

        area = contourArea(contours[idx]);

        if (area > 100) {

            percepUnit *thisUnit = new percepUnit(frame, contourImage, 0, 0, 100,100, area);
            percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
        }
    }

    /* The following does not:
    findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);

    for (int idx = 0; idx < contours.size(); idx++) {
        area = contourArea(contours[idx]);
    }*/

    /* Neither does this:
    for (int idx = 0; idx < 10; idx++) {
        percepUnit *thisUnit = new percepUnit(frame, contourImage, 0, 0, 100,100, area);
        percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
    }*/
}

int main(int argc, const char** argv)
{
    int frameCount = 78298; 
    Mat frame, foreground;
    BackgroundSubtractorMOG2 MOG2model;
    list<percepUnit*> scratchPercepUnitsForeground;

    // add rusage stuff
    struct rusage usage; // memory usage.

    for(int i=0; i<= 75; i++)
    {
        // run full segmenter here.  (background disabled)

        readFrame(frameCount, frame); // was frame = readFrame();

        // Only process if this frame actually loaded (non empty)
        if ( not frame.empty() ) {

            MOG2model(frame,foreground); // Update MOG2 model, downscale?

            // before we segment again clear scratch
            // TODO how to delete the actual memory allocated? Run delete on everything?
            for (list<percepUnit*>::iterator percepIter = scratchPercepUnitsForeground.begin(); 
                 percepIter != scratchPercepUnitsForeground.end();
                 percepIter++) {

                delete *percepIter; // delete what we point to.
                //percepIter = scratchPercepUnitsForeground.erase(percepIter); // remove the pointer itself, and update the iterator.
            }
            // Added with EDIT1
            scratchPercepUnitsForeground.clear();

            // Segment the foreground regions and generate boolImage to extract from background.
            segmentForeground(scratchPercepUnitsForeground, foreground, frame);

        }

        frameCount++;

        getrusage(RUSAGE_SELF, &usage);
        cout << "DEBUG leakTest_bug_report " << i << " " << usage.ru_maxrss/1024.0 << endl;
    }

    return 0;
}

如果你使用这里提供的图像(http://www.ekran.org/tmp/images.tar.gz),你会发现程序的内存使用量增加了,而且它似乎随着前景轮廓的数量而增加。由于我正在为每一帧清除我的存储空间 (scratchPercepUnitsForeground),我不明白为什么应该增加内存使用量。 segmentForeground() 函数应该退出,为每一帧释放所有使用的内存。内存使用量应该随着时间的推移保持不变,因为我们只在函数退出后检查内存使用量。似乎有些东西我无法弄清楚。

如果我只运行 findContours() 部分而没有 percepUnit() 构造函数,则内存使用量是不变的,正如我所料。如果我只运行不带 findContours() 的 percepUnit() 构造函数,则内存使用量是恒定的。仅当我同时使用两者时,内存使用量才会增加。请参阅上面的 segmentForeground() 中的注释代码。

我已经在我的两台机器(AMD64、linux)上确认了这个问题,并运行 opencv 2.4.6.1 和 2.4.5。

编辑1

上面的代码已更改为包含以下建议,但问题仍然存在。

这是内存增加的样子:


(来源:ekran.org

红线是调用 findContours() 和构造函数时看到的内存增加( 与上面链接的测试图像相关)。下面的稳定行是我们运行 findContours() 或构造函数的两种情况。

Valgrind 输出

==2055== Memcheck, a memory error detector
==2055== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==2055== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==2055== Command: ./leakTest
==2055== 
==2055== 
==2055== HEAP SUMMARY:
==2055==     in use at exit: 217,751,704 bytes in 112 blocks
==2055==   total heap usage: 800,066 allocs, 799,954 frees, 29,269,767,865 bytes allocated
==2055== 
==2055== 568 bytes in 1 blocks are still reachable in loss record 1 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x63A720A: __fopen_internal (iofopen.c:76)
==2055==    by 0xA8BC050: libjpeg_general_init (in /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2)
==2055==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==2055==    by 0x400F3DE: _dl_init (dl-init.c:52)
==2055==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==2055== 
==2055== 2,072 bytes in 1 blocks are still reachable in loss record 2 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x1495E4AE: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x14950888: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==2055==    by 0x400F3DE: _dl_init (dl-init.c:52)
==2055==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==2055== 
==2055== 2,072 bytes in 1 blocks are still reachable in loss record 3 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x1495E0EF: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x14950890: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==2055==    by 0x400F3DE: _dl_init (dl-init.c:52)
==2055==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==2055== 
==2055== 2,072 bytes in 1 blocks are still reachable in loss record 4 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x14971A6F: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x14950898: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==2055==    by 0x400F3DE: _dl_init (dl-init.c:52)
==2055==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==2055== 
==2055== 2,072 bytes in 1 blocks are still reachable in loss record 5 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x1499024F: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x149508A0: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==2055==    by 0x400F3DE: _dl_init (dl-init.c:52)
==2055==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==2055== 
==2055== 2,072 bytes in 1 blocks are still reachable in loss record 6 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x149610EF: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==2055==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==2055==    by 0x400F3DE: _dl_init (dl-init.c:52)
==2055==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==2055== 
==2055== 4,096 bytes in 1 blocks are still reachable in loss record 7 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0xA8BC067: libjpeg_general_init (in /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2)
==2055==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==2055==    by 0x400F3DE: _dl_init (dl-init.c:52)
==2055==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==2055== 
==2055== 1,555,228 bytes in 1 blocks are possibly lost in loss record 8 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x4E87A90: cv::fastMalloc(unsigned long) (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4ECDBF1: cv::Mat::create(int, int const*, int) (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4ECE378: cv::_OutputArray::create(int, int, int, int, bool, int) const (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4F52F7D: cv::Mat::copyTo(cv::_OutputArray const&) const (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x40253C: cv::Mat::clone() const (mat.hpp:335)
==2055==    by 0x4028D5: percepUnit::percepUnit(cv::Mat&, cv::Mat&, int, int, int, int, int) (leakTest.cpp:43)
==2055==    by 0x401E0E: segmentForeground(std::list<percepUnit*, std::allocator<percepUnit*> >&, cv::Mat&, cv::Mat&) (leakTest.cpp:63)
==2055==    by 0x40202D: main (leakTest.cpp:114)
==2055== 
==2055== 37,325,024 bytes in 8 blocks are possibly lost in loss record 9 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x4E87A90: cv::fastMalloc(unsigned long) (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4ECDBF1: cv::Mat::create(int, int const*, int) (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4ECE378: cv::_OutputArray::create(int, int, int, int, bool, int) const (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4F52F7D: cv::Mat::copyTo(cv::_OutputArray const&) const (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x40253C: cv::Mat::clone() const (mat.hpp:335)
==2055==    by 0x402897: percepUnit::percepUnit(cv::Mat&, cv::Mat&, int, int, int, int, int) (leakTest.cpp:42)
==2055==    by 0x401E0E: segmentForeground(std::list<percepUnit*, std::allocator<percepUnit*> >&, cv::Mat&, cv::Mat&) (leakTest.cpp:63)
==2055==    by 0x40202D: main (leakTest.cpp:114)
==2055== 
==2055== 52,877,752 bytes in 34 blocks are indirectly lost in loss record 10 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x4E87A90: cv::fastMalloc(unsigned long) (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4ECDBF1: cv::Mat::create(int, int const*, int) (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4ECE378: cv::_OutputArray::create(int, int, int, int, bool, int) const (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4F52F7D: cv::Mat::copyTo(cv::_OutputArray const&) const (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x40253C: cv::Mat::clone() const (mat.hpp:335)
==2055==    by 0x4028D5: percepUnit::percepUnit(cv::Mat&, cv::Mat&, int, int, int, int, int) (leakTest.cpp:43)
==2055==    by 0x401E0E: segmentForeground(std::list<percepUnit*, std::allocator<percepUnit*> >&, cv::Mat&, cv::Mat&) (leakTest.cpp:63)
==2055==    by 0x40202D: main (leakTest.cpp:114)
==2055== 
==2055== 125,971,956 bytes in 27 blocks are indirectly lost in loss record 11 of 12
==2055==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x4E87A90: cv::fastMalloc(unsigned long) (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4ECDBF1: cv::Mat::create(int, int const*, int) (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4ECE378: cv::_OutputArray::create(int, int, int, int, bool, int) const (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x4F52F7D: cv::Mat::copyTo(cv::_OutputArray const&) const (in /usr/local/lib/libopencv_core.so.2.4.5)
==2055==    by 0x40253C: cv::Mat::clone() const (mat.hpp:335)
==2055==    by 0x402897: percepUnit::percepUnit(cv::Mat&, cv::Mat&, int, int, int, int, int) (leakTest.cpp:42)
==2055==    by 0x401E0E: segmentForeground(std::list<percepUnit*, std::allocator<percepUnit*> >&, cv::Mat&, cv::Mat&) (leakTest.cpp:63)
==2055==    by 0x40202D: main (leakTest.cpp:114)
==2055== 
==2055== 178,856,428 (6,720 direct, 178,849,708 indirect) bytes in 35 blocks are definitely lost in loss record 12 of 12
==2055==    at 0x4C2B1C7: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2055==    by 0x401DD3: segmentForeground(std::list<percepUnit*, std::allocator<percepUnit*> >&, cv::Mat&, cv::Mat&) (leakTest.cpp:63)
==2055==    by 0x40202D: main (leakTest.cpp:114)
==2055== 
==2055== LEAK SUMMARY:
==2055==    definitely lost: 6,720 bytes in 35 blocks
==2055==    indirectly lost: 178,849,708 bytes in 61 blocks
==2055==      possibly lost: 38,880,252 bytes in 9 blocks
==2055==    still reachable: 15,024 bytes in 7 blocks
==2055==         suppressed: 0 bytes in 0 blocks
==2055== 
==2055== For counts of detected and suppressed errors, rerun with: -v
==2055== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)

所以基本上这似乎告诉我们在执行 clone() 的 percepUnit 构造函数中可能存在问题。在没有 findContours() 的情况下运行构造函数不会显示内存增加(如上所述),其中包括使用“new”。 jpeg 阅读器也经过单元测试,没有增加内存。因此,valgrind 的输出似乎根本没有帮助。

这应该是可重现的!在提供答案之前,请确保您可以复制它。

EDIT2(修改代码和valgrind输出,去掉指针方法)

在这里,我将列表从指针列表更改为实例列表。确认内存增加。

#include <sys/resource.h> // memory management.
#include <stdio.h>
#include <iostream>
#include <iomanip>

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/background_segm.hpp"

using namespace std;
using namespace cv;

// Load frame from disk.
void readFrame(int frameNum, Mat &frame) {
    // Construct filenames
    Mat image;
    stringstream number, filename;

    number << setw(7) << setfill('0') << frameNum; // expecting over 1e10 images over the installation period.
    filename << "../images/store-" << number.str() << ".jpg"; // assumes jpegs!//
    cout << "Loading filename: " << filename.str() << endl;

    image = imread( filename.str() );

    if (image.empty() or !image.data) {
        cout << "Input image empty:\n";
    }

    frame = image.clone();
}

// Class to hold the perceptual chunks.
class percepUnit {
    
    public:
        cv::Mat image; // percept itself
        cv::Mat mask; // alpha channel
        
        // constructor method
        percepUnit(cv::Mat &ROI, cv::Mat &alpha, int ix, int iy, int iw, int ih, int area)  {
            image = ROI.clone();
            mask = alpha.clone();
        }
};

// Segment foreground from background
void segmentForeground(list<percepUnit> &percepUnitsForeground, Mat &foreground, Mat &frame) {
    Mat contourImage = Mat(foreground.rows, foreground.cols, CV_8UC1, Scalar::all(0));
    vector<vector<Point>> contours;
    int area;

    // The following causes strange spikes in memory usage:
    // find contours
    findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);

    for (int idx = 0; idx < contours.size(); idx++) {

        area = contourArea(contours[idx]);

        if (area > 100) {

            percepUnit thisUnit = percepUnit(frame, contourImage, 0, 0, 100,100, area);
            percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
        }
    }

    /* The following does not:
    findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);

    for (int idx = 0; idx < contours.size(); idx++) {
        area = contourArea(contours[idx]);
    }*/

    /* Neither does this:
    for (int idx = 0; idx < 10; idx++) {
        percepUnit thisUnit = percepUnit(frame, contourImage, 0, 0, 100,100, area);
        percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
    }*/
}

int main(int argc, const char** argv)
{
    int frameCount = 78298; 
    Mat frame, foreground;
    BackgroundSubtractorMOG2 MOG2model;
    list<percepUnit> scratchPercepUnitsForeground;

    // add rusage stuff
    struct rusage usage; // memory usage.

    for(int i=0; i<= 75; i++)
    {
        // run full segmenter here.  (background disabled)
        
        readFrame(frameCount, frame); // was frame = readFrame();

        // Only process if this frame actually loaded (non empty)
        if ( not frame.empty() ) {

            MOG2model(frame,foreground); // Update MOG2 model, downscale?

            // before we segment again clear scratch
            scratchPercepUnitsForeground.clear();
    
            // Segment the foreground regions and generate boolImage to extract from background.
            segmentForeground(scratchPercepUnitsForeground, foreground, frame);

        }

        frameCount++;

        getrusage(RUSAGE_SELF, &usage);
        cout << "DEBUG leakTest_bug_report " << i << " " << usage.ru_maxrss/1024.0 << endl;
    }

    return 0;
}

这是对应的valgrind输出:

==3562== Memcheck, a memory error detector
==3562== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3562== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3562== Command: ./leakTest
==3562== 
==3562== 
==3562== HEAP SUMMARY:
==3562==     in use at exit: 15,024 bytes in 7 blocks
==3562==   total heap usage: 795,556 allocs, 795,549 frees, 29,269,731,785 bytes allocated
==3562== 
==3562== 568 bytes in 1 blocks are still reachable in loss record 1 of 7
==3562==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562==    by 0x63A720A: __fopen_internal (iofopen.c:76)
==3562==    by 0xA8BC050: libjpeg_general_init (in /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2)
==3562==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562==    by 0x400F3DE: _dl_init (dl-init.c:52)
==3562==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562== 
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 2 of 7
==3562==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x1495E4AE: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x14950888: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562==    by 0x400F3DE: _dl_init (dl-init.c:52)
==3562==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562== 
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 3 of 7
==3562==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x1495E0EF: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x14950890: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562==    by 0x400F3DE: _dl_init (dl-init.c:52)
==3562==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562== 
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 4 of 7
==3562==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x14971A6F: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x14950898: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562==    by 0x400F3DE: _dl_init (dl-init.c:52)
==3562==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562== 
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 5 of 7
==3562==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x1499024F: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x149508A0: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562==    by 0x400F3DE: _dl_init (dl-init.c:52)
==3562==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562== 
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 6 of 7
==3562==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562==    by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x149610EF: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562==    by 0x400F3DE: _dl_init (dl-init.c:52)
==3562==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562== 
==3562== 4,096 bytes in 1 blocks are still reachable in loss record 7 of 7
==3562==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562==    by 0xA8BC067: libjpeg_general_init (in /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2)
==3562==    by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562==    by 0x400F3DE: _dl_init (dl-init.c:52)
==3562==    by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562== 
==3562== LEAK SUMMARY:
==3562==    definitely lost: 0 bytes in 0 blocks
==3562==    indirectly lost: 0 bytes in 0 blocks
==3562==      possibly lost: 0 bytes in 0 blocks
==3562==    still reachable: 15,024 bytes in 7 blocks
==3562==         suppressed: 0 bytes in 0 blocks
==3562== 
==3562== For counts of detected and suppressed errors, rerun with: -v
==3562== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)

然而,内存仍然无法解释地增加:(与上图相比,整体增加是由于在 valgrind 中运行此测试所致。)


(来源:ekran.org

对于相同的代码,这里是地块输出:http://www.ekran.org/tmp/massif.print.leak 未调用 findContours() 的非泄漏情况的地块输出,只有 percepUnit 构造函数:http://www.ekran.org/tmp/massif.print.noLeak

EDIT3

在跨线程(http://answers.opencv.org/question/19172/bug-increasing-memory-usage-per-iteration-when/)中提示我读proc而不是使用rusage方法,看看这个,内存并没有稳步增加:(!)


(来源:ekran.org

这确实与地块输出相似!所以我想我需要重做所有的单元测试。有人有理由我不应该在这里放弃并考虑使用问题吗?

【问题讨论】:

  • 您真的应该考虑使用valgrind 来查找内存泄漏,您会看到泄漏的位置并为您节省一个月的调试时间
  • 我将在上面添加我的 valgrind 输出。我一直在使用它,但发现它没用,因为大部分输出是由确认不会泄漏的代码生成的,而且我看不出有什么办法告诉实际问题出在哪里。当然,上次我使用它时,程序要大 5 倍。上面还有更多的cmets。
  • 我可以在 Centos6 上看到这个。尝试升级到 libpixman-1.so.0.32.4 并发生相同的泄漏。来自 Valgrind:1 个块中的 2,064 字节仍然可以在 6 个中的 6 个丢失记录中达到 6 ==15199== at 0x4A069EE: malloc (vg_replace_malloc.c:270) ==15199== by 0x5BA8B7A: ??? (在 /usr/lib64/libpixman-1.so.0.32.4 中)

标签: c++ opencv


【解决方案1】:

您的 ptr-list 清理的 for 循环正在跳过项目,因为在循环体本身中有一个增量子句 (percepIter++) 和一个迭代器重新分配(来自 erase() 调用的返回值)。

换句话说,您将迭代器加倍,跳过每个偶数槽项。

我已经在下面标记了:

        for (list<percepUnit*>::iterator percepIter = scratchPercepUnitsForeground.begin(); 
             percepIter != scratchPercepUnitsForeground.end();
             percepIter++) { // ADVANCES ITERATOR

            delete *percepIter; // delete what we point to.
            percepIter = scratchPercepUnitsForeground.erase(percepIter); // ADVANCES ITERATOR
        }

您可以通过多种方式解决此问题。例如,通过从 for 循环中删除增量表达式:

       for (list<percepUnit*>::iterator percepIter = scratchPercepUnitsForeground.begin(); 
             percepIter != scratchPercepUnitsForeground.end();) {

            delete *percepIter; // delete what we point to.
            percepIter = scratchPercepUnitsForeground.erase(percepIter);
        }

同样,您可以简单地枚举列表,然后在释放所有指向的对象后调用列表clear() 方法。

       for (list<percepUnit*>::iterator percepIter = scratchPercepUnitsForeground.begin(); 
             percepIter != scratchPercepUnitsForeground.end();
             ++percepIter) 
        {
            delete *percepIter;
        }
        scratchPercepUnitsForeground.clear();

就个人而言,如果我必须选择其中一个,我更喜欢后者。它不仅速度更快,而且更易读。

但是如果我对此进行编码,我会使用智能指针,这会使这完全无关紧要,因为您可以简单地触发 scratchPercepUnitsForeground.clear(); 并完成它。当列表内容被清除时,所有的智能指针析构函数都会触发,然后它们会为你删除它们的对象。这个概念被称为资源获取即初始化,或简称为 RAII,它只是意味着所有事物,包括动态分配,都具有基于范围的生命周期,并在范围退出时自动回收资源。 You can read more about it here.

不管怎样,这肯定有一个泄漏,而且从外观上看是一个很大的泄漏。

【讨论】:

  • 谢谢。我在这里交叉发布了这个问题:answers.opencv.org/question/19172/…
  • 查看我的 cmets 以了解类似的答案。不幸的是,这不是问题的原因。事实上,我看到的泄漏比我通过应用您建议的修复所节省的要大得多。 (参见上面的 EDIT1)。如果您可以重现我在图表中看到的内容,我们可以继续前进。
  • @b 很遗憾它没有解决你的问题,因为那张海报和我自己都非常正确地认为这是一个彻底的内存泄漏。无论您要查找的是否是泄漏,您都应该对向量进行实例级处理,使用智能指针,或者按照他/她和我本人在这里的建议修复泄漏。如果我有机会运行它,我会运行它,但由于我既没有在这个设备上安装你的测试数据也没有安装 OpenCV,“复制”它远非快速周转。
  • 非常感谢您抽出宝贵的时间。我并不着急,一个多月前我在一个更大的程序中发现了这些问题,直到现在才煞费苦心地把它变成一个我可以分享的简单测试用例。我真的束手无策,并且已经与其他一些更有经验的 C++ 程序员交谈过。其中一位建议我从旧的实例列表切换到指针! (我进行了更改以证明这不是问题)。我同意使用实例列表更简洁,但无论哪种方式,这个奇怪的问题仍然存在。
  • @b 我不想问这个问题,但是在应用我在此处显示的更改之后,Valgrind 输出(并感谢您发布)之后?我不想追逐红鲱鱼。泄漏摘要上方的最后一个是我认为在这里发现的泄漏,但我想确定一下。如果可能(并且当您有机会时),请应用此处描述的更改和您的交叉发布并重新拍摄 valgrin 会话,并在最后的 EDIT 附录中更新问题。根据那个转储,segmentForeground() 中的new 是万恶之源,这个答案至少应该解决这个问题。
【解决方案2】:

对我来说,您使用 rusage 进行的图形分析单调增加的事实令人怀疑。

rusage 的文档指出:

使用的最大驻留集大小,以千字节为单位。即进程同时使用的物理内存的最大千字节数。

由于您可能只收集多个图像的同一进程的统计信息,因此rusage 报告所有图像的最大内存使用率的结论是有道理的。

确实,使用不同的分析工具(OS X 上的仪器)会生成显示当前内存使用情况的图表:

这与您使用proc 的结果非常相似。我会得出结论,rusage 不是您想要的分析工具。

【讨论】:

    【解决方案3】:

    我在谷歌搜索似乎是同一个问题后发现了这个问题(valgrind 报告“仍然可以访问”的内存,回溯包括 _dl_init 和 libpixman-1)。我正在使用 Fedora 18 64 位。我设法用一个绝对最小的 C 可执行文件和 .so 库重现了这个问题...... C 程序只有一个返回 0 的 main 函数,而 .so 只有一个函数(从未实际调用过),它也立即退出0. 然而 valgrind 仍然报告 5 个未释放的块,总计 10,360 字节。我决定发布这个以防其他一些可怜的灵魂花了一个月的时间调试同样的问题!无论错误是什么,它似乎都不太可能是由可执行文件和库引起的,两者都不做任何事情。

    【讨论】:

    • RSS 会随着时间的推移而增加吗?
    【解决方案4】:

    经过进一步调查,当我在 Ubuntu 13.04(GCC 4.7.3,与 Fedora 18 的 GCC 4.7.2 相比)上重新编译可执行文件和.so 文件时,内存泄漏消失了。我没有做任何其他更改,所以问题绝对不在于我的代码。即使在我将其转移回 Fedora 机器之后,固定的 .so 仍然是 valgrind 完美的。我建议使用较新版本的 gcc 重新编译您正在使用的任何/所有共享对象。至于内存使用量随时间增加的问题,我不知道,因为我的项目很小,分配的内存很少,退出很快。

    【讨论】:

      猜你喜欢
      • 2020-06-18
      • 1970-01-01
      • 2018-03-14
      • 1970-01-01
      • 2011-01-18
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2011-10-14
      相关资源
      最近更新 更多