理解托管分配的 !address -summary答案

【问题标题】：Making sense of !address -summary for managed allocations理解托管分配的 !address -summary
【发布时间】：2014-10-29 13:32:38
【问题描述】：

我试图了解 !address -summary 如何报告托管分配。我有以下代码来尝试测试它。我基本上改变了数组的大小，然后附加 windbg 来检查 !address -summary 的结果

static void Main(string[] args)
        {
int size = 2000;
            var something = new byte[size];
            for (int i = 0; i < something.Length; i++)
            {
                something[i] = 10;
            }
            Console.WriteLine("Attach Windbg now");
            Console.Read();

    // not sure if its needed but have it so that this object is still referenced and optimizations don't remove it proactively
            for (int i = 0; i < something.Length; i++)
            {
                Console.WriteLine(something[i]);
            }
        }

大小 = 50000

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                                     53     7fff`e1546000 ( 128.000 Tb)          100.00%
<unknown>                                61        0`18a1d000 ( 394.113 Mb)  80.32%    0.00%
Image                                   163        0`031da000 (  49.852 Mb)  10.16%    0.00%
MappedFile                               12        0`019f8000 (  25.969 Mb)   5.29%    0.00%
Stack                                    12        0`01000000 (  16.000 Mb)   3.26%    0.00%
Heap                                     22        0`002f3000 (   2.949 Mb)   0.60%    0.00%
Other                                     8        0`001bf000 (   1.746 Mb)   0.36%    0.00%
TEB                                       4        0`00008000 (  32.000 kb)   0.01%    0.00%
PEB                                       1        0`00001000 (   4.000 kb)   0.00%    0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_PRIVATE                             101        0`19d0b000 ( 413.043 Mb)  84.18%    0.00%
MEM_IMAGE                               163        0`031da000 (  49.852 Mb)  10.16%    0.00%
MEM_MAPPED                               19        0`01bc5000 (  27.770 Mb)   5.66%    0.00%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE                                 53     7fff`e1546000 ( 128.000 Tb)          100.00%
MEM_RESERVE                              51        0`1aec5000 ( 430.770 Mb)  87.79%    0.00%
MEM_COMMIT                              232        0`03be5000 (  59.895 Mb)  12.21%    0.00%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
PAGE_EXECUTE_READ                        24        0`0262d000 (  38.176 Mb)   7.78%    0.00%
PAGE_READONLY                            71        0`00dfb000 (  13.980 Mb)   2.85%    0.00%
PAGE_WRITECOPY                           31        0`00417000 (   4.090 Mb)   0.83%    0.00%
PAGE_READWRITE                           81        0`00335000 (   3.207 Mb)   0.65%    0.00%
PAGE_EXECUTE_WRITECOPY                    3        0`00043000 ( 268.000 kb)   0.05%    0.00%
PAGE_EXECUTE_READWRITE                   15        0`00016000 (  88.000 kb)   0.02%    0.00%
PAGE_READWRITE|PAGE_GUARD                 5        0`00015000 (  84.000 kb)   0.02%    0.00%
PAGE_EXECUTE                              2        0`00003000 (  12.000 kb)   0.00%    0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ----------
Free                                      0`7fff0000     7ff5`7f760000 ( 127.959 Tb)
<unknown>                                 0`02e62000        0`0ffde000 ( 255.867 Mb)
Image                                  7ffd`0ab5d000        0`010e4000 (  16.891 Mb)
MappedFile                                0`0150c000        0`01234000 (  18.203 Mb)
Stack                                     0`1ba90000        0`003f9000 (   3.973 Mb)
Heap                                      0`02741000        0`00101000 (   1.004 Mb)
Other                                     0`011b0000        0`00181000 (   1.504 Mb)
TEB                                    7ff5`ff928000        0`00002000 (   8.000 kb)
PEB                                    7ff5`ff926000        0`00001000 (   4.000 kb)

这就是它在 !dumpheap -stat 输出中的样子

00007ffd0ad95740        4        50609 System.Byte[]

这就是它在 !EEHeap -gc 输出中的样子

Number of GC Heaps: 1
generation 0 starts at 0x0000000002ec1030
generation 1 starts at 0x0000000002ec1018
generation 2 starts at 0x0000000002ec1000
ephemeral segment allocation context: ephemeral segment allocation context: none
none
 segment     begin allocated  size
0000000002ec0000  0000000002ec1000  0000000002ed3fe8  0x12fe8(77800)
Large object heap starts at 0x0000000012ec1000
 segment     begin allocated  size
0000000012ec0000  0000000012ec1000  0000000012ec9720  0x8720(34592)
Total Size:              Total Size:              Size: 0x1b708 (112392) bytesSize: 0x1b708 (112392) bytes.
.
------------------------------
------------------------------
GC Heap Size:            GC Heap Size:            Size: 0x1b708 (112392) bytesSize: 0x1b708 (112392) bytes.

大小 = 2000

0:003> !address -summary
Mapping file section regions...
Mapping module regions...
Mapping PEB regions...
Mapping TEB and stack regions...
Mapping heap regions...
Mapping page heap regions...
Mapping other regions...
Mapping stack trace database regions...
Mapping activation context regions...

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                                     50     7fff`e1941000 ( 128.000 Tb)          100.00%
<unknown>                                57        0`1861b000 ( 390.105 Mb)  80.16%    0.00%
Image                                   163        0`031da000 (  49.852 Mb)  10.24%    0.00%
MappedFile                               12        0`019f8000 (  25.969 Mb)   5.34%    0.00%
Stack                                    12        0`01000000 (  16.000 Mb)   3.29%    0.00%
Heap                                     22        0`002fa000 (   2.977 Mb)   0.61%    0.00%
Other                                     8        0`001bf000 (   1.746 Mb)   0.36%    0.00%
TEB                                       4        0`00008000 (  32.000 kb)   0.01%    0.00%
PEB                                       1        0`00001000 (   4.000 kb)   0.00%    0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_PRIVATE                              97        0`19910000 ( 409.063 Mb)  84.05%    0.00%
MEM_IMAGE                               163        0`031da000 (  49.852 Mb)  10.24%    0.00%
MEM_MAPPED                               19        0`01bc5000 (  27.770 Mb)   5.71%    0.00%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE                                 50     7fff`e1941000 ( 128.000 Tb)          100.00%
MEM_RESERVE                              50        0`1aadc000 ( 426.859 Mb)  87.71%    0.00%
MEM_COMMIT                              229        0`03bd3000 (  59.824 Mb)  12.29%    0.00%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
PAGE_EXECUTE_READ                        24        0`0262d000 (  38.176 Mb)   7.84%    0.00%
PAGE_READONLY                            71        0`00dfb000 (  13.980 Mb)   2.87%    0.00%
PAGE_WRITECOPY                           31        0`00417000 (   4.090 Mb)   0.84%    0.00%
PAGE_READWRITE                           79        0`00326000 (   3.148 Mb)   0.65%    0.00%
PAGE_EXECUTE_WRITECOPY                    3        0`00043000 ( 268.000 kb)   0.05%    0.00%
PAGE_EXECUTE_READWRITE                   15        0`00016000 (  88.000 kb)   0.02%    0.00%
PAGE_READWRITE|PAGE_GUARD                 4        0`00012000 (  72.000 kb)   0.01%    0.00%
PAGE_EXECUTE                              2        0`00003000 (  12.000 kb)   0.00%    0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ----------
Free                                      0`7fff0000     7ff5`7f7a0000 ( 127.959 Tb)
<unknown>                                 0`02912000        0`0ffee000 ( 255.930 Mb)
Image                                  7ffd`0ab5d000        0`010e4000 (  16.891 Mb)
MappedFile                                0`0102c000        0`01234000 (  18.203 Mb)
Stack                                     0`1b350000        0`003f9000 (   3.973 Mb)
Heap                                      0`022a8000        0`00101000 (   1.004 Mb)
Other                                     0`00cd0000        0`00181000 (   1.504 Mb)
TEB                                    7ff5`ff968000        0`00002000 (   8.000 kb)
PEB                                    7ff5`ff967000        0`00001000 (   4.000 kb)

这就是它在 !dumpheap -stat 输出中的样子

00007ffd0ad95740        4         2609 System.Byte[]

这里是 !EEHeap -gc 的输出

Number of GC Heaps: 1
generation 0 starts at 0x0000000002bd1030
generation 1 starts at 0x0000000002bd1018
generation 2 starts at 0x0000000002bd1000
ephemeral segment allocation context: none
 segment     begin allocated  size
0000000002bd0000  0000000002bd1000  0000000002bd7fe8  0x6fe8(28648)
Large object heap starts at 0x0000000012bd1000
 segment     begin allocated  size
0000000012bd0000  0000000012bd1000  0000000012bd9720  0x8720(34592)
Total Size:              Size: 0xf708 (63240) bytes.
------------------------------
GC Heap Size:            Size: 0xf708 (63240) bytes.

当我将数组大小更改为 5000000 时，这就是我在 !dumpheap -stat 中看到的

00007ffd0ad95740        4      5000609 System.Byte[]

这是 !EEHeap -gc 命令的输出

Number of GC Heaps: 1
generation 0 starts at 0x0000000002bd1030
generation 1 starts at 0x0000000002bd1018
generation 2 starts at 0x0000000002bd1000
ephemeral segment allocation context: none
 segment     begin allocated  size
0000000002bd0000  0000000002bd1000  0000000002bd7fe8  0x6fe8(28648)
Large object heap starts at 0x0000000012bd1000
 segment     begin allocated  size
0000000012bd0000  0000000012bd1000  000000001309e298  0x4cd298(5034648)
Total Size:              Size: 0x4d4280 (5063296) bytes.
------------------------------
GC Heap Size:            Size: 0x4d4280 (5063296) bytes.

地址汇总如下

0:003> !address -summary


Mapping file section regions...
Mapping module regions...
Mapping PEB regions...
Mapping TEB and stack regions...
Mapping heap regions...
Mapping page heap regions...
Mapping other regions...
Mapping stack trace database regions...
Mapping activation context regions...

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                                     50     7fff`e153a000 ( 128.000 Tb)          100.00%
<unknown>                                61        0`18a1d000 ( 394.113 Mb)  80.31%    0.00%
Image                                   163        0`031da000 (  49.852 Mb)  10.16%    0.00%
MappedFile                               12        0`019f8000 (  25.969 Mb)   5.29%    0.00%
Stack                                    12        0`01000000 (  16.000 Mb)   3.26%    0.00%
Heap                                     22        0`002ff000 (   2.996 Mb)   0.61%    0.00%
Other                                     8        0`001bf000 (   1.746 Mb)   0.36%    0.00%
TEB                                       4        0`00008000 (  32.000 kb)   0.01%    0.00%
PEB                                       1        0`00001000 (   4.000 kb)   0.00%    0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_PRIVATE                             101        0`19d17000 ( 413.090 Mb)  84.18%    0.00%
MEM_IMAGE                               163        0`031da000 (  49.852 Mb)  10.16%    0.00%
MEM_MAPPED                               19        0`01bc5000 (  27.770 Mb)   5.66%    0.00%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE                                 50     7fff`e153a000 ( 128.000 Tb)          100.00%
MEM_RESERVE                              51        0`1aa05000 ( 426.020 Mb)  86.82%    0.00%
MEM_COMMIT                              232        0`040b1000 (  64.691 Mb)  13.18%    0.00%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
PAGE_EXECUTE_READ                        24        0`0262d000 (  38.176 Mb)   7.78%    0.00%
PAGE_READONLY                            71        0`00e1d000 (  14.113 Mb)   2.88%    0.00%
PAGE_READWRITE                           81        0`007df000 (   7.871 Mb)   1.60%    0.00%
PAGE_WRITECOPY                           31        0`00417000 (   4.090 Mb)   0.83%    0.00%
PAGE_EXECUTE_WRITECOPY                    3        0`00043000 ( 268.000 kb)   0.05%    0.00%
PAGE_EXECUTE_READWRITE                   15        0`00016000 (  88.000 kb)   0.02%    0.00%
PAGE_READWRITE|PAGE_GUARD                 5        0`00015000 (  84.000 kb)   0.02%    0.00%
PAGE_EXECUTE                              2        0`00003000 (  12.000 kb)   0.00%    0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ----------
Free                                      0`7fff0000     7ff5`7f7d0000 ( 127.959 Tb)
<unknown>                                 0`02be2000        0`0ffee000 ( 255.930 Mb)
Image                                  7ffd`0ab5d000        0`010e4000 (  16.891 Mb)
MappedFile                                0`0134c000        0`01214000 (  18.078 Mb)
Stack                                     0`1b760000        0`003f9000 (   3.973 Mb)
Heap                                      0`0265d000        0`00101000 (   1.004 Mb)
Other                                     0`00fd0000        0`00181000 (   1.504 Mb)
TEB                                    7ff5`ff996000        0`00002000 (   8.000 kb)
PEB                                    7ff5`ff99e000        0`00001000 (   4.000 kb)

dumpheap 和 eeheap 的输出完全合理。我不明白这些托管分配应该显示在 !address -summary 中的什么位置

【问题讨论】：

标签： c# .net-4.0 heap-memory windbg sos

【解决方案1】：

请尝试以下代码，它分配了 20 MB：

    static void Main(string[] args)
    {
        Console.WriteLine("Attach Windbg now");
        Console.ReadLine();

        int size = 20000000;
        var something = new byte[size];

        Console.WriteLine("Look again");
        Console.ReadLine();

// not sure if its needed but have it so that this object is still referenced and optimizations don't remove it proactively
        something[0] = 1;
    }

一开始我的机器显示

<unknown>    49        0`19502000 ( 405.008 Mb)  82.39%    0.00%
MEM_RESERVE  45        0`1b0d4000 ( 432.828 Mb)  88.05%    0.01%
MEM_COMMIT  240        0`03ac0000 (  58.750 Mb)  11.95%    0.00%

在!address -summary的输出中。

现在做一个

.loadby sos clr    
bp kernel32!VirtualAlloc
g

然后在控制台应用程序中按 Enter。它会打断点，你会看到

0:000> k
Child-SP          RetAddr           Call Site
00000000`001ce2b8 000007fe`f77c265a KERNEL32!VirtualAlloc
...
00000000`001ce5f0 000007fe`f77c7ece clr!WKS::GCHeap::Alloc+0x348
00000000`001ce640 000007fe`981600f7 clr!JIT_NewArr1+0x49e
...

0:000> !clrstack
OS Thread Id: 0x5c4 (0)
        Child SP               IP Call Site
00000000001ce778 0000000076cb5c68 [HelperMethodFrame: 00000000001ce778] 
00000000001ce8f0 000007fe981600f7 ManagedVirtualAlloc.Program.Main(System.String[])
00000000001cec30 000007fef7829e03 [GCFrame: 00000000001cec30]

因此您可以从 Main 方法中看到 .NET 运行时尝试分配虚拟内存以适应新的字节数组（如 JIT_NewArr1 所示）。

但是，查看传递给 VirtualAlloc() 的参数（注意kb 对 64 位有误导），我们可以看到

0:000> r
...
rcx=0000000012392000 // address of an already reserved memory region
rdx=000000000130b000 // size: 19968000
 r8=0000000000001000 // MEM_COMMIT
 r9=0000000000000004 // PAGE_READWRITE
...

0:000> kb; *** Warning: misleading arguments for 64 bit!
RetAddr           : Args to Child                                                           : Call Site
000007fe`f77c265a : 00000000`00000000 00000000`00000000 00000000`001ce369 000007fe`00000000 : KERNEL32!VirtualAlloc

我们看到它指定了一个之前保留的地址，现在应该提交。调用后（输入g），我们看到一些（~20 MB）保留内存已转换为已提交内存，而<unknown> 的数量仍然相同。

<unknown>     49        0`19502000 ( 405.008 Mb)  82.39%    0.00%
MEM_RESERVE   45        0`19dc1000 ( 413.754 Mb)  84.17%    0.00%
MEM_COMMIT   238        0`04dd3000 (  77.824 Mb)  15.83%    0.00%

结论：

.NET 会提前预留大量内存。在启动应用程序后，该内存已经显示为<unused>。如果可能，它将使用保留和提交的内存，因此并非每次分配都会导致新的 VirtualAlloc() 调用。

更多内部结构：

我们还看到 .NET 框架非常聪明：它只分配了 19968000 字节而不是 20000000 字节，因为它能够重用已经处于提交状态的内存块的一些字节。看指针：

VirtualAlloc pointer: 0x0000000012392000
"something" variable: 0x0000000012389710

something 变量指向由 VirtualAlloc() 调用返回的内存块前面的内存区域。

【讨论】：

谢谢。我希望我能投票 10 次以获得非常好的解释
由于 64 位上的误导性 kb 信息，我不得不更新我的答案。但结论是一样的。

【解决方案2】：

托管分配发生在 CLR 使用 VirtualAlloc API 分配的段中。它们在“未知”部分下报告。

【讨论】：

有 50000 个未知类别显示大小为 394.113 Mb。 5000000 个未知类别仍显示大小为 394.113 Mb。那不应该涨吗？
段由 CLR 在初始化时保留。然后在分配对象时根据需要提交部分段。如果您的 5,000,000 个对象适合已分配的段，则未知类别不会增加。