【发布时间】：2016-09-13 12:08:18
【问题描述】：
我按照tutorial 将我的图量化为 8 位。我不能在这里分享确切的图，但我可以说这是一个简单的卷积神经网络。
当我在原始网络和量化网络上运行 benchmark tool 时，很明显量化网络要慢得多（100 毫秒对 4.5 毫秒）。
原始网络中最慢的节点：
time average [ms] [%] [cdf%] [Op] [Name]
1.198 26.54% 26.54% MatMul fc10/fc10/MatMul
0.337 7.47% 34.02% Conv2D conv2/Conv2D
0.332 7.36% 41.37% Conv2D conv4/Conv2D
0.323 7.15% 48.53% Conv2D conv3/Conv2D
0.322 7.14% 55.66% Conv2D conv5/Conv2D
0.310 6.86% 62.53% Conv2D conv1/Conv2D
0.118 2.61% 65.13% Conv2D conv2_1/Conv2D
0.105 2.32% 67.45% MaxPool pool1
量化网络中最慢的节点：
time average [ms] [%] [cdf%] [Op] [Name]
8.289 47.67% 47.67% QuantizedMatMul fc10/fc10/MatMul_eightbit_quantized_bias_add
5.398 5.33% 53.00% QuantizedConv2D conv5/Conv2D_eightbit_quantized_conv
5.248 5.18% 58.18% QuantizedConv2D conv4/Conv2D_eightbit_quantized_conv
4.981 4.92% 63.10% QuantizedConv2D conv2/Conv2D_eightbit_quantized_conv
4.908 4.85% 67.95% QuantizedConv2D conv3/Conv2D_eightbit_quantized_conv
3.167 3.13% 71.07% QuantizedConv2D conv5_1/Conv2D_eightbit_quantized_conv
3.049 3.01% 74.08% QuantizedConv2D conv4_1/Conv2D_eightbit_quantized_conv
2.973 2.94% 77.02% QuantizedMatMul fc11/MatMul_eightbit_quantized_bias_add
这是什么原因？ 我使用的是从源代码编译的 tensorflow 版本，没有 gpu 支持。
【问题讨论】：
-
你在 GPU 上运行吗？如果是，浮点图将被放置在 GPU 上，从而加快速度，但量化操作目前没有 GPU 实现，因此它们将被放置在 CPU 上，从而导致速度减慢。也许看看您的操作位置并告诉我们？