Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions

The left part has two residual blocks with different dilation factors. The right part is a standard convolutional layer with batch normalization [30]. The final output of the convolutional layers is the sum of the separate output of the two parts.

Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions

 

 


Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions

扩张卷积与普通的卷积相比,除了卷积核的大小以外,还有一个扩张率(dilation rate)参数,主要用来表示扩张的大小。扩张卷积与普通卷积的相同点在于,卷积核的大小是一样的,在神经网络中即参数数量不变,区别在于扩张卷积具有更大的感受野。感受野是卷积核在图像上看到的大小,例如3×3卷积核的感受野大小为9。

(a) 普通卷积,1-dilated convolution,卷积核的感受野为3×3=9。 
(b) 扩张卷积,2-dilated convolution,卷积核的感受野为7×7=49。 
(c) 扩张卷积,4-dilated convolution,卷积核的感受野为15×15=225。

扩展卷积在保持参数个数不变的情况下增大了卷积核的感受野,同时它可以保证输出的特征映射(feature map)的大小保持不变。一个扩张率为2的3×3卷积核,感受野与5×5的卷积核相同,但参数数量仅为9个,是5×5卷积参数数量的36%。

dilated的好处是不做pooling损失信息的情况下,加大了感受野,让每个卷积输出都包含较大范围的信息。在图像需要全局信息或者语音文本需要较长的sequence信息依赖的问题中,都能很好的应用dilated conv。

作者:谭旭
链接:https://www.zhihu.com/question/54149221/answer/192025860
来源:知乎
著作权归作者所有。商业转载请联系作者获得授权,非商业转载请注明出处。
 

deconv的其中一个用途是做upsampling,即增大图像尺寸。而dilated conv并不是做upsampling,而是增大感受野。

可以形象的做个解释:

对于标准的k*k卷积操作,stride为s,分三种情况:

(1) s>1,即卷积的同时做了downsampling,卷积后图像尺寸减小;

(2) s=1,普通的步长为1的卷积,比如在tensorflow中设置padding=SAME的话,卷积的图像输入和输出有相同的尺寸大小;

(3) 0<s<1,fractionally strided convolution,相当于对图像做upsampling。比如s=0.5时,意味着在图像每个像素之间padding一个空白的像素后,stride改为1做卷积,得到的feature map尺寸增大一倍。

而dilated conv不是在像素之间padding空白的像素,而是在已有的像素上,skip掉一些像素,或者输入不变,对conv的kernel参数中插一些0的weight,达到一次卷积看到的空间范围变大的目的。

当然将普通的卷积stride步长设为大于1,也会达到增加感受野的效果,但是stride大于1就会导致downsampling,图像尺寸变小。

参考:https://www.cnblogs.com/a-little-v/p/9300703.html


标准conv:

Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions

扩张conv:

Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions

参考:https://www.leiphone.com/news/201904/45AOuBGmDaPSI27C.html

相关文章: