【WSI】《Faster R-CNN-Based Glomerular Detection in Multistained Human Whole Slide Images》

MDPI-2018

the size of the WSI is relatively large（long side exceeding 220,000 pixels）
although the glomerulus is a spherical tissue（圆形的组织）,varies in size and shape（slicing angle and pressure when obtaining thin samples），not only normal glomeruli but also many abnormal glomeruli，the texture and the internal structure of the glomeruli are more variable（make it difficult to detect）
should be robust against variations caused by differences in staining（染色）

2 Advantages

CNN-based approaches （Inception-ResNet instead of VGG-16，compare to hand-craft feature）
feeds a fixed-sized window to Faster –CNN（with multiple anchor boxes with different scales（4） and aspect ratios（3））
set a large-sized window（1099*1099） to include multiple glomeruli and sufficient background（make use of Faster RCNN）
use of a large-sized sliding window reduces the number of scans

In particular, detection of objects of interest or lesions（病灶） in high-resolution digital pathological images is a key part of diagnosis and is one of the most labor-intensive tasks for pathologists

3 Previous Work

Hand-crafted features

R-HOG（rectangle）
S-HOG（segmental）
LBP（local binary patterns）

与CNN对比

[24] Gallego, J.; Pedraza, A.; Lopez, S.; Steiner, G.; Gonzalez, L.; Laurinavicius, A.; Bueno, G. Glomerulus
Classification and Detection Based on Convolutional Neural Networks. J. Imaging 2018, 4, 20.

4 Materials and Methods

4.1 Datasets

A total of 800 renal biopsy（肾活检） microscope slides collected at the University of Tokyo hospital during 2010 to 2017.
Max image size 225000*109000（0.23 $μ m$ / pixels）
Down sampled 5*magnification 后（1.82 $μ m$ / pixels）
- Average 16733*4005
- Maximum 28160*13728
- Minimum 1536*1248
Four kinds of stain
- PAS (periodic acid‐Schiff)
  PAS is widely used for glomerular observation and is mainly used for evaluating the mesangial region by highlighting
  the glycoprotein in red.
- PAM (periodic acid‐methenamine silver)
  PAM stains collagen darkly and is used mainly for evaluating the basement
  membrane of the glomerulus.
- MT(Masson trichrome)
- Azan
  MT and Azan are used mainly to evaluate sclerosis or fibrosis of the glomerulus by making these regions blue and red, respectively.

4.2 Faster R-CNN

作者的使用
Slide windows + Faster R-CNN
- Anchor：4 kinds of scales and 3 kinds of aspect ratios
- Backbone：VGG换成 Inception v4
- Training：window size to 1099 * 1099 pixels (2000 * 2000 $μ m$ )，以目标为中心，split 缩小5倍的图
- Evaluation：slide windows（1099*1099），overlap 为 10%，merge bounding boxes overlapping by 35%
- Evalutaion metrics：F1-score

4.3 Experiments settings

introduction validation sets（train：validation：test，6：2：2）
fivefold cross-validation
WSI-wise splitting not image-wise（同一个WSI split出来的图片只能在测试集或者训练集，而不是把所有 WSI split 之后按图片分数据集）
vertical and horizontal flip

5 Results

5.1 Experiments

yellow frames：annotations
red bounding frames：proposed by Faster R-CNN model

The lower performance of MT and Azan than of PAS and PAM may be due to increased variation in glomerular appearance due to emphasis on glomerular sclerosis（肾小球硬化）.

不同数据集（数量不同）四种 stain 上的 performance 对比

5.2 Post-Evaluation

yellow frames：annotations
red bounding frames：proposed by Faster R-CNN model

Notion：

TP（True Positive） is the case in which the proposed bounding box sufficiently overlaps with the ground truth.
FP（False Positive） is the case in which the proposed bounding box overlaps with the ground truth insufficiently. （误检，IoU< thresold）
FN（False Negative） is the ground truth that could not be detected. （漏检）

To determine whether the proposed bounding box overlaps with the ground truth sufficiently, we used intersection over union (IoU), which is an evaluation metric used to measure the accuracy of detection of an object.

a）、b）、c）好理解，d）是annotations出问题的情况（漏标），e）是annotation 错标

6 Discussion

Speed：our model takes 64 s only on average to extract the glomeruli from a WSI with the trained network（Average 16733*4005，slide windows 1099*1099，大约60多张 per WSI，每张 1s 左右）
Several methods to improve detection accuracy would be considered：
- correcting the inconsistent data
- 把 FP（容易把 renal tubules——肾小管误检为肾小球，都是圆形的）中的 renal tubules 当成目标，把二分类（glomerulus、background）问题转化为三分类问题（glomerulus、background、renal tubules）
- 把 MT 和 Azan stain中的 sclerosed glomeruli 当成一类

代码

In our experiment, we used the Tensorflow Object Detection API which is an open source object
detection framework that is licensed under Apache License 2.0. An overview and usage of
the Tensorflow Object Detection API is described in the following URL:
(https://github.com/tensorflow/models/blob/master/research/object_detection/README.md)
We also used a pre-trained model of Faster R-CNN with Inception-ResNet which had been
trained on the COCO dataset. This pre-trained model can be downloaded from the following
URL:
(http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017.tar.gz)
To facilitate further research to build upon our results, the source code, network configurations,
and the trained network-derived results are available at the following URLs. By using these
materials, it is possible to perform glomerular detection on WSIs. We also provided a fewWSIs and annotations to validate them:
(https://github.com/jinseikenai/glomeruli_detection/blob/master/README.md; https://github.com/jinseikenai/glomeruli_detection/blob/master/config/glomerulus_model.config)

参考

【1】深度学习目标检测模型全面综述：Faster R-CNN、R-FCN和SSD