文章目录
- 1.Introduction to Perception
- 2.Sebastian’s Take
- 3.Computer Vision
- 4.Camera Images
- 5.LiDAR Images
- 6.Machine Learning
- 7.Neural Network
- 8.Backpropagation
- 9.Convolutional Neural Network
- 10.Detection and Classification
- 11.Tracking
- 12.Segmentation
- 13.Apollo Perception
- 14.Sensor Data Comparisons
- 15.Perception Fusion Strategy
- 16.Vehicle Detection Project from
- 17.Summary
1.Introduction to Perception
2.Sebastian’s Take
3.Computer Vision
无人驾驶车有四个感知世界的核心任务
- Detection
- Classification
- Tracking
- Segmentation, 语义分割意味着将图像中的每个像素与语义类别进行匹配, such as road, car, or sky.
4.Camera Images
To get more intuition about how to quantify a color, you can try RGB Slider
5.LiDAR Images
激光雷达传感器创建环境的点云表征，提供了难以通过摄像头图像获得的信息。如 距离 和 高度。
激光雷达传感器使用光线，尤其是激光来测量与环境中反射该光线的物体之间的距离。激光雷达发生激光脉冲并测量物体，将每个激光脉冲反射回传感器所花费的时间，反射需要的时间越长，物体离传感器越远。
激光雷达正是通过这种方式来构建世界的视觉表征。收集的数据形成一团点或“点云”。这些点云可以告诉我们关于物体的许多信息，例如其 形状 和 表面纹理 。通过对点进行聚类和分析，这些数据提供了足够的对象检测、跟踪或分类信息。
Footnote on Lidar
There are other possibilities to scan the laser beams. Instead of rotating the lasers or having a rotating mirror, we can scan the lidar with a vibrating micromirror. Those lidars are in development but none are commercially available now (as of March 2017).
Instead of mechanically moving the laser beam, a similar principle to phased array radar can be employed. Dividing a single laser beam into multiple waveguides, the phase relationship between the waveguides can be altered and thereby the direction of the laser beam shifted. The advantage is that the form factor can be much smaller and that there are no moving parts.
Another possibility is to use the laser as a gigantic flash like with a camera and then measuring the arrival times for all the objects with one big imaging photodiode array. This is in effect a 3D camera. The components are currently very expensive and currently this is used more in space and in terrain mapping applications.
6.Machine Learning
7.Neural Network
8.Backpropagation
9.Convolutional Neural Network
10.Detection and Classification
Recommended Resources
Please click the link below to explore more about advanced CNN structures for detection and classification:
R-CNN, Fast R-CNN, Faster RCNN, yolo, SSD
11.Tracking
Recommended Resources
If you want to know more about the image features, please. Click the link below. LBP and HOG.
12.Segmentation
语义分割涉及到对图像的每个像素进行分类，它用于尽可能详细地了解环境。并确定车辆可驾驶区域。
语义分割依赖于一种特殊类型的CNN，它被称为全卷积网络或FCN
为了分割像素，输出尺寸必须与原始图像的尺寸相匹配。对中间输出进行上采样处理。
FCN paper
13.Apollo Perception
Apollo开放式软件栈可感知障碍物，交通信号灯和车道
对于三维对象检测，Apollo在高精度地图上使用感兴趣区域(ROI)来重点关注相关对象，Apollo将ROI过滤器应用于点云和图像数据，以缩小搜索范围并加快感知。
然后，输出用于构建围绕对象的三维边界框
最后，我们使用被称为检测跟踪关联的算法，来跨时间步识别单个对象。
该算法先保留在每个时间步要跟踪的对象列表，然后在下一个时间步中找到每个对象的最佳匹配。
Apollo使用YOLO网络，来检测车道线和动态物体
14.Sensor Data Comparisons
感知通常依赖于摄像头、激光雷达和雷达
- 摄像头非常适用于分类, 在 Apollo 中，摄像头主要用于交通信号灯分类以及车道检测。
- 激光雷达的优势在于障碍物检测，即使在夜间，在没有自然光的情况下。
- 雷达在探测范围和应对恶劣天气方面占优势。
通过融合这三种传感器的数据，可实现最佳聚合性能。
15.Perception Fusion Strategy
用于融合输出的主要算法为卡尔曼滤波
卡尔曼滤波有两个步骤：
- Predict State.
- Update Measurement.
传感器融合可提高感知性能和减少跟踪误差