ALGORITHM FOR COMPLEXING MULTIPLE DATA SOURCES INTO A SINGLE OCCUPANCY MAP
Abstract
The paper deals with the problem of constructing a passability model of environment with
a large number of dynamic objects based on data from several different sensors. The aim of the
work is to improve the algorithm for constructing the occupancy map by adding data from both
existing algorithms for moving obstacles detection and from millimeter-wave automotive radar.
The study solves the problem of combining data on static environment and dynamic objects into
one general passability model for further trajectory planning. The modification of the algorithm
presented in the article is able to combine data from both occupancy maps based on a threedimensional point cloud from any sensor such as lidar or radar, and arrays of bounding boxes of
objects with known coordinates, sizes, and orientation. Data aggregation occurs at the level of
building occupancy maps and does not impose requirements on the source of information about
dynamic obstacles. The algorithm is able to refine the data on the position and size of the dynamic
object by speed from the radar, which allows to plan the trajectory taking into account the movement
of dynamic objects. The parallel use of the classical approach allows to detect obstacles in
the event of an error in the output of the dynamic obstacle detection algorithm. The developed
algorithm works in real time on the Jetson AGX Xavier module, and is tested in real conditions on
a mobile robotic platform in autonomous mode. Promising directions for further research to improve
the presented approach are formulated.
References
Vol. 22, No. 6, pp. 46-57.
2. Jaspers H., Himmelsbach M., Wuensche H.J. Multi-modal local terrain maps from vision and
lidar, 2017 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2017, pp. 1119-1125.
3. Huang J. et al. An online multi-lidar dynamic occupancy mapping method, 2019 IEEE Intelligent
Vehicles Symposium (IV). IEEE, 2019, pp. 517-522.
4. Xu L. et al. An occupancy grid mapping enhanced visual SLAM for real-time locating applications
in indoor GPS-denied environments, Automation in Construction, 2019, Vol. 104, pp.
230-245.
5. Wang Y. et al. A Fusion of Dynamic Occupancy Grid Mapping and Multi-object Tracking
Based on Lidar and Camera Sensors, 2020 3rd International Conference on Unmanned Systems
(ICUS). IEEE, 2020, pp. 107-112.
6. Eppenberger T. et al. Leveraging Stereo-Camera Data for Real-Time Dynamic Obstacle Detection
and Tracking, arXiv preprint arXiv:2007.10743, 2020.
7. Mohajerin N., Rohani M. Multi-step prediction of occupancy grid maps with recurrent neural
networks, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
2019, pp. 10600-10608.
8. Schreiber M., Hoermann S., Dietmayer K. Long-term occupancy grid prediction using recurrent
neural networks, 2019 International Conference on Robotics and Automation (ICRA).
IEEE, 2019, pp. 9299-9305.
9. Geiger A., Lenz P., Urtasun R. Are we ready for autonomous driving? the kitti vision benchmark
suite, 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2012,
pp. 3354-3361.
10. Caesar H. et al. Nuscenes: A multimodal dataset for autonomous driving, Proceedings of the
IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11621-11631.
11. He Y. et al. Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality
Collaboration, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition, 2021, pp. 2969-2980.
12. Wu H. et al. 3D Multi-Object Tracking in Point Clouds Based on Prediction Confidence-
Guided Data Association, IEEE Transactions on Intelligent Transportation Systems, 2021.
13. Belyaev A.A. et al. Object Detection in an Urban Environment Using 77GHz Radar, 2019 Radiation
and Scattering of Electromagnetic Waves (RSEMW). IEEE, 2019, pp. 436-439.
14. Wang T. et al. Integrating millimeter wave radar with a monocular vision sensor for on-road
obstacle detection applications, Sensors, 2011, Vol. 11, No. 9, pp. 8992-9008.
15. Meyer M., Kuschk G. Deep learning based 3d object detection for automotive radar and camera,
2019 16th European Radar Conference (EuRAD). IEEE, 2019, pp. 133-136.
16. Chavez-Garcia R.O., Aycard O. Multiple sensor fusion and classification for moving object
detection and tracking, IEEE Transactions on Intelligent Transportation Systems, 2015,
17. Vol. 17, No. 2, pp. 525-534.
18. Chang S. et al. Spatial Attention fusion for obstacle detection using mmwave radar and vision
sensor, Sensors, 2020, Vol. 20, No. 4, pp. 956.
19. John V., Mita S. RVNet: deep sensor fusion of monocular camera and radar for image-based
obstacle detection in challenging environments, Pacific-Rim Symposium on Image and Video
Technology. Springer, Cham, 2019, pp. 351-364.
20. Oh S.I., Kang H.B. Fast occupancy grid filtering using grid cell clusters from LIDAR and stereo
vision sensor data, IEEE Sensors Journal, 2016, Vol. 16, No. 19, pp. 7258-7266.
21. Yin T., Zhou X., Krähenbühl P. Center-based 3d object detection and tracking, arXiv preprint
arXiv:2006.11275, 2020.
22. Shepel' I.O. Modifitsirovannyy algoritm postroeniya karty zanyatosti po oblaku tochek ot
neskol'kikh lidarov [A modified algorithm for constructing an employment map based on a
point cloud from several lidars], Programmnye produkty i sistemy [Software products and Systems],
2020, Vol. 33, No 2.
23. Kelly A., Stentz A. Rough terrain autonomous mobility–part 2: An active vision, predictive
control approach, Autonomous Robots, 1998, Vol. 5, No. 2, pp. 163-198.
24. Abramenko A.A. Kalibrovka vzaimnogo raspolozheniya stereokamery i trekhmernogo
skaniruyushchego lazernogo dal'nomera [Calibration of the relative position of a stereo camera
and a three-dimensional scanning laser rangefinder], Komp'yuternaya optika [Computer optics],
2019, Vol. 43, No. 2.
25. Lee C. L. et al. Extrinsic and temporal calibration of automotive radar and 3D LiDAR, 2020
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020,
pp. 9976-9983.
