EXPERIMENTAL ESTIMATION OF ERRORS IN RECONSTRUCTING THE STRUCTURE OF THE OBSERVED SCENE FROM A SERIES OF IMAGES BY VARIOUS CAMERAS
Abstract
The article is devoted to the study of the influence of using various mathematical models of cameras,
and therefore models of scene image formation, when restoring the 3-D structure of a scene from a
set of 2-D images during camera movement (restoring the structure from motion, hereinafter referred to
as LEDs). A comparative assessment is carried out for two camera models: the classic central projection
camera model and the relatively new omnidirectional camera model. The article provides a brief
description of the mathematical model of an omnidirectional camera, the described model is used during
experiments, and also describes ways to represent images from omnidirectional cameras. Additionally,
a description of the mathematical model of the classical camera of the central projection is given.
The described model is also used during experiments. The analytical calculations used in solving the
problem of restoring structure from motion are briefly mentioned in the article. An algorithm for obtaining
3-D coordinates of the points of the observed scene from a sequence of images in motion is also
described. The experiments carried out as part of the study are described in detail in this article. The
process of setting visual landmarks and determining their true 3-D coordinates is revealed. The steps for
the formation of data sets for obtaining comparative estimates are described. At the end of the work, an
analysis of the experimental results is given, models are identified that reduce the errors in restoring the
3-D coordinates of the observed visual landmarks
References
modeling, ISPRS Journal of Photogrammetry and Remote Sensing, 2015, Vol. 101, pp. 275-290.
2. Murtiyoso A., Grussenmeyer P. Documentation of heritage buildings using close-range uav
images: dense matching issues, comparison and case studies, The Photogrammetric Record,
2017, Vol. 32, pp. 206-229.
3. Liao J., Yue Y., Zhang D., Tu W., Cao R., Zou Q., Li Q. Automatic tunnel crack inspection
using an efficient mobile imaging module and a lightweight cnn, IEEE Transactions on Intelligent
Transportation Systems, 2022.
4. Zhang C., Cui Y., Zhu Z., Jiang S., Jiang W. Building height extraction from gf-7 satellite images
based on roof contour constrained stereo matching, Remote Sensing, 2022, Vol. 14, pp. 1566.
5. da Silveira T.L., Jung C.R. Dense 3d scene reconstruction from multiple spherical images for 3-dof+ vr
applications, in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces, 2019, pp. 9-18.
6. Leutenegger S., Chli M., Siegwart R. BRISK: Binary Robust Invariant Scalable Keypoints,
Computer Vision (ICCV), 2011, pp. 2548-2555.
7. Calonder M., Lepetit V., Strecha C., Fua P. BRIEF: Binary Robust Independent Elementary
Features, 11th European Conference on Computer Vision (ECCV), 2010, pp. 778-792.
8. Wen W., Zhou Y., Zhang G., Fahandezh-Saadi S., Bai X., Zhan W., Tomizuka M., Hsu L.-T.
Urbanloco: A full sensor suite dataset for mapping and localization in urban scenes, in 2020
IEEE International Conference on Robotics and Automation (ICRA), 2020, pp. 2310-2316.
9. Pagani A., Stricker D. Structure from motion using full spherical panoramic cameras, in 2011
IEEE International Conference on Computer Vision Workshops, 2011, pp. 375-382.
10. da Silveira T.L., Pinto P.G., Murrugarra-Llerena J., Jung C.R. 3d scene geometry estimation
from 360 imagery: A survey, ACM Computing Surveys, 2022, Vol. 55, pp. 1-39.
11. Anguelov D., Dulong C., Filip D., Frueh C., Lafon S., Lyon R., Ogale A., Vincent L., Weaver J.
Google street view: Capturing the world at street level, Computer, 2010, Vol. 43, pp. 32-38.
12. Zhang X., Zhao P., Hu Q., Ai M., Hu D., Li J. A uav-based panoramic oblique photogrammetry
(pop) approach using spherical projection, ISPRS Journal of Photogrammetry and Remote
Sensing, 2020, Vol. 159, pp. 198-219.
13. Lai P.K., Xie S., Lang J., Laganiere R. Real-time panoramic depth maps from omni-directional
stereo images for 6 dof videos in virtual reality, in 2019 IEEE Conference on Virtual Reality
and 3D User Interfaces (VR), 2019, pp. 405-412.
14. Hartmann W., Havlena M., Schindler K. Recent developments in large-scale tie-point matching,
ISPRS Journal of Photogrammetry and Remote Sensing, 2016, Vol. 115, pp. 47-62.
15. Zhao Q., Feng W., Wan L., Zhang J. Sphorb: A fast and robust binary feature on the sphere,
International journal of computer vision, 2015, Vol. 113, pp. 143-159.
16. Torii A., Havlena M., Pajdla T. From google street view to 3d city models, in 2009 IEEE 12th international
conference on computer vision workshops, ICCV Workshops, 2009, pp. 2188-2195.
17. Tian Y., Fan B., Wu F. L2-net: Deep learning of discriminative patch descriptor in euclidean
space, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
2017, pp. 661-669.
18. Murrugarra-Llerena J., da Silveira T.L., Jung C.R. Pose estimation for two-view panoramas
based on keypoint matching: A comparative study and critical analysis, in Proceedings of the
IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5202-5211.
19. Jiang S., Jiang C., Jiang W. Efficient structure from motion for large-scale UAV images:
A review and a comparison of sfm tools, ISPRS Journal of Photogrammetry and Remote Sensing,
2020, Vol. 167, pp. 230-251.
20. Mouragnon E., Lhuillier M., Dhome M., Dekeyser F., Sayd P. Real time localization and 3d
reconstruction, in 2006 IEEE Computer Society Conference on Computer Vision and Pattern
Recognition, 2006, Vol. 1, pp. 363-370.
21. Scaramuzza D., Martinelli A., Siegwart R. A flexible technique for accurate omnidirectional
camera calibration and structure from motion, in Fourth IEEE International Conference on
Computer Vision Systems, 2006, pp. 45-45.
