IMAGE MATCHING SYSTEM WITH USING INTUITIONISTIC FUZZY SETS
Abstract
This paper presents a fully learnable system for solving the problem of matching two images. All the main elements of the system are trainable, i.e. their final form corresponds to the target dataset on which the training was carried out. The fact that the system is trainable, the methods used in training and the architecture of the system allow using the system to solve a large number of various computer vision problems. The system consists of a convolutional neural network that serves both to extract key points and their descriptors, as well as a trainable matcher of the extracted key points based on their description and mutual arrangement in the observed scene. The used convolutional neural network processes full-size images and calculates both the location of interest points with pixel accuracy and the descriptors associated with them in a single forward pass. Matching key points is a separate step and is performed after the forward pass of the neural network. In the process of training the model for calculating the positions of key points and their descriptors, a method for forming a training sample is used, called homographic adaptation - an approach that helps to increase the repeatability and accuracy of detecting key points. The process of training the feature point detection model consists of obtaining new weights in the process of additional training of the base detector, which represents the initialization weights of the model. The final feature point detection model, trained on the universal MS-COCO image set using homographic adaptation, repeatedly outperforms the original base detector in terms of the number, reliability and repeatability of feature points, and also outperforms any other traditional corner detector based on classical approaches
##article.references##
1. Balntas V., Lenc K., Vedaldi A., Mikolajczyk K. Hpatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2017, pp. 5173-5183.
2. Chen H., Luo Z., Zhou L. [et al.]. Aspanformer: Detector-free image matching with adaptive span trans-former, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alami-tos: IEEE Computer Society, 2022, pp. 5277-5287.
3. Schonberger J.L., Frahm J.-M. Structure from-motion revisited, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2016, pp. 4104-4113.
4. Sun J., Shen Z., Wang Y., Bao H., Zhou X. Loftr: Detector-free local feature matching with transformers, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2021, pp. 8918-8927.
5. Truong P., Danelljan M., Van Gool L., Timofte R. Learning accurate dense correspondences and when to trust them, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2021, pp. 5710-5720.
6. DeTone D., Malisiewicz T., Rabinovich A. Superpoint: Self-supervised interest point detection and de-scription, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2018, pp. 224-236.
7. Karpur A., Perrotta G., Martin-Brualla R. [et al.]. Lfm-3d: Learnable feature matching across wide baselines using 3d signals, Proceedings of the International Conference on 3D Vision (3DV). Los Alamitos: IEEE, 2024, pp. 1234-1245.
8. Zuev V.M. Sravnenie obnaruzheniya ob"ektov sredstvami iskusstvennogo intellekta v sravnenii s klas-sicheskimi metodami [Comparison of object detection using artificial intelligence methods versus classi-cal methods], Problemy iskusstvennogo intellekta [Problems of Artificial Intelligence], 2024,
No 3 (34), pp. 30-35.
9. Dai A., Chang A.X., Savva M. [et al.]. Scannet: Richly-annotated 3D reconstructions of indoor scenes, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2017, pp. 5828-5837.
10. Silveira T.L., Jung C.R. Dense 3D scene reconstruction from multiple spherical images for 3-DOF+ VR applications, IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Los Alamitos: IEEE, 2019, pp. 9-18.
11. Tang L., Jia M., Wang Q. [et al.]. Emergent correspondence from image diffusion, arXiv. 2023. arXiv:2306.03881.
12. Torii A., Havlena M., Pajdla T. From Google Street View to 3D city models, Proceedings of the Inter-national Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2009, pp. 2188-2195.
13. Tian Y., Fan B., Wu F. L2-Net: Deep learning of discriminative patch descriptor in Euclidean space, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2017, pp. 661-669.
14. Noskov V.P., Kur'yanov A.N. Ispol'zovanie kompleksirovannykh deskriptorov v reshenii SLAM-zadachi [Using complex descriptors for SLAM task solution], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2022, No. 1 (225), pp. 268-278.
15. Morev K.I., Bozhenyuk A.V. Sopostavlenie izobrazheniy po osobym tochkam razlichnykh kategoriy [Im-age matching by feature points of various categories], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2020, No. 3 (213), pp. 192-201.
16. Truong P., Danelljan M., Van Gool L., Timofte R. Learning accurate dense correspondences and when to trust them, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2021, pp. 5710-5720.
17. Tyszkiewicz M., Maninis K.-K., Popov S., Ferrari V. RayTran: 3D pose estimation and shape recon-struction of multiple objects from videos with ray-traced transformers, Proceedings of the European Conference on Computer Vision. Cham: Springer, 2022, pp. 456-472.
18. Bondarenko V.A., Kaplinskiy G.E., Pavlova V.A., Tupikov V.A. Metod poiska i sopostavleniya klyuchevykh osobennostey izobrazheniy dlya raspoznavaniya obrazov i soprovozhdeniya ob"ektov [Method of search and matching of key image features for pattern recognition and object tracking], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2019, No. 1 (203), pp. 281-293.
19. Vaswani A., Shazeer N.M., Parmar N. [et al.]. Attention is all you need, Advances in Neural Infor-mation Processing Systems. Red Hook: Curran Associates, 2017, pp. 5998-6008.
20. Verdie Y., Yi K.M., Fua P., Lepetit V. TILDE: A temporally invariant learned detector, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2015, pp. 5263-5271.
21. Ma J., Jiang X., Fan A. [et al.]. Image matching from handcrafted to deep features: A survey, Interna-tional Journal of Computer Vision, 2020, Vol. 129, No. 1, pp. 23-79.
22. Ono Y., Trulls E., Fua P.V., Yi K.M. LF-Net: Learning local features from images, Advances in Neural Information Processing Systems. Red Hook: Curran Associates, 2018, pp. 6234-6244.
23. Radenovic F., Iscen A., Tolias G., Avrithis Y., Chum O. Revisiting Oxford and Paris: Large-scale image retrieval benchmarking, Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni-tion. Los Alamitos: IEEE Computer Society, 2018, pp. 5706-5715.
24. Deng J., Dong W., Socher R. [et al.]. ImageNet: A large-scale hierarchical image database, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2009, pp. 248-255.
25. Atanassov K. Remark on a property of the intuitionistic fuzzy interpretation triangle, Notes on Intuition-istic Fuzzy Sets, 2002, No. 8 (1), pp. 34-36.
26. Roessle B., Nießner M. End2end multi-view feature matching with differentiable pose optimization, Pro-ceedings of the International Conference on Computer Vision. Paris: IEEE, 2023, pp. 477-487.
27. Li Z., Snavely N. MegaDepth: Learning single-view depth prediction from internet photos, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE, 2018,
pp. 2041-2050.
28. Ustenko V.Yu., Bondarenko V.I. Razrabotka programmnogo kompleksa annotirovaniya dannykh dlya zadach komp'yuternogo zreniya: ob"ektno-orientirovannyy podkhod na osnove winforms [Development of data annotation software complex for computer vision tasks: object-oriented approach based on Win-Forms], Problemy iskusstvennogo intellekta [Problems of Artificial Intelligence], 2024, No. 4 (35), pp. 151-163.
29. Xue F., Budvytis I., Cipolla R. SFD2: Semantic-guided feature detection and description, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer So-ciety, 2023, pp. 5206-5216.
