Detecting dents in car bodies using machine learning and structured light projection

Izabela Potasz , Sławomir Potasz, Michał Laska

VUMO sp. z o.o.

DOI:

https://doi.org/10.7494/cmms.2024.3.0836

Abstract:

This article discusses feasible methods for detecting dents in car bodies caused by transportation damage, commuting collisions, and hail. The authors review existing approaches exploiting their limitations, including smartphone-based ML detection algorithms and drive-through tunnels. The paper details the setup for capturing dents using computer vision with industry-grade cameras and structured light projection, emphasizing optimized data acquisition and computer vision setup. A particular emphasis is placed on acquiring high-quality input data thanks to the proper calibration and alignment of cameras, structured light, and the synchronization between them. Challenges related to obtaining high-quality footage in real-life conditions, such as car speed, body color, and lighting conditions, are thoroughly discussed. The method covers algorithms for detecting car paint, optimizing camera parameters, and identifying dents. Data annotation methods are described in detail, ensuring robust training datasets. Validation of the method is based on comparing the results of an inspection by professional car appraisers with algorithm detection outcomes. The results demonstrate the effectiveness of the proposed methods. Additionally, the article explores future research opportunities, such as scratch detection, damage severity estimation, and integrating these systems into automated production lines. The potential for enhancing vehicle inspection processes through advanced computer vision and structured light techniques is also considered.

Cite as:

Potasz, I., Potasz, S. & Laska, M. (2024). Detecting dents in car bodies using machine learning and structured light projection. Computer Methods in Materials Science, 24(3), 41-50. https://doi.org/10.7494/cmms.2024.3.0836

Article (PDF):

Keywords:

Car body inspection, Painted surfaces, Structured light projection, Dents, Liquid crystals, Light valve, Automotive inspection

References:

Arnal, L., Solanes, J.E., Molina, J., & Tornero, J. (2017). Detecting dings and dents on specular car body surfaces based on optical flow. Journal of Manufacturing Systems, 45, 306–321. http://doi.org/10.1016/j.jmsy.2017.07.006.

Chen, H. (2008). Automatic Dent Detection on Car Bodies [Master thesis, Hong Kong University of Science and Technology]. Rare & Special e-Zone. https://doi.org/10.14711/thesis-b1023250.

Gembicky, M., Oss, D., Fuchs R., & Coppens, P. (2005). A fast mechanical shutter for submicrosecond time-resolved synchrotron experiments. Journal of Synchrotron Radiation, 12, 665–669. https://doi.org/10.1107/S090904950501770X.

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C.L. (2014). Microsoft COCO: Common objects in context. In D. Fleet, T. Pajdla, B. Schiele, T. Tuytelaars (Eds.), Computer Vision – ECCV 2014. 13th European Conference, Zurich, Switzerland, September 6–12, 2014, Proceedings (pt. 5, pp. 740–755). Springer Cham. https://doi.org/10.1007/978-3-319-10602-1_48.

Mumuni, A., & Mumuni, F. (2022). Data augmentation: a comprehensive survey of modern approaches. Array, 16, 100258. https://doi.org/10.1016/j.array.2022.100258.

Park, S.H., Tjolleng, A., Chang, J., Cha, M., Park J., & Jung, K. (2020). Detecting and localizing dents on vehicle bodies using region-based convolutional neural network. Applied Sciences, 10(4), 1250. https://doi.org/10.3390/app10041250.

Patil, K., Kulkarni, M., Sriraman, A., & Karande, S. (2017). Deep learning based car damage classification. 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA). https://doi.org/10.1109/ICMLA.2017.0-179.

Pham, L.H., Tran, D.N.N., Byun, J.Y., Rhie, Ch.H., & Jeon, J.W. (2021). A smartphone-based laser measuring system for gap and flush assessment in car body. IEEE Transactions on Industrial Electronics, 68(7), 6297–6307. https://doi.org/10.1109/TIE.2020.2992971.

RadhaKrishna, M.V.V., Govindh, M.V., & Veni, P.K. (2021). Review on image processing sensor. Journal of Physics: Conference Series, 1714, 012055. https://doi.org/10.1088/1742-6596/1714/1/012055.

Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). YOLO You Only Look Once: unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2016.91.

Ren, S., He, H., Girshick, R., & Sun, J. (2017). Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.

Shin, H., Ryu, J.W., Gwag, J.S., Lee, H.-M., Hong, J.-Y., Kim, S.-W., Lee, Y.-S., & Kim, J.M. (2022). Temperature-controllable liquid crystal window shutter. Journal of the Korean Physical Society, 80, 273–278. https://doi.org/10.1007/s40042-022-00422-0.

Šmíd, M., & Matas, J. (2019). Rolling shutter camera synchronization with sub-millisecond accuracy. arXiv:1902.11084, https://doi.org/10.48550/arXiv.1902.11084.

Tian, Z., Shen, C., Chen, H., & He, T. (2019). FCOS: fully convolutional one-stage object detection. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/ICCV.2019.00972.

Van der Jeught, S., & Dirckx, J. (2019). Deep neural networks for single shot structured light profilometry. Optics Express, 27(12), 17091–17101. https://doi.org/10.1364/OE.27.017091.

Wang, Ch.-Y., Bochkovskiy, A., & Liao, H.-Y.M. (2020). Scaled-YOLOv4: scaling cross stage partial network. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR46437.2021.01283.

Wang, Ch.-Y., Yeh, I.-H. & Liao, H.-Y. (2021). You Only Learn One Representation: Unified Network for Multiple Tasks. arXiv:2105.04206. https://doi.org/10.48550/arXiv.2105.04206.

Wang, Y., Zhu, D., Yang, Y., Lee, K., Mishra, R., Go, G., Oh, S.-H., Kim, D.-H., Cai, K., Liu, E., Pollard, S.D., Shi, S., Lee, J., Teo, K.L., Wu, Y., Lee, K.-J., & Yang, H. (2019). Magnetization switching by magnon-mediated spin torque through an antiferromagnetic insulator. Science, 366(6469), 1125–1128. https://doi.org/10.1126/science.aav8076.

Wei, L., Anguelov, D., Erhan, D., Szegedy, Ch., Reed, S., Fu, Ch.-Y., & Berg, A.C. (2016). SSD: single shot multibox detector. In B. Leibe, J. Matas, N. Sebe, M. Welling (Eds.), Computer Vision – ECCV 2016. 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings (pt. 1, pp. 21–37). Springer Cham. https://doi.org/10.1007/978-3-319-46448-0_2.