An evaluation of the capabilities of image-based metal component defect recognition with deep learning techniques
Michał P. Wójcik
, Kacper Pawlikowski, Łukasz Madej
AGH University of Krakow, Mickiewicza 30, 30-059 Krakow, Poland.
DOI:
https://doi.org/10.7494/cmms.2024.3.0839
Abstract:
In the era of Industry 4.0, deploying highly specialised machine learning models trained on unique and often scarce datasets is an attractive solution for advancing automated quality control and minimising production costs. Therefore, the main aim of this research is to evaluate the capabilities of three deep learning models (ResNet-18, ResNet-50 and SE-ResNeXt-101 (32 × 4d)) in the identification of surface defects in forged products. Leveraging advanced photography techniques, including studio lighting and a shadowless box, high-quality images of complex product surfaces were acquired for the training data set. Given the relatively small size of the image dataset, aggressive data augmentation techniques were introduced during the training and evaluation process to ensure robust model generalisation ability. The results obtained demonstrate the significant impact of data augmentation on model performance, highlighting its importance in training and evaluating deep learning models with limited data. This research also emphasises the need for innovative data pre-processing strategies in an efficient and robust machine learning model delivery to the industrial environment.
Cite as:
Wójcik, M.P., Pawlikowski, K. & Madej, Ł. (2024). An evaluation of the capabilities of image-based metal component defect recognition with deep learning techniques. Computer Methods in Materials Science, 24(3), 33-40. https://doi.org/10.7494/cmms.2024.3.0839
Article (PDF):
Keywords:
Deep learning, Convolutional neural networks, Image classification, Data augmentation, Quality control, Surface defect recognition, Forging
References:
Bhatt, P. M., Malhan, R. K., Rajendran, P., Shah, B. C., Thakar, S., Yoon, Y. J., & Gupta, S. K. (2021). Image-based surface defect detection using deep learning: A review. Journal of Computing and Information Science in Engineering, 21(4), 040801. https://doi.org/10.1115/1.4049535.
Chai, J., Zeng, H., Li, A., & Ngai, E. W. T. (2021). Deep learning in computer vision: A critical review of emerging techniques and application scenarios. Machine Learning with Applications, 6, 100134. https://doi.org/10.1016/j.mlwa.2021.100134.
Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., & Le, Q. V. (2019). AutoAugment: Learning augmentation strategies from data. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/CVPR.2019.00020.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE. https://doi.org/10.1109/CVPR.2009.5206848.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale. arXiv. https://doi.org/10.48550/arXiv.2010.11929.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/CVPR.2016.90.
Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE. https://doi.org/10.1109/CVPR.2018.00745.
Huang, G., Liu, Z., van der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/CVPR.2017.243.
Jia, Z., Wang, M., & Zhao, S. (2024). A review of deep learning-based approaches for defect detection in smart manufacturing. Journal of Optics, 53(2), 1345–1351. https://doi.org/10.1007/s12596-023-01340-5.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539.
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022). A ConvNet for the 2020s. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/CVPR52688.2022.01167.
Lugin, S., Müller, D., Finckbohner, M., & Netzelmann, U. (2023). Automated surface defect detection in forged parts by inductively excited thermography and magnetic particle inspection. Quantitative InfraRed Thermography Journal, 1–13. https://doi.org/10.1080/17686733.2023.2266901.
Niccolai, A., Caputo, D., Chieco, L., Grimaccia, F., & Mussetta, M. (2021). Machine learning-based detection technique for NDT in industrial manufacturing. Mathematics, 9(11), 1251. https://doi.org/10.3390/math9111251.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533–536. https://doi.org/10.1038/323533a0.
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, Y., Mollura, D., & Summers, R. M. (2016). Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5), 1285–1298. https://doi.org/10.1109/tmi.2016.2528162.
Tabernik, D., Šela, S., Skvarč, J., & Skočaj, D. (2020). Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 31(3), 759–776. https://doi.org/10.1007/s10845-019-01476-x.
Tan, Ch., Sun, F., Kong, T., Zhang, W., Yang, Ch., & Liu, Ch. (2018). A survey on deep transfer learning. In V. Kůrková, Y. Manolopoulos, B. Hammer, B., I. Lazaros, I. Maglogiannis (Eds.), Artificial Neural Networks and Machine Learning – ICANN 2018. 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4–7, 2018, Proceedings (pt. 3, pp. 270–279). Springer Cham. https://doi.org/10.1007/978-3-030-01424-7_27.
Tan, M., & Le, Q. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning (vol. 97, pp. 6105–6114). Retrieved from http://proceedings.mlr.press/v97/tan19a.html.
Xie, S., Girshick, R., Dollár, P., Tu, Z., & He, K. (2017). Aggregated residual transformations for deep neural networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/CVPR.2017.634.
Yang, J., Li, S., Wang, Z., Dong, H., Wang, J., & Tang, S. (2020). Using deep learning to detect defects in manufacturing: A comprehensive survey and current challenges. Materials, 13(24), 5755. https://doi.org/10.3390/ma13245755.