A universal convolutional neural network for the pixel-level detection and monitoring of weld beads

Zhuo Wang, Metin Kayitmazbatir, Mihaela Banu

Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI 48109, USA.

DOI:

https://doi.org/10.7494/cmms.2024.2.0835

Abstract:

In weld-based manufacturing processes such as welding and metal deposition additive manufacturing (AM), the weld bead is a direct indicator of manufacturing quality. For example, the geometry of the weld bead was optimized to a net shape which outperformed conventional geometries. Automatic monitoring of weld bead is thus of prime importance for welding process control and quality assurance. This paper develops a general-purpose convolutional neural network (CNN) for pixel-level detection and monitoring of beads, regardless of welding materials, machine, manufacturing conditions, etc. To achieve the generality, we collected a great variety of welding images containing 2677 single-line beads from 231 research articles, followed by pixel-wise hand-annotation. Consequently, the trained CNN can recognize different beads from various backgrounds at a pixel level. Case studies show that compared to the image-level classification in prior research, its pixel-level labeling permits real-time, complete characterization of weld beads (e.g., detailed morphology, discontinuity, spatter, and uniformity) for more informed process control. This research represents a significant step towards developing a truly human-like monitoring system with low-level scene understanding ability and general applicability.

Cite as:

Wang, Z., Kayitmazbatir, M., & Banu, M. (2024). A universal convolutional neural network for the pixel-level detection and monitoring of weld beads. Computer Methods in Materials Science, 24(2), 27–38. https://doi.org/10.7494/cmms.2024.2.0835

Article (PDF):

Keywords:

Weld bead, Additive manufacturing, Machine learning, Process monitoring

References:

Althouse, A. D., Turnquist, C. H., Bowditch, W. A., Bowditch, K. E., & Bowditch, M. A. (2004). Modern welding. Goodheart-Wilcox.

Artaza, T., Suárez, A., Veiga, F., Braceras, I., Tabernero, I., Larrañaga, O., & Lamikiz, A. (2020). Wire arc additive manufacturing Ti6Al4V aeronautical parts using plasma arc welding: Analysis of heat-treatment processes in different atmospheres. Journal of Materials Research and Technology, 9(6), 15454–15466. https://doi.org/10.1016/j.jmrt.2020.11.012.

Assunção, P. D. C., Ribeiro, R. A., Dos Santos, E. B. F., Braga, E. M., & Gerlich, A. P. (2019). Comparing CW-GMAW in direct current electrode positive (DCEP) and direct current electrode negative (DCEN). The International Journal of Advanced Manufacturing Technology, 104, 2899–2910. https://doi.org/10.1007/s00170-019-04175-2.

Charalampous, P., Kostavelis, I., Kopsacheilis, C., & Tzovaras, D. (2021). Vision-based real-time monitoring of extrusion additive manufacturing processes for automatic manufacturing error detection. The International Journal of Advanced Manufacturing Technology, 115, 3859–3872. https://doi.org/10.1007/s00170-021-07419-2.

Chaudhari, R., Parmar, H., Vora, J., & Patel, V. K. (2022). Parametric study and investigations of bead geometries of GMAW-based wire–arc additive manufacturing of 316L stainless steels. Metals, 12(7), 1232. https://doi.org/10.3390/met12071232.

Cho, H.-W., Shin, S.-J., Seo, G.-J., Kim, D. B., & Lee, D.-H. (2022). Real-time anomaly detection using convolutional neural network in wire arc additive manufacturing: Molybdenum material. Journal of Materials Processing Technology, 302, 117495. https://doi.org/10.1016/j.jmatprotec.2022.117495.

DebRoy, T., & David, S. (1995). Physical processes in fusion welding. Reviews of Modern Physics, 67(1), 85–112. https://doi.org/10.1103/RevModPhys.67.85.

Ding, D., Pan, Z., Cuiuri, D., & Li, H. (2015). A practical path planning methodology for wire and arc additive manufacturing of thin-walled structures. Robotics and Computer-Integrated Manufacturing, 34, 8–19. https://doi.org/10.1016/j.rcim.2015.01.003.

Dinovitzer, M., Chen, X., Laliberte, J., Huang, X., & Frei, H. (2019). Effect of wire and arc additive manufacturing (WAAM) process parameters on bead geometry and microstructure. Additive Manufacturing, 26, 138–146. https://doi.org/10.1016/j.addma.2018.12.013.

Geng, H., Li, J., Xiong, J., Lin, X., & Zhang, F. (2017). Optimization of wire feed for GTAW based additive manufacturing. Journal of Materials Processing Technology, 243, 40–47.

Jafari, D., Vaneker, T. H. J., & Gibson, I. (2021). Wire and arc additive manufacturing: Opportunities and challenges to control the quality and accuracy of manufactured parts. Materials & Design, 202, 109471. https://doi.org/10.1016/j.matdes.2021.109471.

Jamrozik, W., & Górka, J. (2020). Assessing MMA welding process stability using machine vision-based arc features tracking system. Sensors, 21(1), 84. https://doi.org/10.3390/s21010084.

Kohler, M., & Langer, S. (2020). Statistical theory for image classification using deep convolutional neural networks with cross-entropy loss. arXiv. https://doi.org/10.48550/arXiv.2011.13602.

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.

Lee, T. H., Kim, Ch., Oh, J. H., & Kam, D. H. (2022). Visualization of cathode spot control using laser irradiation and oxide addition in wire arc additive manufacturing of titanium alloys. Journal of Laser Applications, 34(4), 042024. https://doi.org/10.2351/7.0000738.

Le-Hong, T., Lin, P. C., Chen, J.-Z., Pham, T. D. Q., & Van Tran, X. (2021). Data-driven models for predictions of geometric characteristics of bead fabricated by selective laser melting. Journal of Intelligent Manufacturing, 34, 1241–1257. https://doi.org/10.1007/s10845-021-01845-5.

Li, Y., Wu, C., Wang, L., & Gao, J. (2016). Analysis of additional electromagnetic force for mitigating the humping bead in high-speed gas metal arc welding. Journal of Materials Processing Technology, 229, 207–215. https://doi.org/10.1016/j.jmatprotec.2015.09.014.

Mandal, S., Kumar, S., Bhargava, P., Premsingh, C., Paul, C. P., & Kukreja, L. M. (2015). An experimental investigation and analysis of PTAW process. Materials and Manufacturing Processes, 30(9), 1131–1137. https://doi.org/10.1080/10426914.2014.984227.

Meng, Y., Gao, M., & Zeng, X. (2018). Quantitative analysis of synergic effects during laser-arc hybrid welding of AZ31 magnesium alloy. Optics and Lasers in Engineering, 111, 183–192. https://doi.org/10.1016/j.optlaseng.2018.08.013.

Nuchitprasitchai, S., Roggemann, M. C., & Pearce, J. M. (2017). Three hundred and sixty degree real-time monitoring of 3-D printing using computer analysis of two camera views. Journal of Manufacturing and Materials Processing, 1(1), 2. https://doi.org/10.3390/jmmp1010002.

Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings (pt. III, pp. 234–241). Springer Cham. https://doi.org/10.1007/978-3-319-24574-4_28.

Ruiz, C., Jafari, D., Subramanian, V. V., Vaneker, T. H. J., Ya, W., & Huang, Q. (2022). Prediction and control of product shape quality for wire and arc additive manufacturing. Journal of Manufacturing Science and Engineering, 144(11), 111005. https://doi.org/10.1115/1.4054721.

Scime, L., & Beuth, J. (2018). A multi-scale convolutional neural network for autonomous anomaly detection and classification in a laser powder bed fusion additive manufacturing process. Additive Manufacturing, 24, 273–286. https://doi.org/10.1016/j.addma.2018.09.034.

Song, G.-H., Lee, C.-M., & Kim, D.-H. (2021). Investigation of path planning to reduce height errors of intersection parts in wire-arc additive manufacturing. Materials, 14, 6477. https://doi.org/10.3390%2Fma14216477.

Sreenathbabu, A., Karunakaran, K., & Amarnath, C. (2005). Statistical process design for hybrid adaptive layer manufacturing. Rapid Prototyping Journal, 11(4), 235–248. https://doi.org/10.1108/13552540510612929.

Tang, S., Wang, G., Huang, C., Li, R., Zhou, S., & Zhang, H. (2020). Investigation, modeling and optimization of abnormal areas of weld beads in wire and arc additive manufacturing. Rapid Prototyping Journal, 26(7), 1183–1195. https://doi.org/10.1108/RPJ-08-2019-0229.

Wang, Z., Yang, W., Liu, Q., Zhao, Y., Liu, P., Wu, D., Banu, M., & Chen, L. (2022a). Data-driven modeling of process, structure and property in additive manufacturing: A review and future directions. Journal of Manufacturing Processes, 77, 13–31. https://doi.org/10.1016/j.jmapro.2022.02.053.

Wang, Z., Yang, W., Xiang, L., Wang, X., Zhao, Y., Xiao, Y., Liu, P., Liu, Y., Banu, M., Zikanov, O., & Chen, L. (2022b). Multi-input convolutional network for ultrafast simulation of field evolvement. Patterns, 3(6), 100494. https://doi.org/10.1016/j.patter.2022.100494.

Wright, W. J., Darville, J., Celik, N., Koerner, H., & Celik, E. (2022). In-situ optimization of thermoset composite additive manufacturing via deep learning and computer vision. Additive Manufacturing, 58, 102985. https://doi.org/10.1016/j.addma.2022.102985.

Wu, D., Hua, X., Li, F., & Huang, L. (2017). Understanding of spatter formation in fiber laser welding of 5083 aluminum alloy. International Journal of Heat and Mass Transfer, 113, 730–740. https://doi.org/10.1016/j.ijheatmasstransfer.2017.05.125.

Wu, D., Hu, M., Huang, Y., Zhang, P., & Yu, Z. (2021). In situ monitoring and penetration prediction of plasma arc welding based on welder intelligence-enhanced deep random forest fusion. Journal of Manufacturing Processes, 66, 153–165. https://doi.org/10.1016/j.jmapro.2021.04.007.

Xiong, J., Zhang, G., Qiu, Z., & Li, Y. (2013). Vision-sensing and bead width control of a single-bead multi-layer part: Material and energy savings in GMAW-based rapid manufacturing. Journal of Cleaner Production, 41, 82–88. https://doi.org/10.1016/j.jclepro.2012.10.009.

Yuan, L., Pan, Z., Ding, D., He, F., Duin, S. van, Li, H., & Li, W. (2020). Investigation of humping phenomenon for the multi-directional robotic wire and arc additive manufacturing. Robotics and Computer-Integrated Manufacturing, 63, 101916. https://doi.org/10.1016/j.rcim.2019.101916. Zhang, Z., Wen, G., & Chen, S. (2019). Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding. Journal of Manufacturing Processes, 45, 208–216. https://doi.org/10.1016/j.jmapro.2019.06.023