Inline closed-loop control of bending angles with machine learning supported springback compensation

Abstract

Closed-loop control of product properties is becoming increasingly important in forming technology research and enables users to counteract unavoidable uncertainties in semi-finished product properties and process environments. Therefore, closed-loop controlled forming processes are considered to have the potential to reduce tolerances on desired product properties, resulting in consistent qualities. The achievement of associated increases in robustness and reliability is linked to enormous requirements, which in particular include the inline recording of the product properties to be controlled and the subsequent adaptation of the process control through the targeted derivation of manipulated variables. The present paper uses the example of an air bending process to show how the bending angle can be controlled camera-based and how springback can be compensated within a stroke by recording force signals and subsequently predicting the loaded bending angle using machine learning algorithms. The results show that the combined application of camera-based control and machine learning assisted springback compensation leads to highly accurate bending angles, whereby the results strongly depend on the machine learning algorithms and associated data transformation processes used.

Introduction

Uncertainty plays a decisive role in metal forming and significantly influences product quality and its variance. [1] Manufacturers strive for consistent quality and thus repeatable production of selected product properties. For this reason, manufacturers try to keep their processes, semi-finished products, tools and associated environmental influences as stationary as possible over the production time, which leads to reliable products but not to robust processes. Unavoidable uncertainties resulting, for example, from tool wear [2], machine stiffness [3] or variations in semi-finished product properties [4] must either be monitored individually or, if not taken into account, are reflected in fluctuating product properties [5]. A promising approach that has been increasingly explored in academic research in recent years is aimed at closed-loop control of product properties [6]. The basic idea of this approach is to equip forming machines and corresponding tools with sensors and actuators, whereby product properties can be recorded stroke-by-stroke or inline and subsequently manipulated. However, the realisation of such closed-loop controls is highly challenging. Product properties must be sensed or observed with sufficient frequency and models between the drives of the machine and product properties must be available that allow a controlled manipulation.

The quality of these closed-loop controls is determined by their accuracy and flexibility. The increasing use of servo presses with freely programmable drive motion paths [7] or task space controlled ram movements [8] has significantly increased the accuracy and flexibility of forming processes in recent years . For example, Groche et al. investigate in a deep-drawing process on a 3D Servo Press how multiple degrees of freedom of the press can be used to manipulate wrinkles and how different control strategies influence the process [9]. The increasing use of machine learning algorithms in manufacturing [10] and their ability to establish correlations between sensed semi-finished product properties and trajectory parameters or manipulated variables of the control system, which in turn lead to desired product properties, also offers potential for achieving consistent product quality. The aforementioned potentials resulting from the fusion of closed-loop controls, machine learning algorithms and high-precision servo presses have so far been insufficiently exploited in forming technology yet.

A process that is often closed-loop controlled in forming technology research is sheet metal bending. Different influencing parameters affect the springback, such as the geometry of the tools [11], temperatures [12], material properties and sheet thickness [13]. Numerous authors have dealt with the control of bending angles in recent years. The first data-driven models for the control of springback were introduced by Forcellesse et al. [14], Inamdar et al. [15] and Viswanathan et al. [16], who used neural networks to predict process control variables. Heller et al. [17] and Wang et al. [18] applied control strategies based on semi-analytical and data-driven models, respectively, which require several load-unloading cycles to accurately control the bending angle. Further work on correlations between process, material and geometry parameters using data-driven models or machine learning models is presented by Ma et al. [19], Liu et al. [20] and Şenol et al. [21]. Welo et al. use an analytical model that predicts an optimized rotation that leads to the desired bending angle of aluminum extrusions by feeding in the torques applied by the bending machine and other material parameters. [22] In a further publication, they present the analytical model used in more detail and conclude that the use of additional process data could possibly improve the performance of the closed-loop control approach developed. [23] Predictions of springback-compensating stroke heights were made by Groche et al. via linear regressions, whereby knowledge of material parameters considerably increased the prediction quality. [24] Further research approaches were aimed at the recording of force signals in multi-stage processes, in which force signals from upstream stages are used for feedforward control of bending angles. Experimental results were presented by Groche et al. [25], Havinga et al. demonstrated the suitability of such control approaches in a simulation environment using the example of an industrial progressive die [26]. Measuring systems that measure springback as accurately and cost-effectively as possible are a prerequisite for controlling bending angles. For example, Ha et al. present a measuring system that uses lasers and simple trigonometric relationships to measure springback during tube bending. [27]

The presented approaches demonstrate the relevance of closed-loop control of bending angles, but also show that real-time inline control has not been investigated so far. Either the approaches require several loading-unloading cycles or control bending angles from stroke to stroke. For this reason, the aim of this paper is to closed-loop control bending angles in real-time under fluctuating semi-finished product properties. It is demonstrated how the actual bending angle can be sensed inline based on camera recordings and how the loaded bending angle before springback can be predicted based on force signals using different machine learning algorithms. The paper is organized as follows: “Methodology” presents the methodology consisting of the experimental setup, the chosen closed-loop control strategy and the procedure for predicting the springback-compensating bending angle using machine learning algorithms. “Experimental results” presents the experimental results. In “Major variations of semi-finished products and sheet thickness”, tests are carried out with different sheet thicknesses and semi-finished product variations. Subsequently in “Minor variations of sheet thickness”, tests are carried out with small sheet thickness variations, which are intended to represent sheet thickness variations within a coil. In “Comparison between white and black box model”, the prediction qualities of developed machine learning algorithms are compared with a white box model. Finally, the results are concluded and an outlook is provided in “Conclusion and outlook”.

Methodology

As presented in “Introduction”, data-driven regression models are suitable for predicting loaded bending angles or stroke heights that lead to desired unloaded bending angles. The literature is currently limited to stroke to stroke controls [25], simulative studies [26] or regression models whose inputs are material parameters [24]. Furthermore to the best of the author’s knowledge, there are no publications that sense the bending angle inline with high temporal resolution and derive control variables for the machine drives from it. In the approaches found in the literature, the machine drives are not controlled in real time based on measured bending angles, but stroke heights are predicted via regression models. For this reason, it will be experimentally demonstrated in an air bending process, that

• 1. different machine learning algorithms without knowledge of semi-finished product properties can predict the loaded bending angle with high accuracy based on inline recorded force signals,

• 2. bending angles can be sensed with high temporal resolution using low-cost CCD cameras and computationally efficient image processing,

• 3. predictions of the optimal loaded bending angle and camera-based measurements of the current bending angle can be integrated into the position closed-loop control of the press.

For this purpose, the experimental setup (“Experimental setup”), the closed-loop control strategy (“Closed-Loop control strategy”) and the procedure for developing the machine learning models (“Machine learning prediction”) are presented in the following.

Experimental setup

The bending experiments are carried out with two different materials and three different sheet thicknesses. The goal of the experiments is to test whether the chosen prediction-control approach can identify and control variations in material properties and thus compensate different springback behaviour. It is investigated whether different sheet thicknesses and materials can be bent with high precision using the presented approach without providing the algorithm information about used semi-finished product properties (“Major variations of semi-finished products and sheet thickness”). The materials used for the experiments are AlMg3 and DC01, the most important material parameters of which can be found in Table 1. It also will be investigated whether minor variations in sheet thickness and thus slightly different springback behaviour within a coil or between different coils can be compensated ( “Minor variations of sheet thickness”).

Table 1 Material parameters of the semi-finished products used

The experiments were carried out on the prototype of the 3D Servo Press. The 3D Servo Press is a flexible forming machine with three degrees of freedom in the ram, driven by three eccentric drives decoupled from each other and two spindle drives. This allows vertical translations as well as tilting of the ram around two axes orthogonal to the translation direction. Since the considered air bending process requires only a vertical translation of the tool, the two tilting degrees of freedom are locked within the scope of this work.

Fig. 1

Block diagram of used closed-loop control strategy

Closed-Loop control strategy

The proposed closed-loop control strategy in this paper consists of different elements of the block diagram in Fig. 1. Three piezoelectric force sensors (Kistler 9031A, shown in color in the block element ’Tool’ in Fig. 1) are integrated in the upper tool to measure the process force during the bending operation. In addition, a CCD camera (Basler acA1600-60gm) is mounted on the press table to record the bending process, allowing the bending angle to be measured using image processing techniques. Only the machine-integrated drives are used as actuators for the closed-loop control strategy, no tool-integrated actuators are required. In the first step of the bending process, the workpiece is bent to a bending angle of 23.75$^\circ$. During this time, the three force signals $\textbf{s}_i$ are recorded by a real-time controller (cRIO 9082 National Instruments) with a sampling frequency of 10 kHz, stored as csv-files and sent to a computer that is linked with the programmable logic controller of the press. The computer runs a Python script that reads in the csv-files, preprocesses and transforms them as well as predicts the optimal loaded bending angle $\alpha _{\textrm{pre,l}}$ using machine learning algorithms. The desired unloaded bending angle $\alpha _\textrm{des,u}$ is provided as an additional input to the machine learning algorithms used. The exact procedure for developing the machine learning models is described in “Machine learning prediction. The prediction of the optimum loaded bending angle $\alpha _\textrm{pre,l}$ is used in the control loop as a stationary reference variable and determines the extent to which the component is overbent and thus the springback is compensated. The CCD camera captures images of the process $\textbf{S}_\textrm{im}$ and communicates with a LabVIEW Vision Assistant Express VI running on the computer, which uses edge detection algorithms to calculate the actual loaded bending angle $\hat{\alpha }_\textrm{act,l}$ with a temporal resolution of 40 Hz. The calculation of the deviation between predicted and actual loaded bending angle leads to the control deviation $\textbf{e}_\alpha =[\alpha _\textrm{pre,l}-\hat{\alpha }_\textrm{act,l}=\mathrm {\Delta }\alpha , 0, 0]^T$. Using position control concepts from robotics, manipulated variables for the drives can now be calculated from the bending angle deviation and the control law $\textbf{G}_\textrm{ctl}$. For this purpose, a nonlinear P-controller is used, which multiplies the bending angle deviation with a stationary control gain matrix $\textbf{K}_\textrm{P}$ and an inverse Jacobian matrix $\textbf{J}^{-1}(\mathbf {q)}$ linearized at the respective operating point of the drive positions $\textbf{q}$. In this way, the angular velocities for the eccentric drives are calculated according to

$$\begin{aligned} \dot{\varvec{\varphi }}_\textrm{ctl}=\textbf{K}_\textrm{P} \cdot \textbf{J}^{-1}(\textbf{q}) \cdot \textbf{e}_\alpha . \end{aligned}$$

(1)

It should be noted that the control deviation $\textbf{e}_\alpha$ is a three-dimensional vector, where the second and third entries refer to the control deviations of the tilt angles of the ram. These are included, but are set to 0 due to the exclusive vertical motion of the ram, so that the Jacobian is square and invertible. As already presented by Hoppe et al. [28], the transfer function of the drives can be approximated as a time-delayed PT1 with time constant $\tau$. For a deeper insight into the position control of the press, the reader is referred to [28].

An important property of the presented bending angle control loop is the stationary accuracy. If the transfer function of the control deviation in the Laplace s-domain is given by

(2)

and $\textbf{I}_3 \in \mathbb {R}^{3\times 3}$ represents an identity matrix, the elimination of steady state control deviations can be demonstrated using linear control theory. If Eq. 2 is rearranged to the control deviation $e_\alpha$ and it is assumed that the predicted bending angle is fed into the control loop as a step function, so that $\varvec{\alpha }_\textrm{pre,l}(s) = \varvec{\alpha }_\textrm{pre,l} \cdot s^{-1}$,

$$\begin{aligned} \textbf{e}_\alpha (s) = \frac{\textbf{I}_3 + \textbf{I}_3 \tau s}{\textbf{I}_3 s + \textbf{I}_3 \tau s^2+\textbf{K}_\textrm{P}} \cdot \mathbf {\varvec{\alpha }}_\textrm{pre,l} \end{aligned}$$

(3)

follows. Now, by applying a temporal limit theorem

$$\begin{aligned} \lim _{t\rightarrow \infty }\textbf{e}_\alpha (t)=\lim _{s\rightarrow 0} s \cdot \textbf{e}_\alpha (s) \\= \lim _{s\rightarrow 0} \frac{(\textbf{I}_3 s + \textbf{I}_3 \tau s^2)\cdot \varvec{\alpha }_\textrm{pre,l}}{\textbf{I}_3 s + \textbf{I}_3 \tau s^2+\textbf{K}_\textrm{P}}=\textbf{0}, \end{aligned}$$

(4)

it can be shown that steady state control deviations are eliminated. This is essential for a highly accurate bending angle control, as it takes into account disturbance variables that occur between the drives and the workpiece in the form of punch wear, machine compliance or bearing clearance. These influences were not taken into account in control approaches presented in “Introduction”, so it is assumed that the presented control strategy leads to more accurate bending angles.

Fig. 2

Procedure for estimating the springback-compensating bending angle based on the KDTEA process [29]

Machine learning prediction

As presented in “Closed-Loop control strategy”, different machine learning algorithms are used to predict the desired loaded bending angle $\alpha _\textrm{pre,l}$ based on the force signals $\textbf{S}_\textrm{for}$ from early stages of the bending operation and the desired unloaded bending angle $\alpha _\textrm{des,u}$. The procedure for developing the models is strongly based on the (KDTEA) process by Kubik et al. [29], which provides a systematical procedure for developing machine learning models based on engineering data. In the following, it is shown how three data-driven algorithms with different degrees of complexity, namely a Multiple Linear Regression (MLR), a Neural Network (NN) and a Convolutional Neural Network (CNN), are developed and which methods are applied in the process stages of data acquisition, data preparation, data transformation, modelling and evaluation. An overview of the development of the three models can be seen in Fig. 2.

In the data acquisition step, three time series are recorded (Fig. 3) from the three piezoelectric force sensors mounted in the upper tool for the i-th experimental test.

Fig. 3

Schematic procedure of data pre processing

In the process stage of data pre processing, the three force signals are summed up according to $\textbf{s}_i=\sum _{j=1}^3 \textbf{s}_{ij}$, so that only one time series $\textbf{s}_{i}$ is available for the i-th experimental test. Subsequently, the signals are normalised to a value range between 0 and 1 and scaled to a uniform length. This leads to the time series matrix $\textbf{S}_\textrm{for} \in \mathbb {R}^{m \times n}$, which contains m experimental trials and thus m time series with n datapoints. From the step of data transformation, the procedures must be adapted to the models developed. Shallow learning algorithms, such as MLR and NN, require extensive dimensionality reduction, which can be realized by feature extraction and feature selection. The goal of dimensionality reduction is to select features that contain as much information as possible about the optimal loaded bending angle. Deep Learning algorithms like the CNN can process high dimensional data like images or time series and do not require hand-crafted feature extraction and selection. The dimensionality reduction is realized by an alternating arrangement of convolutional and pooling layers, which are used to automatically learn optimal features. For feature extraction from time series, the Time Series Feature Extraction Library (TSFEL) [30] is used, which extracts 390 different feature values from the temporal, statistical and spectral domains for each time series. In addition, a principal component analysis (PCA) is applied to the time series matrix $\textbf{S}_\textrm{For}$ and the first two principal components are extracted for each time series. As an additional feature, the desired unloaded bending angle $\alpha _\textrm{des,u}$ of each time series is extracted, which is strongly correlated with the loaded bending angle $\alpha _\textrm{pre,l}$. The result of the feature extraction process is a feature matrix $\textbf{F}_\textrm{extr} \in \mathbb {R}^{m \times 393}$, which contains 393 features for m time series.

In the next step, the extracted features are selected dependent on the used model by different feature selection algorithms. Feature selection algorithms are divided into filters and wrappers, whose functionality is fundamentally different. Filter algorithms evaluate the relevance of a feature correlation-based without feeding it into the machine learning algorithm. Wrapper algorithms select the best possible feature subset by successively using different feature subsets as inputs to the machine learning algorithm and selecting the subset that leads to the best model performance. [31] Since wrapper algorithms are significantly more computationally intensive than filter algorithms and their application is problematic, especially for computationally intensive machine learning algorithms, a feed forward selection wrapper algorithm is only used for feature selection of the MLR. This involves splitting the m time series into training and validation data and increasing the number of features until the Mean squared error (MSE) on the validation data set does not decrease further. The procedure results in the feature matrix $\textbf{F}_{\textrm{MLR}} \in \mathbb {R}^{m \times k}$, where k represents the number of selected features. To select features for the NN, Pearson correlation coefficients are calculated between the i-th feature and the measured springback

$$\begin{aligned} s_\textrm{act}=\alpha _\textrm{des,l}-\alpha _\textrm{des,u} \end{aligned}$$

(5)

Table 2 Overview over optimized hyperparameters

according to

$$\begin{aligned} r_i = \frac{\sum _{j=1}^m (f_{i,j}- \overline{f}_i) (s_{\textrm{act},j}- \overline{s}_{\textrm{act}})}{\sqrt{\sum _{j=1}^m (f_{i,j}- \overline{f}_i)^2 \cdot \sum _{j=1}^m (s_{\textrm{act},j}- \overline{s}_{\textrm{act}})^2}}. \end{aligned}$$

(6)

Only features whose correlation coefficient satisfies condition $r_i > r_{\textrm{lim}}$ are selected, from which the feature matrix $\textbf{F}_{\textrm{NN}} \in \mathbb {R}^{m \times l}$ with l selected features is derived. To stabilize the neural network training process, the selected features in the feature matrix $\textbf{F}_{\textrm{NN}}$ are transformed to standard normally distributed random variables so that $\textbf{f}_{i,\textrm{NN}} \sim \mathcal {N}(0,1)$.

Fig. 4

Schematic representation of the models developed: a) CNN, b) NN, c) MLR

In the process stage of modelling, the models are hyperparameter optimized and trained. For the multiple linear regression there are no hyperparameters, its solution is analytically solvable. To find the optimal topology of the NN and the CNN, Bayesian optimizations are applied with different hyperparameters using the Python library Keras. An overview of the optimized hyperparameters and the constrained areas in which they are optimized, is given in Table 2. The topologies of the applied models with input and output variables are shown in Fig. 4. The data is split into 80% training data and 20% test data, with 20% of the training data used as validation data.

The initial learning rate (exponential decay, $\eta _{\textbf{NN}}=\eta _{\textbf{CNN}}=0.001$) and batch size ($b_{\textbf{NN}}=b_{\textbf{CNN}}=1$) are not optimized, but determined heuristically. Both networks use Adam [32] as an optimizer. The CNN is trained over 50 epochs, the NN over 30 epochs. An early stopping criterion stops the training process if no significant improvement of the MSE on the validation data set was achieved in the last 10 (CNN) or 5 epochs (NN).

Fig. 5

Visualisation of predicted and desired as well as loaded and unloaded bending angles

The goodness of fit of the models is evaluated during the training processes based on the MSE between the predicted loaded bending angle $\alpha _{\textbf{pre,l}}$ and real loaded bending angle $\alpha _{\textbf{des,l}}$, that leads to the desired unloaded bending angle, according to

$$\begin{aligned} x_\textbf{MSE} =\frac{1}{N} \sum _{i=1}^N (\alpha _{\textbf{pre,l},i}-\alpha _{\textbf{des,l},i})^2. \end{aligned}$$

(7)

A visualisation between predicted and desired as well as loaded and unloaded bending angles is given in Fig. 5. The loaded bending angles approached in the process are used as labels and the final unloaded bending angle as a feature. However, since the quality of the control does not depend on the prediction of the loaded bending angle, the desired unloaded bending angle $\alpha _{\textbf{des,u}}$ should be compared with the real unloaded bending angle $\alpha _{\textbf{pre,u}}$ resulting from the application of the machine learning-based closed-loop control strategy. For this purpose, additional experiments are carried out to validate the accuracy of the bending angle control after training the machine learning models. This allows the goodness of fit of the machine learning models to be calculated using the residuals between the unloaded desired and unloaded predicted bending angle according to

$$\begin{aligned} \mathrm {\Delta } \alpha = \alpha _{\textbf{pre,u}} - \alpha _\textbf{des,u}. \end{aligned}$$

(8)

Subsequently, probability density functions of the residuals are estimated for the respective models, which allow statements to be derived about the suitability of the models and the closed-loop control strategy.

Experimental results

“Experimental results” presents and discusses the experimental results on the two use cases. In “Major variations of semi-finished products and sheet thickness”, different semi-finished sheet thickness combinations are used to train and subsequently test the models. In “Minor variations of sheet thickness”, a data set is recorded where the workpieces are subject to minor sheet thickness variations. This data set is again used to train the models presented in “Machine learning prediction and evaluate the control based on its ability to compensate for different springback behavior. In the last “Comparison between white and black box model”, the results of the developed black-box models are compared with an analytical white-box model.

Major variations of semi-finished products and sheet thickness

Training of the models

The data set used to train and test the models for the first use case consists of 363 bending experiments. For each of the six material-sheet thickness combinations, loaded bending angles between $30^\circ$ and $50^\circ$ are approached in one degree steps and each test is repeated three times. Only for the workpieces made of AlMg3 with a sheet thickness of 1.5 mm, the loaded bending angles are approached between $40^\circ$ and $50^\circ$ in two-degree increments. Table 3 provides an overview of the test plan.

Table 3 Overview over experiment plan for major variations

Before the models are trained, the camera-based acquisition of the bending angles is validated. For this purpose, 30 randomly selected workpieces are measured with an optical 3D measurement system (GOM ATOS 5) and the bending angles determined from this are compared with those of the camera. Figure 6 shows the bending angle deviations between the two measurement systems.

Fig. 6

Probability density function for bending angle deviations between camera and optical 3D measurement system based on 30 measurements

It can be seen that the bending angle averaged over 30 workpieces agrees well between the two measuring systems, for 28 measurements the deviations scatter by a maximum of $0.4^\circ$. Two bending angles differ by more than $0.5^\circ$. Nevertheless, the camera-based bending angle detection can be said to have a high degree of accuracy.

Figure 7 presents the mean values of normalized force signals with 99% confidence intervals for the first $23.75^\circ$ of bending operations for the six sheet thickness semi-finished product combinations. It can be seen that the force signals of the different sheet thickness-semi-finished product combinations differ significantly. This ensures that the machine learning models used can identify semi-finished product properties from the force signals and thus enable a property-dependent prediction of the loaded bending angle.

Fig. 7

Mean value and 99% confidence interval for normalized force signals between $0.1^\circ$ and $23.75^\circ$ bending angle

Fig. 8

Boxplots of springback for different sheets

The property-dependent prediction is particularly necessary when taking Fig. 8 into account, which shows the springback of the material combinations as boxplots. The springback correlates negatively with the elastic modulus and the sheet thickness, so that a stronger overbending is necessary for thin and soft sheets. Strengths of springback differ significantly between material combinations, which is also reflected in the force signals. Therefore, it is assumed that machine learning models are able to identify correlations between force signals and springback, so that optimal loaded bending angles can be predicted without information about the material used. In the further procedure, the pre-processed force signals shown in Fig. 7 are used as basis for the further process stages of the KDTEA model. In the step of data transformation for the MLR, the application of feed forward selection wrapper algorithm leads to a selection of a fixed number of features. The determination of this number can be explained using Fig. 9.

Fig. 9

MSE of training and validation data for the MLR as a function of the number of features used

While the MSE on the training data constantly decreases with increasing number of features, it reaches a minimum on the validation data set at 22 features and does not decrease further with higher number of features. Since the performance of the models on unknown data sets is to be optimised, 22 features are selected to serve as input variables for the MLR, resulting in the feature matrix $\textbf{F}_{\textrm{MLR}} \in \mathbb {R}^{363 \times 22}$. To select the features for the neural network, the required correlation coefficient $r_{\textrm{lim}}$ between features and springback is set heuristically to 0.6, which leads to a selection of 46 features and thus to the feature matrix $\textbf{F}_{\textrm{NN}} \in \mathbb {R}^{363 \times 46}$. The two feature matrices $\textbf{F}_{\textrm{MLR}}$ and $\textbf{F}_{\textrm{NN}}$ as well as the time series matrix $\textbf{S}_\textrm{For}$ now serve as input variables for the three different models. The Bayesian optimisations of the hyperparameters listed in Table 2 lead for the NN and the CNN to the parameters listed in Table 4.

Table 4 Optimized hyperparameters for CNN and NN

The two neural networks are trained with the specified topologies of the networks and defined input and output variables. The corresponding loss functions for training and validation data sets are shown in Fig. 10. The loss functions of the neural network lead to a significantly lower MSE in the early training phases and then oscillate strongly until the training is stopped after 24 epochs due to the early stopping criterion. In contrast, the loss function of the CNN is much smoother and converges towards $15 \cdot 10^{-3}$ $^{\circ 2}$ after 50 epochs.

Fig. 10

Training and validation loss function of the NN and CNN

Table 5 MSE in $^{\circ 2}$ on training and test data set for all created models

Table 5 presents the MSE on the training and test data set for all three models. Here, the CNN leads to the lowest MSE, the MLR reveals a slight overfitting due to the deviations of the MSE between the training and test data set. The NN performs worst with a MSE of $13.44 \cdot 10^{-2}$ $^{\circ 2}$ on the test data set.

The models can also be compared in terms of their computing efficiency. For this purpose, the time intervals required by each model to predict the bending angle are measured. Table 6 provides an overview of the computing times required.

While the times required for data preparation are the same for all models, 134 ms are required for data transformation when using MLR and NN. The computing time is required to extract features through the TSFEL library and PCA analysis. The execution times of the models vary greatly due to the different model complexities. The MLR requires only 5 ms to convert the features into a prediction, the NN and CNN require considerably longer due to the matrix multiplications performed. Overall, however, the total execution time of all models is similar and is around 200 ms.

Validation of the accuracy of the closed-loop bending angle control

In order to validate the closed-loop bending angle control and to analyse the effects of the different model performances on the accuracy of the unloaded bending angle, 144 additional experiments are performed. An overview of the experimental plan is given in Table 7. Each of the three models is used to perform experiments with each parameter combination contained in Table 7, so that 48 experiments are performed for each model.

Table 6 Comparison of calculation efficiency based on the time required to predict the bending angle

Table 7 Overview over experiment plan for validation of the closed-loop bending angle control

The results of the deviations $\mathrm {\Delta }\alpha$ between the desired unloaded bending angle $\alpha _\textrm{des,u}$ and the actual unloaded bending angle $\alpha _\textrm{pre,u}$ are captured in Fig. 11. It can be seen that the workpieces tend to be overbent and most of the outliers ($|\mathrm {\Delta \alpha }|>1^\circ$) are found in the positive area. Major differences in the accuracy of the different models can also be observed. The application of the CNN leads to significantly more accurate bending angles than the MLR. Only one of the 48 workpieces whose loaded bending angle was predicted by the CNN deviates from the desired bending angle by more than one degree. In contrast, there are five outliers based on the prediction of the MLR, the NN leads to ten outliers. The MSE values of the validation experiments as well as the maximum bending angle deviations can be found in Table 8.

The results of the validation experiments show that the accuracies of the bending angles strongly depend on the performance of the models used. Tendencies that emerge from the MSE on the test data in Table 5 can also be observed for the MSE of the unloaded bending angles, so it is assumed that the choice of the model and the corresponding data transformation steps have a significant influence on the bending angle accuracy. This is supported by Fig. 12, which shows model- and material-specific probability density functions for bending angle deviations.

Fig. 11

Residual diagram of the bending angle deviations $\mathrm {\Delta } \alpha$ for all created models

Table 8 Evaluation of all created models based on the validation experiments

Fig. 12

Estimated probability density function for deviations $\mathrm {\Delta } \alpha$ between desired bending angle $\alpha _\textrm{des,u}$ and real bending angle $\alpha _\textrm{pre,u}$ for all created models

The bending angle deviations depend strongly on the material used. The deviations of the steel (DC01) show only small systematic errors, the expected value of the deviations (shown dashed) is close to 0 for all models. In contrast, the softer aluminium alloy (AlMg3) tends to be overbent, and the systematic bending angle error is higher than $0.5^\circ$ for all three models used. This is particularly remarkable in view of the greater springback of the aluminium alloy. Although all models systematically overestimate the springback of the aluminium alloy, the models can nevertheless be said to have the ability to differentiate between both materials and sheet thicknesses. If the springback from Fig. 8 is taken as a basis, differences in springback of up to $2.5^\circ$ occur between the individual material-sheet thickness combinations. The deviations between the desired and real bending angle when using machine learning-supported bending angle control are mostly limited to $\pm 1^\circ$.

Minor variations of sheet thickness

To investigate whether the models can detect and compensate for fluctuating semi-finished product properties, experiments are carried out with minor sheet thickness variations. Minor variations in material properties within a coil can be detected by machine learning algorithms [33]. How these affect the springback in the form of sheet thickness fluctuations and how these fluctuations can be controlled will be presented in the following subsection. For this purpose, the workpieces are ground, their thickness is reduced and measured by a caliper gauge.

Training of the models

The data set provided for the sheet thickness variations consists of 171 experiments in which loaded bending angles between $30^\circ$ and $48^\circ$ are approached. Workpieces made of DC01 with three different sheet thicknesses (1.0 mm, 1.5 mm, 2.0 mm) are ground and the sheet thickness of the individual workpieces is reduced by up to 0.27 mm. An overview of the experiment plan can be found in Table 9.

Table 9 Overview over experiment plan for minor sheet thickness variations

Similar to the procedure in “Training of the models”, the features of the MLR are selected by a feed forward selection wrapper algorithm, resulting in a selection of four features. The correlation-based application of the filter algorithm ($r_i>0.7$) for the NN leads to the selection of 56 features, so that the feature matrices have the dimension $\textbf{F}_{\textrm{MLR}} \in \mathbb {R}^{171 \times 4}$ and $\textbf{F}_{\textrm{NN}} \in \mathbb {R}^{171 \times 56}$, respectively. Bayesian optimisation of the models leads to similar hyperparameters to those in the previous subsection, which are shown in Table 10.

Table 10 Optimized hyperparameters for CNN and NN for minor sheet thickness variations

The performance in terms of MSE on training and test data set of the three models is shown in Table 11. Since the models are not trained and tested on different materials, the models achieve lower MSE values than in subsection 4.1.1. The CNN achieves the highest accuracy on the test data set, the MLR leads to a similar MSE. The worst results are generated by the NN.

Table 11 MSE in $^{\circ 2}$ on training and test data set for minor sheet thickness variations

Validation of the closed-loop bending angle control for sheet thickness variations

To validate the machine learning supported closed-loop control strategy, 108 experiments are conducted, whose properties are listed in Table 12.

Table 12 Overview over experiment plan for validation of the closed-loop bending angle control with minor sheet thickness variation

Figure 13 shows the bending angle deviations between the desired and the resulting bending angle. Compared to the results from Fig. 11, it can be seen that the accuracies increase significantly when using one single material. Bending angle deviations of $>1^\circ$ occur only once when using the NN. A visual comparison of the performances in the form of estimated probability density functions of the models is provided in Fig. 14. All models lead to slightly positive systematic bending angle deviations, with the expected values of CNN and MLR close to 0 and the NN showing the largest systematic deviation. Large differences can be seen in the variances of the distributions. The NN leads to more fluctuating bending angle deviations, the probability density functions of CNN and MLR are characterised by peaked curves.

An insight into the interdependencies between sheet thickness and springback is given in Fig. 15, which shows the measured springbacks $s_{\textrm{act}}$ (top) and the springbacks predicted by the individual models $s_{\textrm{mod}}$ as a function of the sheet thickness t. For each sheet thickness range, corresponding linear regressions between springback and sheet thickness are plotted, and their coefficient of determination $R^2$ is shown. It can be seen that springback correlates negatively with sheet thickness for each sheet thickness range, with the strength of the correlation decreasing with increasing sheet thickness. While the gradient of the regression line is strongly negative for thin sheets ($t<1.0$ mm), only minor negative gradients can be observed for medium (1.3 mm $< t < 1.5$ mm) and high sheet thickness ($t>1.8$ mm).

Fig. 13

Residual diagram of the bending angle deviations $\mathrm {\Delta } \alpha$ for all created models

Fig. 14

Estimated probability density function for deviations $\mathrm {\Delta } \alpha$ between desired bending angle $\alpha _\textrm{des,u}$ and resulting bending angle $\alpha _\textrm{res,u}$ for minor sheet thickness variations

Fig. 15

Comparison of measured and predicted springback with minor sheet thickness variations

Considering the predicted springbacks of the models, these trends are most closely reflected in the predictions of the CNN. Negative correlations and thus negative regression gradients can be seen for all three sheet thickness ranges. The MLR also reproduces these trends for the thin and thick sheets, but leads to a positive correlation between springback and sheet thickness in the medium sheet thickness range. In contrast, the NN maps these trends insufficiently. For the lower sheet thickness range, the predictions of the NN lead to a strong positive correlation between springback and sheet thickness. Similar trends can be observed for the medium sheet thickness range. This suggests that the NN, in contrast to MLR and CNN, does not adequately reflect the sheet thickness vs. springback relationships. The Pearson correlation coefficients $r_{\textrm{act,mod}}$ between the measured springbacks $s_{\textrm{act}}$ and those predicted by the individual models $s_{\textrm{mod}}$ also demonstrate the superiority of the MLR and the CNN over the NN and can be found in Table 13.

Table 13 Evaluation based on the validation experiments for minor sheet thickness variations

The measured and predicted springbacks of CNN and MLR correlate strongly with each other, the correlation coefficient of the NN is significantly lower. This proves that force signals contain information about slight variations in sheet thickness, which in turn correlate with the amount of springback. These correlations can be uncovered with the help of machine learning algorithms, whereby the quality of the predictions strongly depends on the applied data transformation processes and models.

Comparison between white and black box model

Another interesting approach is the comparison between the predictive quality of analytical white and presented black box models. For this reason, the analytical model of Buranathiti and Cao [34] is used to calculate the springback for each of the validation experiments from “Validation of the accuracy of the closed-loop bending angle control” and “Validation of the closed-loop bending angle control for sheet thickness variations” and compared with the predictions of the CNN. The inputs for the analytical model are material parameters, which are used to calculate the bending moments acting on the sheet and then determine the springback. The springback s can thus be described as

$$\begin{aligned} s = f_\textrm{ana}(E,Y,t,R,v,n,K,\alpha _\textrm{l}), \end{aligned}$$

(9)

where E is the elastic modulus, Y is the yield strength, t is the sheet thickness, R is the die corner radius, v is the Poisson’s ratio, n is the strain hardening exponent, K is the Ludwik constant and $\alpha _\textrm{l}$ is the loaded bending angle. An essential advantage of analytical modelling compared to black box models is the possibility to take uncertainty into account. If the input parameters $x_i$ and their uncertainty $\delta x_i$ are known, the uncertainty of the bending angle prediction $\delta s$ can be calculated via a Gaussian error propagation

$$\begin{aligned} \delta s = \sqrt{\sum _{i=1}^N \left( \frac{df_\textrm{ana}}{dx_i} \cdot \delta x_i\right) ^2} \end{aligned}$$

(10)

as presented in [35]. The input parameters and their uncertainty for the analytical model are shown in Table 14. The die corner radius R and its uncertainty $\delta R$ are assumed to be 0 because the die is not rounded.

Table 14 Input parameters and corresponding uncertainty for the analytic model

Fig. 16

Comparison between measured and predicted springback based on data-driven and analytical models for a) major material / sheet thickness variations and b) minor sheet thickness variations

The comparison between the springback predictions from the analytical model and the CNN can be seen in Fig. 16 for both the material and sheet thickness variations (“Validation of the accuracy of the closed-loop bending angle control”) as well as for the minor sheet thickness variations (“Validation of the closed-loop bending angle control for sheet thickness variations”). Deviations between measured springback and the predictions of both models are small over large parts of the test series and rarely exceed one degree. Nevertheless, systematic deviations occur for both models. The prediction quality of the analytical model for the major material sheet thickness variations is quite high. Many of the measured bending angles are in the confidence range highlighted in red, systematic underestimations occur for 2 mm thick steel parts. In contrast, the CNN underestimates the springback of most aluminium components. One reason for this could be that the force signals of the aluminium components have similarities to those of steel components with low sheet thicknesses (see Fig. 7), so that the predictions are influenced by the springback of steel components. Comparing the coefficients of determination of both models, the deviations of the CNN for aluminium components lead to a worse performance of the CNN. The CNN leads to a coefficient of determination of $R^2_\textbf{CNN} = 0.34$, while the analytical model leads to a coefficient of determination of $R^2_\textbf{ana} = 0.52$.

However, if the CNN is trained only on parts of one single material, as in the case of the minor sheet thickness variations, the CNN reproduces the springback considerably better. The analytical model systematically underestimates the springbacks and leads to a negative coefficient of determination of $R^2_\textbf{ana} = -0.40$. In contrast, the measured springbacks and predictions of the CNN agree well, so that the coefficient of determination is $R^2_\textbf{CNN} = 0.52$. Deviations rarely exceed $0.3 \circ$, which according to Fig. 6 corresponds approximately to the measurement noise of the camera. From these findings, it can be concluded that data-driven models lead to more accurate predictions of the bending angle, provided there are no ambiguities in the data. If different materials have similar force signals but significant differences in springback behaviour, this will affect the predictions and result in poorer model performance.

Conclusion and outlook

In this paper, an air bending process is used to demonstrate how the bending angle can be closed-loop controlled by a hybrid application of machine learning algorithms and camera-based inline detection. A control strategy was presented to manipulate the bending angle and derives manipulated variables for the drives of a multi-axis servo press. An optimal loaded bending angle predicted by machine learning algorithms, that takes into account the springback of the material, serves as reference variable for the closed-loop control. The machine learning algorithms use force signals from early phases of the bending process as input variables, which in turn correlate with semi-finished product properties and thus enable a material- and workpiece-specific prediction of the loaded bending angle. To demonstrate the performance of the control strategy, two different use cases were considered. On the one hand, tests were carried out with different semi-finished product/sheet thickness combinations, and tests were carried out with minor sheet thickness variations. The results show that the machine learning algorithms allow workpiece-specific predictions of the bending angle and can differentiate between different sheet thicknesses and materials. The machine learning algorithms also reflect correlations between minor sheet thickness variations and springback. The accuracies of the bending angles depend heavily on the models used and data transformation processes, with a hyperparameter-optimised Convolutional Neural Network leading to the most accurate bending angles.

In the future, similar strategies can be used to control different processes. In multi-stage forming processes with progressive dies in which actuated bending stages are used, force signals from previous forming stages can be used to predict and feed-forward control workpiece-specific bending angles. It has already been demonstrated that correlations exist between such force signals and springback [26], but experimental investigations in high-speed processes are not yet available. Furthermore, reinforcement learning algorithms can help to detect and control variations in semi-finished product properties due to their higher adaptability to environmental conditions. This makes it possible to transfer findings that have already been gained from deep drawing FEM simulations [36] to bending processes.