Figure 2: Structure of multi-sensor data fusion

The multi-sensor system (comprising n sensors) fuses the information obtained from the object and its environment. The system develops m fusion nodes to fuse n pieces of information. The information output from sensor 1 (S1) and sensor 2 (S2) fuses onto information S_12 in fusing node 1. This fused information further fuses into S_123 with the information of sensor3 in node 2. The fusing continues until all information from n sensors is fused, resulting in information S. The fusing database receives this information S. This fusing database (a part of the database of the whole intelligent control system) stores the results of the information fusion. A few specifications for the above fusing process are as follows:

1. The fusing node’s input and output are in vector form and can fuse multiple input information.
2. Only one fusing node (m=1) can exist. In such a scenario, the information of all n sensors is the input information of the node.
3. The results of the middle nodes are used as output and sent directly to the fusing database (represented in the figure with black lines).

The modules (extreme right boxes) mentioned in figure 2 have the following names and specifications:

1. Database of expert knowledge: Only those with the necessary field knowledge and proper fusing algorithm (especially in practical industrial control systems) can meet the rigours of information fusion. Such knowledge constitutes the database of expert knowledge.
2. Database of sensor models: it stores the sensor models in use. This database partly describes a few sensor features and how the environment influences these features.
3. Information coordination and management: Sensors generally describe the same feature from different coordinate systems. These should be standardisation of the space-time reference coordinates system keeping in mind their difference in time, space, and presentation methods. The module chooses the appropriate sensor to function correctly and adapt to the given environment. Such selection depends on the coordination of different factors like space and time.
4. Methods for information fusion: Different tasks and objects adopt different methods. The neural networks method has advantages in fusing the sensor information in a non-linear and uncertain environment, the best of which is the ART network. The following diagram shows an ART -2 network that can deal with the analogue information:

Figure 3: ART-2 neural network

The bottom layer of the neural network accepts the input information from the object and environment, with one input unit receiving an element of the input pattern vector. The unit at the top is a prototype and provides the cluster class of a similar input pattern. The interconnecting value between the bottom and top layers records the features of the input data. The principles of ART – 2 are based upon the mechanism of competitive learning: an input pattern applied to the network is pre-processed by the network with filtering, increasing contrast and unification. You can then compare the pattern with the prototype. The winner is the one that most resembles the input prototype. If the similarity between the winner and the input is greater than the pre-chosen warning parameter, learning is needed, and the weight value of the winner is revised to show the features of the input pattern.

Conversely, if the similarity is less than the minimum warning parameter, the current winner is emptied, and the search process repeats. If no winner exhibits similarity with the input, a new unit is established on the top layer and made a prototype similar to the input by learning. In this way, the port on the top layer corresponds with a type of input information class, thus completing the combination of the input modes and realising the fusion of the input information.

Classification and comparison of fusion methods

The fusion methods or strategies classify into low, medium, and high levels. These three levels differ in the complexity of processing inputs from data sources.

Low-level data fusion (LLDF) is the simplest technique to realise a combination of inputs. Low-level data is rearranged into a new data matrix, where the variables, extracted from different sources, are successively placed. The combined data matrix variables are the sum of the previously separated data sets. The concatenated data, in most cases, is then pretreated before the final creation of the classification or regression model. However, it is possible to conduct specific elementary operations before putting them together.

Medium-(mid-) level data fusion (MLDF) or “feature-level" fusion eliminates the insufficiently diverse, non-informative variables from the datasets. It is based on a preliminary feature extraction that continues to maintain the relevant variables. You can use the developed algorithms to choose these features before merging them into one matrix that finds use in a chemometric method.

High-level data fusion (HLDF) works on a decision level. These models are regression models that provide continuous responses for the classifications or input data, deciding the class membership of the new samples. The combination of decisions from these models forms a complex model that can create the final estimation. HLDF may unify the outputs in one decision model to arrive at a better estimate.

Figure 4: Comparison of fusion methods

Application of sensor data fusion in laser welding

The following figure shows a multi-sensor system that includes an ultraviolet/visible (UVV) visual sensor, an x-ray visual imaging sensor, a laser reflective photoelectric sensor, a spectral graph, a visible light photoelectric sensor, and an auxiliary illumination visual sensor. You can use all these sensors during laser welding for timely detection and analysis.