The CNN Deep Learning-Based Melting Process Prediction of Czochralski Monocrystalline Silicon

To solve seeding failures due to the misjudgment caused by manual observation in the traditional melting process of Czochralski (CZ) monocrystalline silicon, a method for predicting the melting progress of CZ monocrystalline silicon based on Convolutional Neural Network (CNN) deep learning was proposed. The deep learning method and image classification of the melting process were combined. By taking CNN as the research object, the AlexNet network-based melting classification model was constructed. Meanwhile, the comparative analysis was performed by adjusting the number of AlexNet network convolution layers and the size of the convolution kernel. After several experiments, a CNN-based melting stage classification model was finally determined. Simulation results showed that the model could achieve higher accuracy when predicting the melting process. This paper focuses on the key technical issues such as polycrystalline silicon melting and temperature predication in the growth process of the monocrystalline silicon, and predicts the melting process of silicon materials, which lays the foundation for the quality improvement of monocrystalline silicon growth process in the semiconductor field.


I. INTRODUCTION
Monocrystalline silicon is the most essential material for semiconductors. More than 90% of semiconductor devices and integrated circuit chips are made of monocrystalline silicon. The growth process of monocrystalline silicon mainly includes material melting, seeding, shouldering, equal diameter, and finishing [1], [2]. This process begins at the melting stage. When the temperature in the furnace reaches the expected temperature, the silicon material is completely melted by heating the quartz crucible. Then, the seed crystal is immersed in the melt. The melting completeness of silicon material determines the seed crystal adsorption force and the initial temperature of crystal growth. Under the requirements of the simultaneous pulling and rotating process, the premature immersion of the seed crystal in the melt will result in the failures for solidifying the melt along the top of the meniscus at the crystal-melt and interface. Consequently, the temperature of the solid-liquid interface will increase, thereby The associate editor coordinating the review of this manuscript and approving it for publication was Chong Leong Gan .
reducing the temperature gradient of the growth interface and failing to form a monocrystal. Because of the hysteresis of the temperature regulation of the crystal growth system, the temperature of the solid-liquid interface has been regulated, lengthening the crystal growth time and affecting the formation of monocrystalline silicon. Meanwhile, the limited service life of the quartz crucible delays the melting process, and further greatly shortens the effective length of the crystal equal-diameter growth.
The prediction of melting progress is vital for analyzing the melting process of Czochralski (CZ) monocrystalline silicon growth. In the crystal growth system, according to different crystal growth process requirements, the thermal field structure is different, and the phenomenon observed during the melting of the silicon material will be different.
Reasonable prediction of the melting stage can reduce energy consumption to a certain extent, prolong the service life of crucible and other components, and thus improve the economic benefit of the whole process of monocrystalline silicon extraction. Due to the high-temperature in airtight and vacuum growth environment in addition to heat shield installed, it is difficult to effectively observe the material melting process during crystal growth, which leads to judgement errors for the melting time of the silicon material. At present, the detection of the silicon melting process has been researched worldwide [3]- [7]. In China, the melting process of CZ monocrystal growth usually depends on the manual judgment. When taking manual monitoring, the seed crystal and the liquid molten silicon level can only be welded smoothly by regulating the temperature. However, a longer time of temperature regulation increases energy consumption. Lei et al. [8] used the color features and multi-scale morphological gradients of silicon ingot to extract the edge information. Besides, the Bhattacharyya distance was used to measure the similarity of the changes from the watershed changing area to detect whether there was unmelted silicon ingot in the melt. Zeng [9] researched the frequency domain, statistical features, and corner detection methods of the collected image data and obtained a melting process detection method based on pattern recognition. However, it is easy to lose a lot of feature information in the process of feature statistics, which may make the results of this method inaccurate. Secondly, the combination of statistical features and corner detection method has not been realized. Globally, a fully-automatic monocrystal furnace, represented by Vison300 of Kayex [10], detects an area of the liquid molten silicon level by an infrared sensor above the furnace body. When the unmelted silicon material passes through this area, the infrared sensor will send a pulse. If no pulse has been measured within a specified time, the silicon material is considered to be melted. This method is available to detect the melting process; however, the signals obtained by the monitoring are not completely accurate and stable, which may cause a waste of resources. Meanwhile, various factors such as gas flow, centrifugal force generated by the crucible rotation, and thermal convection of the melt, the movement of the polycrystalline silicon block, especially the in the later stage, could trigger irregular movements, which may cause long-term undetectable pulses, resulting in large errors. With the development of deep learning algorithms [11]- [14], the traditional method of extracting features based on artificial experience is converted to automatic image feature extraction through data training. In reference [14], CNN deep learning method was used to classify succulents through AlexNet network transfer learning model, and good classification results were achieved. By abstracting layerwise, more useful features could be learned. In addition, such algorithms shall also take lower image processing, which can avoid the possible effects on results caused by traditional manual extraction of features.
In summary, a deep learning method was used to predict the progress of the melting stage and a convolutional neural network (CNN)-based melting stage classification method was proposed [15], [16]. By adjusting the number of convolution layers and the size of the convolution kernel on the AlexNet network model, a CNN-based melting stage classification model was proposed through simulation experiments. The CNN algorithm in deep learning was used to study the classification performance of the melting stage, which was also applied to the prediction of the crystal growth process through machine learning technology so as to improve efficiency and accuracy.

II. PROBLEMS AND ANALYSIS OF THE MELTING PROCESS
To extract the image features of the melting stage, as well as to choose an efficient and accurate classification method, the analysis of the principle and problems of the material melting process was particularly critical. It was also an initial attempt to classify the melting process of silicon materials in this study.
The melting of the polycrystalline silicon block refers to placing high-purity polycrystalline silicon in a crucible and raising the temperature in the furnace by a graphite heater Therefore, the solid silicon begins to melt into a liquid. Generally, the melting process test detects the melting completeness for the polycrystalline silicon block in the crucible, thereby determining the start for the seeding of crystal growth. The unmelted and fully molten polycrystalline silicon is shown in FIGURE. 1. Melting of polycrystalline silicon is a long and slow process. And the silicon at the bottom of the melt melts first. Over time, the silicon material above the melt surface will collapse [10]; thus, some unmelted silicon material will float on the melt surface. When the polycrystalline silicon block is completely melted, the crystal growth process begins. Otherwise, the effect of pulling will be affected. Also, the heating power should be reduced in time when the silicon material is melted. The excessive thermal energy in the crucible will be absorbed by the molten material. As a result, the temperature in the furnace rises rapidly, exerting various adverse effects on silicon crystal growth: (1) Polycrystalline silicon absorbs heat during the melting process and stops heat absorption after being melted. Excessive heat causes rapid temperature rising in the furnace when the melting is complete. This not only exacerbates the melt convection but also accelerates the corrosion of the quartz crucible and the crucible wall, thereby increasing the oxygen content in the melt.
(2) Increased temperature will also cause the evaporation on the molten silicon surface. The vaporized silicon floats the melt surface, and the molten silicon will ''boil'' in the VOLUME 10, 2022 crucible [10]. The ''boil'' will not only damage the monocrystal furnace but also the surface coating of the quartz crucible.
(3) At the same time, if the temperature rises, it takes a long time to cool down in the furnace. The energy consumption of the system increases accordingly, causing unnecessary waste and shortening the service life of the crucible.

III. CLASSIFICATION STRATEGY OF THE MELTING PROCESS
Before and after the collapsing, the temperature and image features of the silicon material change significantly [2]. Therefore, there are two major methods to determine the melting state for the silicon material. One is to judge the melting of materials through the change of the liquid surface temperature during the melting process [8]; another is to determine the melting completeness of material by the image features of the melting process.
1. The temperature change in liquid surface is used to judge the complete melting: When the melting starts, the temperature of polycrystalline silicon blocks is low. With the continuous heating by the heater, the silicon blocks start melting, and the liquid surface temperature gradually increases. Once most of the silicon blocks were melted into liquid, the liquid surface temperature will rise suddenly. Then, based on artificial experience, the melting is terminated after a while. FIGURE 2 shows the change curve in the surface temperature of the material. Typically, satisfying the following three conditions could refer to the end of the melting stage: (1) the temperature is above 1420 • C; (2) waiting for a while after the temperature has been suddenly rising up; and (3) the contiguous time difference is less than a given temperature threshold.
2. The image features are used to determine the complete melting [17]: The image features are significantly different in case of complete melting. The melt images of the crucible in the furnace are obtained through the charge coupled device (CCD) camera above the furnace body. Then, the changes in the image features have been analyzed to determine the end of melting process. The principles are shown in FIGURE 3.
The most commonly used method is the angle point detection of images, which reflects the expected process by the number of angle points.
The above analysis shows that to improve the quality of the crystal, avoid accidents, and reduce energy consumption, it is critical to classify the melting process of the silicon material and determine the complete melting of the silicon material. Therefore, we used a deep learning method to classify the melting completeness of the silicon material. Also, it used data to automatically abstract features, thereby avoiding a series of complex operations and implementing classification tasks through a classifier. The classification scheme was shown in FIGURE 4.  First, the input images should be collected. Meanwhile, the training and test sets were constructed. Then, the network training was performed on the images in order to obtain image features. Through the classifier, a classification model was obtained. Finally, the classification results were predicted through the classification model. In the melting stage, by analyzing the melting detection process and the problems that might occur, the traditional detection methods of the melting process were learned. The current applied deep learning methods were chosen to complete the image feature extraction and classification tasks.

A. IMAGE DATASET
The image data were obtained from the polycrystalline silicon melting images. The images were taken by a CCD camera in the crucible above a monocrystal furnace body, provided by the Xi'an University of Technology. The data information was in video format. In this study, the middle-late and completion stages of the melting process were selected for classification research. On the one hand, the stages of the melting process were relatively stable. On the other hand, during the late completion stages of the melting process, according to the results of real-time classification, the heater power could be reduced to start the seeding stage.
The video information was converted into image data. Parts of the image data were selected as the research objects, including the unmelted and completely melted silicon material, as shown in FIGURE 5. Meanwhile, the data were divided into a training set and a test set [18], as shown in Table 1:

B. THE DESIGN OF CNN CLASSIFICATION MODEL
A classification model of the melting stage was constructed based on the AlexNet network. The final network structure was determined experimentally. The number of convolution layers and the size of the convolution kernel in the network structure were adjusted to determine the final classification model of the melting stage. Please see the detailed process in the Part IV.
The structure contained four convolution layers and two fully-connected layers. The convolution layer consists of two parts, one is the convolution layer, the other is the pooling layer (sometimes this layer is not needed). The convolutional layer has powerful feature extraction capability, while the pooling layer can reduce the dimension of feature graph and simplify the computational complexity of network. The full connection layer integrates the extracted feature information and reduces the two-dimensional information to one-dimensional information, which is convenient to realize the final classification. The activation function of the convolution layer was ReLu, and the output layer was a softmax classifier. Each layer of the network was described in details as below.
The input layer: A pixel matrix of 227 × 227 × 3 has been obtained by processing the original image.
The first convolution layer: The first layer of this section is a convolution layer. This layer used a total of 64 convolution kernels with a size of 11 × 11 and a sliding step of 4. Since the input image had an RGB three-channel pixel value, the 64 convolution kernels in this layer were also three-channel. Therefore, the size of the feature map after the convolution of this layer was (227 − 11) /4 + 1 = 55. Finally, 64 feature maps of 55 × 55 were obtained.
The second was the pooling layer. The pooling operation was performed on the 64 feature maps of 55 × 55 obtained by the convolution. The pooling window size was 3 × 3, the sliding step was 2, and the size of pooled feature map size was (55 − 3) /2 + 1 = 27. Finally, 64 feature maps of 27 × 27 were obtained.
The second convolution layer: The second layer of convolution used 64 convolution kernels with a size of 3 × 3 and a sliding step of 1. The zero complement operation was performed. Therefore, the size of the feature map after the convolution of this layer was (27 − 3 + 2 × 1) /1 + 1 = 27. Finally, 64 feature maps with a size of 27 × 27 were obtained.
The second was the pooling layer. The pooling operation was performed on the 64 feature maps of 27 × 27 obtained by the convolution. The pooling window size was 3 × 3, the sliding step was 2, and the size of pooled feature map size was (27 − 3) /2+1 = 13. Finally, 64 feature maps of 13×13 were obtained.
The third convolution layer: The third layer of convolution used 64 convolution kernels with a size of 3 × 3 and a sliding step of 1. The zero complement operation was performed. Therefore, the size of the feature map after the convolution of this layer was (13 − 3 + 2 × 1) /1 + 1 = 13. Finally, 64 feature maps of 13 × 13 were obtained. This layer contained no pooling layers.
The fourth convolution layer: The fourth layer of convolution used 64 convolution kernels with a size of 3 × 3 and a sliding step of 1. The zero complement operation was performed. Therefore, the size of the feature map after the convolution of this layer was (13 − 3 + 2 × 1) /1 + 1 = 13. Finally, 64 feature maps with a size of 13 × 13 were obtained.
The second was the pooling layer. The pooling operation was performed on the 64 feature maps of 13 × 13, which was obtained by the convolution. The pooling window size was 3 × 3, the sliding step was 2, and the size of pooled feature map size was (13 − 3) /2 + 1 = 6. Finally, 64 feature maps of 6 × 6 were obtained.
The first fully-connected layer: The first fully-connected layer used 512 neurons. First, the 64 feature maps of 6 × 6 obtained after the above 4 convolutions were flattened into a one-dimensional vector, which corresponded to 2,304 neurons. Similar to the back propagation (BP) neural network, the 512 neurons were fully connected.
The second fully-connected layer: Since it was a binary classification task, the second fullyconnected layer used 2 neurons. The 512 neuron output FIGURE 6. Melting stage classification model based on CNN. In the figure, C represents the convolutional layer, P represents the pooling layer, and F represents the fully-connected layer. The number before @ indicates the number of feature maps, and the number after the sign indicates the size of a single feature map. Kernel represents the size of convolutional kernel, Stride represents the sliding step, and Padding represents the size of zero complement operation.
obtained through the first fully-connected layer was fully connected with the 2 neurons in this layer. The softmax was used to obtain the probability value output by each neuron, i.e., the probability of belonging to each category.
The structure of the CNN classification model was shown in FIGURE 6. The pooling layer in the figure was not specifically indicated. The detailed determination process of the classification model was elaborated in the experiment.

IV. ANALYSIS OF SIMULATION RESULTS
At present, targeting at a specific problem, no rules have been given in designing a classification network structure. The network structure that is most suitable for the dataset can only be found in experiments. Therefore, to determine the appropriate classification model of the melting stage, adjustments were made on the AlexNet network structure. The adjustments mainly focused on two aspects, i.e., the number of convolution layers and the size of the convolution kernel. Therefore, the CNN model with the highest classification accuracy was obtained. During the experiment, the learning rate was 0.0001, and the batch size was 24. The number of fully-connected layers was set to 2, and dropout = 0.5 was used to train the first fully-connected layer. The two main experimental results were given below.

1) NUMBER OF CONVOLUTION LAYERS
For the different number of convolution layers, the network structure of the original AlexNet network was, in which 11, 5, 3, 3, and 3 respectively represented the size of the convolution kernel of each convolutional layer. The sampling layer was ignored. The AlexNet network was directly used for the classification of the melting stage. The loss curve was shown as the black curve in FIGURE 7. The network converged when iterating 10 epochs. The convergence speed was fast, and the accuracy rate reached 97.50%. After the network was adjusted as four convolutional layers of 11-5-3-3, the loss curve was shown as the blue curve in FIGURE 7. The network converged when iterating 8 epochs. The convergence speed was slightly faster than the AlexNet network, and the  accuracy rate was 97.25%. Table 2 showed the classification accuracy of the two network structures.
By taking the experiments of adjusting the number of convolution layers in the network, the accuracy of the original AlexNet network for melting stage classification reached 97.50%. The adjusted network could also converge, and the classification accuracy could reach 97.25%. The accuracy rates of the two networks were similar, but the adjusted network structure converges faster. Therefore, the network structure of 11-5-3-3 was selected as the basis for subsequent experiments.

2) SIZE OF THE CONVOLUTION KERNEL
For the different sized convolution kernel, the final network structure determined in the above experiment was 11-5-3-3. This experiment adjusted the convolution kernel size in the network and adjusted the network to 11-3-3-3. Experiments were performed under these conditions. The change in the final network loss value was shown as the red curve in FIGURE 8. The network converged at approximately 10 epochs, with an accuracy rate of 98.5%, which was 1% higher than the accuracy of the previous network structure. Table 3 showed the classification accuracy of the two network structures.
The above-mentioned increasing as for the accuracy was most likely resulted by the smaller convolution kernel, which causes the convolutional network to extract more key features of the melting stage. Therefore, the network had better generalization ability and a higher accuracy rate. In summary, through two sets of comparative experiments, a network structure with a classification accuracy of 98.5% was selected as the final classification model of melting stage, i.e., the convolutional network had 4 convolutional layers, and the convolution kernel size of each convolutional layer was 11-3-3-3.

V. CONCLUSION
This study classified the melting states for the silicon in the material melting stage. Despite that the crystal growth process is in a closed, high temperature, vacuum and complex environment, which is a great obstacle to observe whether the silicon is melted. This paper adopts CNN strategy to judge the melting completeness for silicon melting stage. This method can replace the traditional manual experience judgment and can more accurately infer the time of complete melting in addition to shorten the time cost and prolong the service life of the crucible and other components.
The CNN-based strategy proposed in this paper for feature extraction of the free surface can not only effectively avoid the ''splash'' caused by collapsing material, but also reduce the concentration of oxygen in the furnace body, thus reducing the corrosion of the inner wall of quartz crucible and improving the service life of the crucible. In addition, the melting process of silicon material is reasonably predicted, so as to judge whether the silicon material has been fused. The original material melting images were used as input to extract the network features. Finally, the classification was completed through the classifier. By adjusting the AlexNet network, a classification network structure more suitable for the melting stage was obtained, which was mainly reflected in the following two aspects: (1) The number of convolutional layers was adjusted. The AlexNet network with five convolutional layers of 11-5-3-3-3, as well as the adjusted four-layered convolutional network of 11-5-3-3, were used to classify the melting images. The simulation experiments confirmed that the network structure of 11-5-3-3 converged slightly faster and the accuracy reached 97.25%.
(2) Second, the size of the convolution kernel was adjusted. The 11-5-3-3 network and the adjusted 11-3-3-3 network were used to classify the melting images. The simulation experiments determined that the accuracy rate of the 11-3-3-3 network structure reached 98.5%. Finally, the CNN model for the melting process classification was determined to be 11-3-3-3.
Experimental results show that the proposed method (the model) can predict the melting process of silicon; and the accuracy is 98.5%, which is feasible.