Air Quality Measurement Based on Double-Channel Convolutional Neural Network Ensemble Learning

Environmental air quality affects people's life, obtaining real-time and accurate environmental air quality has a profound guiding significance for the development of social activities. At present, environmental air quality measurement mainly adopts the method that setting air quality detector at specific monitoring points in cities and timing sampling analysis, which is easy to be restricted by time and space factors. Some air quality measurement algorithms related to deep learning mostly adopt a single convolutional neural network to train the whole image, which will ignore the difference of different parts of the image. In this paper, we propose a method for air quality measurement based on double-channel convolutional neural network ensemble learning to solve the problem of feature extraction for different parts of environmental images. Our method mainly includes two aspects: ensemble learning of double-channel convolutional neural network and self-learning weighted feature fusion. We constructed a double-channel convolutional neural network, used each channel to train different parts of the environment images for feature extraction. We propose a feature weight self-learning method, which weights and concatenates the extracted feature vectors, and uses the fused feature vectors to measure air quality. Our method can be applied to the two tasks of air quality grade measurement and air quality index (AQI) measurement. Moreover, we build an environmental image dataset of random time and location condition. The experiments show that our method can achieve nearly 82% accuracy and a small mean absolute error (MAE) on our test dataset. At the same time, through comparative experiment, we proved that our proposed method gained considerable improvement in performance compared with single channel convolutional neural network air quality measurements.


Introduction
Environmental air quality is closely related to human production and life.The decline of air quality is likely to cause ecological damage and induce human diseases.At present, air quality monitoring mainly adopts the method of setting up monitoring stations in several specific locations in the city, using the air quality detector to regularly sample and measure air pollutants, and finally obtaining the air quality index through calculation and analysis.This method is easy to be limited by time and space, can only obtain air quality at specific monitoring points in specific time.It is arXiv:1902.06942v3[cs.CV] 19 Feb difficult to obtain the air quality information of the random location in real time, and the measurement cost is high.How to obtain the air quality index in real time and accurately is a subject worth studying.
Image-based air quality measurement is a method that use image processing algorithm to extract environmental image features and estimate air quality index based on image features.In recent years, with the rapid development of deep learning technology, using deep learning technology to complete identification, detection and other tasks is efficient.Environmental images under different air quality grades are often different to some extent, therefore, it is feasible and valuable to use deep convolutional neural network to extract features of environmental images and complete the measurement of real-time air quality index at random site.Compared with the traditional air quality measurement method, the air quality measurement based on image and deep learning can obtain the air quality at any time and any place, which has the advantages of real-time and low cost, has been widely concerned by the academic circle in recent years.
At present, the existing air quality measurement methods related to image or deep learning are mainly divided into two types: method based on traditional image processing or deep learning.The methods which based on traditional image processing [1,7], are use traditional machine learning algorithms for feature extraction, such as edge detection, direction gradient histogram, etc.The extracted features are analyzed and calculated to get air quality measurement values.The imagebased deep learning methods [2,3,4,5]  The main contributions of this paper: 1. We construct a double-channel convolutional neural network structure to perform feature extraction for different parts of the environmental image.
2. We propose a weighted feature fusion method, and a feature weight self-learning method to select excellent features.
3. We apply the double-channel convolutional neural network and weighted feature fusion to the classification and regression tasks, complete the tasks of air quality grade measurement and air quality index measurement.

4.
Through experiments, we prove the effectiveness of the proposed method, and demonstrate the influence of different weights and different network structure on system performance.
The rest of this paper is organized as follows.Section 2 is related work, mainly introduces the development of deep learning and related research on air quality measurement.Section 3 introduces the air quality measurement algorithm based on double-channel convolutional neural network ensemble learning, mainly introduces the structure of the double-channel convolutional neural network, the weighted feature fusion and the self-learning method of feature weight.Section 4 is the experimental part, mainly introduces our training and testing methods, shows our experimental results, and compares our system with different network structure, different feature weights ratio, analyzes the existing problems.Section 5 is the conclusion, which concludes the paper and describes the main direction of the future work.
The research on deep learning can be traced back to 1989, when LeCun applied BP algorithm to multi-layer neural network [8].With the LeNet-5 model proposed by LeCun in 1998 [9], the basic structure of deep neural network was formed.In 2006, Geoffrey Hinton, a professor at the university of Toronto, formally proposed the concept of deep learning [10], and deep learning has entered a period of rapid development.Alex proposed AlexNet in 2012 [11], built the first big convolutional neural network, and adopted the ReLu activation function instead of Sigmod, managed to avoid the problem of gradient disappeared in neural network training, the performance of image recognition is much better than traditional methods.VGG [12], GoogleNet [13], ResNet [14] and other network

Weighted Feature Fusion and Weights Self-Learning
We found in our observation that, considering the images input of two channels, the content of some images of the sky is relatively simple, generally simple sky, or containing a small number of clouds or trees, the image complexity is relatively low; The content of the building is relatively rich in composition, and there are a variety of different buildings, roads and trees, etc., and the image complexity is relatively high.Due to the different complexity of the two images, the feature complexity extracted by the two channels is also different, and the effect weight on the final measurement result is different.
Therefore, considering that the image features of the upper and under channels may have different proportion of influence on the measurement results, we propose a method of weighted feature fusion.Before sending the output features of the two feature layers into the classification layer, the weighted feature fusion is carried out first.The weight value is multiplied by the output feature vectors of the upper and under channels by two constants, and then the two vectors are concatenated.The formula of feature fusion is as equation ( 1): Where When training the feature weight, the objective loss and the weight loss constraint function are combined to form the joint loss function, optimizing joint loss function to adjust the weights parameter value.Finally, multiply the two weights obtained after training a by the feature vectors extracted from each channel, and concatenate two weighted features.

Air Quality Measurement
For air quality measurement tasks, we start from the two directions of classification and regression, consider applying our double-channel convolutional neural network in two aspects, air quality grade measurement and air quality index measurement.

Air Quality Grade Measurement
Air quality grade measurement is essentially a classification and recognition task.According to the 6 grades of air quality, the corresponding environmental images are divided into 6 categories and classified in the fully connection layer.Softmax was used for the activation function to conduct one-hot operation on all kinds of labels to obtain the predicted probability value of each grade, and the maximum probability was taken as the measurement result of each grade.At the same time, we put forward a calculation method of air quality index according to the prediction probability of each grade, like equation (3): Where   and   are respectively the upper and lower limits of the predicted grade air quality index, and  is the predicted probability of the predicted grade.According to the calculation, we can get the calculated value of air quality index.

Air Quality Index Measurement
Based on the idea of regression, we consider the direct measurement of air quality index.
Therefore, we add a 1-node fully connection layer after the above double-channel convolutional neural network, the AQI value corresponding to the environment image was used as the training label to conduct regression training.With directly measure the air quality index, we can calculate the air quality grade according to the air quality index value.The loss function adopts the mean square error between the predicted value and the labeled value: Where  is the number of training images,   is the labeled value of the ith image air quality index, and (  ) is the predicted value.

Dataset
In order to establish an effective dataset, we used the method of manual collection, to randomly  of feature fusion layer are frozen, and the negative feedback stochastic gradient descent is adopted to train other network parameters of the double-channel neural network.At the same time, we adopted dropout [15] with probability of 0.5 to prevent network overfitting at the last convolution layer of each channel convolutional neural network.

Feature Fusion Weight Training
When the training of the double-channel convolutional neural network meets the requirements and the loss function value no longer decreases significantly, we stopped the first step of training and froze the network parameters.Next, only two fusion weights of feature fusion layer were trained, and negative feedback stochastic gradient descent was used to update  1 and  2 .After a certain number of iterations, the training was completed.
Training environment configuration, we adopt Intel Xeon e5-2650 v3@2.30GHzCPU, NVIDIA Tesla K40c GPU hardware environment; In terms of software, we adopt TensorFlow1.10.0 deep learning framework and Python3.5 programming language.In terms of training Settings, batch-size was 128, learning rate was 1-e4, the training period was about 3300 epochs, and the number of iterations was 11,000.The first 10,000 times are used to train the parameters of the double-channel convolutional neural network.The last 1000 iterations are used to adjust the feature weights.

Test Methods and Evaluation Criteria
We used test-time augmentation during the testing.That is, random crop was also used in the test, and voting mechanism and average mechanism were introduced.In the test, the above image preprocessing is carried out on the test image first, and each image is subject to 20 times of random crop without random horizontal transformation.For each image, we get 20 groups test data to feed into the trained model for prediction.Finally, 20 predicted results were obtained for each image.For the task of grades classification, the voting mechanism was adopted to take most of the predicted class grades in the 20 predicted results as the final prediction classification.For the AQI measurement task, the average mechanism was adopted, and the mean value of 20 predicted values was taken as the measurement result.
Two evaluation criteria, mean accuracy and mean absolute error (MAE), were used to evaluate the classification accuracy.The mean accuracy is shown in the following formula (5), that is, the ratio of the predicted correct sample number to the total sample number.where  is the total number of test samples,    is the ith sample predicted grade,    is the ith sample labeled grade.
At the same time, because of the particularity of problem, labeling information collected from the nearest location of stations as well as the time is the most closed to the hour of the air quality index value, it is difficult to obtain the location accurate air quality index value, so the annotation information itself has a little error; In addition to the limitation of time and space, the measurement error of the measuring instrument itself makes the images of different grades at the critical point of air quality grade also have the problem of inaccuracy, and the difference is slight.Therefore, we use the mean absolute error (MAE) as the second evaluation standard, that is, to calculate the mean value of the absolute value of the difference between the predicted grade and the true grade of each sample.The formula is as equation ( 6): For the air quality index regression problem, MAE is also used as the evaluation criteria to measure the mean deviation between the measured value and the real AQI value.At the same time, we introduce the mean deviation rate (MDR) as another evaluation criteria of index.The formula of MDR is as equation (7).Where    is the ith a sample air quality index predicted value,    is the ith a sample air quality index of true value.As can be seen from table 5, system performance increases and decreases for different feature weights.For the improper feature weights, the performance of the system is obviously decreased compared with DCEW-C.Relatively speaking, some performance criterions have been improved

MDR =
with more appropriate feature weights.Therefore, adopting the method of weight self-learning is beneficial for the system to automatically find the appropriate feature weight.The double-channel convolutional neural network using the weight self-learning method (DCSLW-C) has improved its performance in terms of accuracy and MAE, compared with DCEW-C and the assigned weights feature fusion method.shown in figure 6.On the whole, the classification method using the double-channel convolutional neural network shows obvious advantages in terms of the accuracy of grade measurement, and the regression method using the double-channel convolutional neural network obtains lower MAE.This advantage is further enhanced with the adoption of self-learning weights.

Figure6. Algorithms performance comparison
Meanwhile, in the experiment, we also analyzed the samples that testing failed.DCSLW-C and DCSLW-R partial measurement wrong samples as shown in figure 7. We found that most wrong samples were high air quality grade samples, and the difference between these images with different grades was small, and there was a similarity with the error category.At the same time, due to the label limitations mentioned above, the labels were approximately collected at the nearest time and place monitoring points, so there are certain errors in the label itself, which are also part of the reasons for the measurement errors.
Figure7.Some measurement wrong images

Conclusion
In this paper, we propose an air quality measurement algorithm based on double-channel convolutional neural network ensemble learning, design a double-channel convolutional neural network structure to measure the air quality grade and index of environmental images.At the same time, we propose a self-learning method of weighted feature fusion.Experimental results show that our method is feasible, achieve certain accuracy and small mean absolute error.On the basis of the double-channel convolutional neural network, the performance is further improved by using the weighted feature fusion method.Compared with other methods, our method achieves better performance.Meanwhile, it is found in the experiment that it is difficult to distinguish the images of adjacent grades and the images of different grades with similar contents.How to recognize such samples will be an important direction of future work.

Figure 1 .
Figure 1.Air quality measurement based on double-channel convolutional neural network ensemble learning algorithm diagram

3 Conv1a3. 1
structures were proposed one after another, which further enhanced the feature extraction ability of deep convolutional neural network.Using deep convolutional neural network to extract image features and complete classification and recognition has become the primary choice and important research direction for more and more researchers.In essence, air quality measurement is an imagebased classification or regression task.Deep convolutional neural network can be used to effectively extract environmental image features and complete the identification task of air quality as shown in the image.In recent years, the measurement of air quality using deep learning method has attracted much attention in academic circles.In the study of air quality measurement related to deep learning, Chao Zhang[2] built a convolutional neural network, improved the convolutional layer activation function and classification layer activation function, proposed an improved activation function of convolutional neural network EPAPL, and used a Negative Log-Log Ordinal Classifier to replace softmax Classifier in the classification layer, used the environment image to train their network model for classification prediction, completed measurement task of  2.5 and  10 in six grades; Avijoy Chakma et al [3] used convolutional neural network training images for feature extraction, combined with random forest classification, and classified the air quality shown in the images into three grades of good, middle, and bad.Nabin Rijal [4] adopts a method of neural network ensemble learning, they used three different convolutional neural networks, VGG16, InceptionV3 and Resnet50 to respectively conduct regression training on image  2.5 values.The predicted values of the input  2.5 of the three networks were input as feature into a feedforward network fortraining to predict the image  2.5 values.Jian Ma[5] combined the dark channel prior theory[6], firstly extracted the dark channel from the image, trained two convolutional neural networks respectively with the original image and the dark channel images, and identified the good and bad air quality of the image in three grades.Xiaoguang Chen et al[7] proposes a traditional image processing algorithms and deep learning combining approach.First, they counted distribution characteristics of image pixel values, statistics the proportion of high brightness points (pixel value > 128) to all pixel points of each image, use edge detector statistics all image the proportion of edge points to all pixels.The two values of each image as input feature to train the BP neural network, to predict air quality index value.Considering the different composition information of different parts of the environmental image, we constructed a double-channel convolutional neural network based on the method of deep convolutional neural network and the idea of ensemble learning to extract features from different parts of the image, proposed a double-channel convolutional neural network ensemble learning algorithm for air quality measurement.3Double-Channel Convolutional Neural Network Ensemble LearningAlgorithm for Air Quality MeasurementAlex proposed AlexNet[11] has achieved good performance in image recognition tasks, in view of the air quality measurement in essence for image recognition tasks, therefore, on the basis of AlexNet[11], we constructed a double-channel convolutional neural network for feature extraction of the sky and building parts of the environmental image, weighted and fused the extracted features, proposed a double-channel convolutional neural network ensemble learning algorithm for air quality measurement.It is composed of two feature extraction convolutional neural networks, a weighted feature fusion layer and a classification layer, as shown in table 1. Table1.Double-channel convolutional neural network structure Upper channel image 64*64*3 Under channel image 64*64*Double-Channel Convolutional Neural Network Structure The structure of the double-channel convolutional neural network is shown in table 1.It is composed of upper and under channel sub-convolutional neural networks, each channel convolutional neural network contains five convolution layers, two pooling layers and one fully connection layer.The first three convolution layers adopt 5*5 convolution kernel, and the last two convolution layers adopt 3*3 convolution kernel for feature extraction of image; Maximum pooling is used in each pooling layer to extract important features from down sampling; The 512-nodes fully connection layer is used to output feature vectors extracted from each network for feature fusion and prediction.For different components of the environment image, the double-channel convolutional neural network adopts the strategy of ensemble learning to receive different parts of the image simultaneously in the upper and under channels for training.Before inputting the environment image into the double-channel convolutional neural network for training, the environment image should be preprocessed first, and the partial sky image and the partial building image should be segmented.For each image, the horizontal central axis average segmentation method is adopted to divide the image into the image mainly containing the upper half of the sky and the image mainly containing the under half of the building.Among them, the upper channel convolutional neural network focuses on feature extraction of the sky.In each round of iterative training, the images of the upper half with more sky elements after cutting are input into the upper channel convolutional neural network for training; The convolutional neural network of the under channel focuses on feature extraction of the building part.In each round of iterative training, the image of the under half with more building elements after cutting is input into the convolutional neural network of the under channel for training.After feature extraction at the fully connection layer of the last layer of each subnetwork, the feature vectors of the upper and under parts were weighted and fused, and the feature vectors containing complete features of the upper and under channels were used for recognition.
1 and  2 are weight values of upper and under channels respectively;   and   are feature vectors extracted by upper and under channels respectively;  is global features after feature fusion.On the basis of artificially assigning feature weights, we propose a self-learning method for feature weights, which also participate in training.In the initial stage, two weights  1 and  2 are set to 0.5 by adopting the strategy of balancing weights.In the training of the double-channel convolutional neural network, we only train other network parameters of the double-channel convolutional neural network with the weights value is frozen.After the training of the doublechannel convolutional neural network, other parameters of the network should be frozen and the two weights are trained to find the appropriate feature ratio.In feature weights training, considering the proportional relationship between the two weights, we propose a weight loss constraint function to limit the training of weight values.The weight loss constraint function is defined as equation (2):

Figure3. Dataset air quality index distribution 4
Figure3.Dataset air quality index distribution

4. 3 . 4
Double-Channel Convolutional Neural Network Air Quality Index Measurement Performance Analysis In addition, from the perspective of regression, we conducted experiments on the direct measurement of the air quality index (AQI) with double-channel convolutional neural network, and classified the images to corresponding grade according to the predicted results.Similarly, we respectively used double-channel self-learning weight for regression (DCSLW-R)、double-channel equal weight for regression (DCEW-R)、single channel for regression (SC-R) for AQI measurement experiments.The experimental results of AQI prediction are shown in figure 4. For the task of AQI measurement, the performance difference between the double-channel convolutional neural network with equal weight and self-learning weight is small, and they have respective advantages in MAE and MDR.It can be seen from the figure 4 that in the AQI measurement, the prediction of lower air quality index is more accurate, and the accuracy decreases with the increase of the index.Compared with single channel convolutional neural network, the measurement error of double channel convolutional neural network is obviously lower.The partial AQI prediction results of DCSLW-R are shown in figure 5.

Figure4.
Figure4.AQI measurements result environmental images at different time of each day in the Beijing area, established a In the study of this task, we screened the dataset.Due to the quality problem of the image itself, we manually removed the images with poor image quality, poor weather conditions, and inappropriate shooting time, such as sunset time light is poor, night by the influence of street lights.Considering the unbalanced dataset samples, images with low air quality index is much more than images with high air quality index, easy to cause an effect to training, makes the final model can be the index of high image recognition rate is too low, so we chose a relatively evenly number of each grades, finally formed a with 567 images dataset, partial sample image is shown in figure2.Among them, 465 training images and 102 test images were included.The number of images of each category is shown in table 3, and the distribution of AQI is shown in figure 3. Table3.Dataset air quality grade distribution

2 Double-Channel Convolutional Neural Network Air Quality Grade Recognition Performance Analysis
Based on the proposed double-channel convolutional neural network, we first tested model performance on recognition air quality grade (double channel equal weight for classify, DCEW-C) under the condition of equal weight, at the same time, we use the following methods for performance comparison: (1) single channel convolutional neural network training and testing on the whole image (baseline); (2) only the upper channel convolutional neural network is used for training and testing the upper part of the image; (3) only the under channel convolutional neural network is used for training and testing the under part of the image.The test results of the above methods are shown in table4.Due to the large similarity between adjacent air quality grade images, we also introduced the Neighbor accuracy as another reference It can be seen from table 4 that the method using double-channel convolutional neural network is much better than the method only applied a single convolutional neural network in terms of accuracy, neighbor accuracy and MAE.For the single-channel convolutional neural network, the under channel convolutional neural network has achieved a high accuracy, but due to the complexity of the under half of the image, its neighbor accuracy is poor.For the single-channel convolutional neural network, due to the extraction of the whole image information, it performs better in the neighbor accuracy.The image features of the upper channel convolutional neural network are relatively simple, so it obtained the lower MAE.For the proposed double-channel convolutional neural network (DCEW-C), because it takes into account different parts of the image information, and adopts the strategy of separately extracting ensemble learning, it has achieved a great performance improvement compared with the singlechannel convolutional neural network.Compared with the optimal performance of each single channel convolutional neural network, the accuracy was improved by more than 6 percentage points,

3 Double-Channel Convolutional Neural Networks with Different Feature Fusion Weights Performance Analysis On
the basis of the equal weight double-channel convolutional neural network for air quality grade recognition, we explore the performance under the weighted fusion of two-channel features.Considering from two aspects, we have conducted experiments with different assigned weights and with double-channel self-learning weight for classify (DCSLW-C).For the weight assignment, we studied the system performance with the feature weight ratio of upper and under channels at 3:7, 4:6, 5:5, 6:4 and 7:3 respectively.The experimental results are shown in table5.