Design of Multiband Switching Illumination With Low-Concentration Lugol Stain for Esophageal Cancer Detection

Narrowband imaging (NBI), flexible spectral imaging color enhancement (FICE), and Lugol chromoendoscopy (LCE) are used in endoscopy of head and neck cancers and esophageal cancers. The NBI and FICE techniques enhance the capillary pattern in cancerous areas, whereas in LCE, normal and cancerous areas are differentiated using Lugol solution. Although invasive, LCE is more sensitive than NBI and FICE for this purpose. In this study, we propose a relatively less invasive and highly sensitive technique for detecting cancerous areas that combines low-concentration Lugol stain and narrowband illumination. This combined technique requires the use of an endoscopic system that can acquire RGB images synchronized with narrowband illumination switching. This technique produces a large number of images that can potentially discriminate smaller differences in color than is possible with conventional RGB imaging. Experiments to predict the performance of the proposed method were performed using four resected specimens, including esophageal cancer, and promising results were obtained.


I. INTRODUCTION
Head and neck (oral, larynx, and pharynx) [1] cancers and esophageal cancers are the 6th and 7th most common cancers worldwide, respectively [2]. They appear superficially in the mucous membrane and gradually infiltrate the underlying tissue [3], [4]. If cancers on the tissue surface can be treated at the early stage, the prognosis of patients is improved [5], [6].
In the clinical field, there are three methods capable of identifying cancers at the early stage: narrowband imaging (NBI) developed by Olympus Corp. (Tokyo, Japan), flexible spectral imaging color enhancement (FICE) developed by Fujifilm Corp. (Tokyo, Japan), and Lugol chromoendoscopy (LCE) [7]. Among these, NBI enhances the capillary pattern in cancerous areas using blue and green narrowband illumination, with wavelengths of 390-445 nm and 530-550 nm, respectively [8]. FICE depicts cancerous areas as regions The associate editor coordinating the review of this manuscript and approving it for publication was Wenming Cao . of high contrast. It uses spectral reflectance (SR) images in the visible light region estimated by the Wiener estimation method from RGB images acquired under conventional white light (CWL) [9]. NBI is used more commonly than FICE because the spectral estimation method of FICE cannot effectively enhance the capillary pattern in the cancerous area, and thus NBI is considered more suitable for the early detection of cancer [10]- [13]. In contrast, in the LCE technique, normal areas are stained with Lugol solution and cancerous areas remain unstained. After Lugol solution is sprayed onto tissues of the head and neck, or esophagus, iodine molecules in the solution react with glycogen in normal tissue, which stains normal areas black, dark brown, or green-brown. The staining reaction is weak in cancerous areas because glycogen has already been consumed by the cancer cells. Thus, this technique clarifies cancerous and normal areas [14]. LCE is much more sensitive and reliable for detecting cancerous areas compared with NBI and FICE [15], [16]. However, LCE imposes a higher burden on patients than NBI and FICE VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ FIGURE 1. Overview of the proposed multispectral imaging system with switching between two sets of RGB illuminations.
because Lugol has adverse effects such as severe chest pain and chest discomfort [17]. Diluting the Lugol solution to reduce the patient burden reduces the ability to discriminate cancerous areas [15]. To take advantage of the high sensitivity of LCE and reduce the patient burden, discrimination performance can be improved via modification of the imaging system to enrich color information. Enriched color information can be obtained by spectral imaging or multiband imaging. Various endoscopic systems capable of acquiring spectral information for lesion diagnosis have been developed. In the fiberscope system, light passing through a fiber optic image guide is diffracted [18], [19], enabling more than tens of bands of images to be acquired in real time. However, the spatial resolution is lower than that of commonly used electronic endoscopes [20]. Leitner et al. proposed a hyperspectral imaging endoscopic system in which fast acousto-optical tunable filters are synchronized with a highly sensitive electronmultiplying CCD camera and a rigid endoscope [21]. This system can acquire eight bands of images in real time; however, tissues cannot be observed in all locations because the endoscope is not flexible.
In this study, we modified a conventional electronic endoscope to capture multiband images (MBI) that can detect the slight color difference between cancerous and normal areas stained by a low-concentration Lugol solution. We employed rapid switching of two or three units of light-emitting diode (LED) light sources and predicted the improvement in the discrimination performance.

II. STUDY SCHEME AND SWITCHING ILLUMINATION DESIGN
A. STUDY SCHEME To acquire MBI using an endoscope, we integrated multiple three-band illuminations (3-BI) into the endoscope. Fig. 1 shows a systematic diagram of the proposed system with two different 3-BIs. Each 3-BI is composed of red, green, and blue LEDs with narrowband light-emission characteristics. Three-band images are obtained with an RGB detector under each illumination. Six-band images can be acquired by switching the two 3-BIs at high speed. In this paper, we assume acquisition of six-or nine-band images using two or three 3-BIs. For example, for a camera frame rate of 30 fps, it is possible to obtain six-and nine-band images at 15 and 10 fps, respectively. After acquiring MBI, cancer discrimination is performed based on a machine learning method.
A flow chart of the study is shown in Fig 2. In the measurement step, hyperspectral images (HSI) of four esophageal cancer specimens were captured using a hyperspectral camera. When designing the illumination step, we calculated pseudo-RGB values based on the spectral reflectance (SR) of cancerous and normal areas randomly extracted from the HSIs. We then designed multiband switching illuminations (MBSIs) composed of multiple 3-BIs.
In the evaluation step, we validated the effectiveness of the proposed system. We first used the designed MBSI to generate a pseudo-RGB image that contains spectral information for each pixel. For comparison, other pseudo-RGB images were generated under CWL conditions and the illumination conditions used in NBI. Cancer discrimination was then performed based on the generated RGB images, and the discrimination performance was compared among all of the generated RGB images.

B. SWITCHING ILLUMINATION DESIGN METHOD
With the aim of discriminating cancerous areas with high accuracy, we designed MBSIs that included two or three 3-BIs. Fig. 3 shows the MBSI design procedure.
First, from the HSIs of each of four specimens, we extracted K SRs in cancerous and in normal areas. Each of the 3-BIs in the MBSI was set by selecting three LEDs from N kinds of LEDs. Based on the peak wavelength of N LEDs, these were divided into N b blue LEDs, N g green LEDs, and N r red LEDs.
Each 3-BI has one blue, one green, and one red LED. As one LED of each color is selected, the total number of the 3-BI combinations is N b N g N r . Therefore, in the case that two or three 3-BIs are used in constructing an MBSI, the total number of combinations with two 3-BIs is N b N g N r C 2 and that with three 3-BIs is N b N g N r C 3 .
Second, the RGB values are calculated as the sum of the spectral product of SR, the camera sensitivity, and intensity distributions of illuminations according to the following equations: where I reflect (λ) is the extracted SR from HSI. E i (λ) is the intensity distribution of the i-th 3-BI among N b N g N r possible 3-BIs. S R (λ), S G (λ), and S B (λ) are the camera sensitivities of red, green, and blue sensors, respectively.
Third, all data (4K ) for each of cancerous and normal areas are divided into training data and test data. We construct a discrimination model using support vector machine (SVM) [22] based on the training data, and apply it to the test data. We use the calculated RGB values as the feature vector for constructing the discrimination model.
Fourth, to evaluate discrimination accuracy, we calculate accuracy based on true positive (TP), true negative (TN), false positive (FP), and false negative (FN), according to the following equation: These procedures are repeated for all combinations of 3-BIs. The MBSI giving the maximum accuracy is then determined.

A. MEASUREMENT EXPERIMENT
Four endoscopically resected specimens of early esophageal cancer were obtained from four patients treated at Chiba University Hospital. Before acquiring HSI of the specimens, we prepared low-(0.2%) and normal-concentration (1.0%) Lugol solution. The low-concentration solution was diluted with distilled water [23]. The HSI acquisition protocol was as follows.
We acquired HSI of these specimens in the order of no stain, 0.2% Lugol stain, and 1.0% Lugol stain. HSI of the 0.2% stained specimens was used in designing the MBSIs and system evaluation and the 1.0% stained specimens were used as the gold standard for detection of cancerous areas.
We then acquired HSI of a standard white diffuser with 99% reflectance for calculating SR, which is used for designing the MBSIs and generating the pseudo-RGB images. SR  images, I reflect (x, y, λ), were calculated using the following equation: Here (x, y) represents the spatial coordinates of pixel position. I (x, y, λ) and W (x, y, λ) denote the HSI of esophageal specimens and the standard white diffuser, respectively. Fig. 4 shows the setup for HSI acquisition. The equipment comprises a high-brightness white LED (LA-HDF 5010RL, Hayashi-Repic Co., Ltd., Tokyo, Japan) with a ring light guide, and a hyperspectral camera (PIKA L, Resonon Inc., MT, USA). An analyzer and a polarizer were used to reduce specular reflection of light from the specimens. The hyperspectral camera can acquire HSI at 400-1000 nm at wavelength intervals of approximately 2.1 nm, and the acquired image size is 900 × 500 pixels. As shown in Fig. 5, the relative intensity distribution range of the high-brightness white LED is 420-720 nm. Therefore, in this experiment, the wavelength range of HSI was the same as this light source. This experiment was approved by the Institutional Review Board of the Graduate School of Medicine, Chiba University (No. 2942).

B. ILLUMINATION DESIGN EXPERIMENT
The number of extraction points K for SR, I reflect (λ), was set to 15,000. Therefore, the total SR data of the four specimens  was 4K , or 60,000. Fig. 6 shows the sensitivity of the camera (IMX174, Sony Corp., Tokyo, Japan).

C. EVALUATION EXPERIMENT
First, using the SR images of specimens stained with 0.2% Lugol, the camera sensitivity, and the illumination intensity distributions, we generated pseudo-RGB images. Regarding the intensity distributions of illuminations, for comparison we used those of CWL and NBI as well as those of the designed MBSI. We assumed a halogen light for CWL [24], whereas we simulated the illumination used in NBI by combining Gaussian functions because the exact spectral characteristics of NBI were not available. As described previously [8], [25], we set the peak wavelength of the two bands as 445 and 540 nm and their full width at half maximum as 30 and 20 nm, respectively. The intensity distributions of CWL and NBI are shown in Fig. 8 (a) and (b), respectively.
Second, we constructed the discrimination model using SVM. We used a radial basis function (RBF) kernel for SVM. Two hyperparameters of SVM (cost parameter C and the parameter of RBF kernel γ ) were set to 1.0 and 1 divided by the number of features, respectively. The SVM was implemented in Python 3.5 using scikit-learn. Before SVM, the training and test data were scaled using the following equation: where x and x are the scaled data and the sample data, respectively. The fundamental statistics, µ and σ , represent the mean value and standard deviation of the sample data, respectively. In this study, we used four specimen images, and two or three RGB images of each specimen image were acquired by switching illumination two (MSBI-6) or three times (MSBI-9). Therefore, there were six (R 1 , G 1 , B 1 , R 2 , G 2 , B 2 ,) or nine (R 1 , G 1 , B 1 , R 2 , G 2 , B 2 , R 3 , G 3 , B 3 ) image channels. The sample data of each image channel were scaled using (4).
Here we used feature vectors extracted from the pseudo-RGB images of three specimens as training data, and then applied the model to all pixels of the pseudo-RGB image of the remaining specimen for evaluation. From each of the cancerous and normal areas in each of three specimens, 15,000 points were re-selected and the RGB values at those points were used as training data. Therefore, the total number of training data was 45,000. Fig. 9 shows the constitution of training and test data. We performed cancer discrimination using each of the four specimens as test data.
Third, we compared the discrimination performance using images generated under the designed MBSI, CWL, and NBI. The discrimination performance was evaluated as accuracy, sensitivity, and specificity, as shown in (2), (5), and (6).

1) EVALUATION OF ROBUSTNESS WHEN DISTANCE AND ANGLE BETWEEN CAMERA AND SPECIMENS VARIES
Although the main purpose of this paper is to provide a proof of concept of our proposed method, a minimum investigation of feasibility in applying the method to practical endoscopy is needed. Because endoscopic imaging is conducted under various imaging geometries, we investigated the impact of imaging geometry on accuracy. We evaluated the distance and the angle between the device and the target as the most important factors in this regard. It is known that in imaging a diffuse surface, the distance and the angle between the device and the target both affect the light intensity, and the relative spectral shape is maintained. Thus, we measured the spatial distribution of illumination intensity with a white plate under various geometries and numerically generated the corresponding pseudo-color images using such distribution data. Variation of accuracy was assessed in the generated images. The details are as follows.
In the experiment, we investigated the combination of two distances and four angles. The distance between the camera and the white plate was either 15 or 20 cm, with rotations of 0, 10, 20, and 30 degrees. Fig. 10 shows the experimental setup. The basic setup (Fig. 4) comprises an RGB camera (VCXU-23C, Baumer, Frauenfeld, Switzerland) and lens (FL-BC1220-9M, Ricoh Company, Ltd., Tokyo, Japan), a set of polarizers, light guide, and a white LED light source. We acquired images of a white plate (ColorChecker White Balance, X-Rite Inc., MI) at eight geometries (combinations of the two distances and four angles). The plate was rotated using a rotation stage (GOH-40A15, SIGMAKOKI CO., LTD., Tokyo, Japan).
The images obtained of the white plate are represented as   white plate obtained at 15 cm and 0 degrees, W 15,0 (x, y, i), as a reference image. The ratio of image intensity in the other geometry to the reference image is used to generate a pseudo-RGB image. Fig. 11 shows the generation flow of specimen images when the distance and angles were changed. First, W 15,0 (x, y, i) is normalized by the maximum value of each band, as follows: where The left-most image in Fig. 11 is a pseudo-RGB image of the specimen, S(x, y, i), which is obtained as the spectral product between the spectral reflectance image I reflect (x, y, λ) shown in (3) and the spectral sensitivity of the RGB camera followed by its spectral integration. In this calculation, we assumed that illumination was both spectrally and spatially flat. The pseudo-RGB image of the specimen was then multiplied by the spatial intensity distribution of illumination to obtain a pseudo-RGB image of the specimen with non-uniform illumination, as shown in the middle of the bottom row in Fig. 11. The ratio distribution between the two geometries was calculated as follows: Finally, the ratio image g dis,deg (x, y, i) was multiplied by the pseudo-RGB image of the specimen at the reference white plate to obtain a color image at several geometries: In the investigation of discrimination performance in this experiment, the pseudo-RGB images without illumination non-uniformity, S(x, y, i), were used for training, and the pseudo-RGB images, S dis,deg (x, y, i), were used as test data. All simulations were programed using MATLAB R2020a (The Mathworks, Inc., Natick, MA, USA). Fig. 12 shows the SR of four specimens: with no stain, and with 0.2% and 1.0% Lugol stain. The images of the specimens shown in Fig. 12 were captured using a digital single-lens reflex camera under a fluorescent lamp. The unstained specimens in Fig. 12(a) differ in color to some extent, as reported in a previous study [26]. For example, specimens A and B are discolored, whereas C and D are reddish. However, the naked eye cannot differentiate between the cancerous and normal areas on the basis of color difference.

A. SPECTRAL CHARACTERISTICS OF ESOPHAGEAL SPECIMENS
The color difference between cancerous and normal areas in the specimens was less with 0.2% Lugol staining ( Fig. 12(b)) than with 1.0% Lugol staining (Fig. 12(c)). Among the specimens stained with 0.2% Lugol, the cancerous area is particularly unclear in specimen D. However, Fig. 12(b) indicates that in the cancerous areas of four specimens, the SR line shows an arched shape in the range 450-540 nm, as does the SR of the unstained specimens in Fig. 12(a). A possible reason for this finding is that dilution of the Lugol solution weakened the staining reaction in cancerous areas compared with staining with 1.0% Lugol, making the arch in the unstained specimen more prominent. The arch is due to the absorption characteristics of hemoglobin, which has peaks at 420 and 560 nm [27]. In addition, Fig. 12(b) shows that at wavelengths >570 nm, the slope of SR is more gentle in cancerous areas than in normal areas. Fig. 13 shows intensity distributions of the designed MBSIs. In designing the MBSIs, we evaluated all combinations of the LEDs mentioned in section III. B. As shown in Fig. 13(a), the MBSI with six bands (MBSI-6) has two 3-BIs, illuminations A and B. Illumination A comprises LEDs with peaks at 430, 525, and 700 nm; whereas illumination B comprises LEDs with peaks at 470, 565, and 630 nm. As shown in Fig. 13(b), the MBSI with nine bands (MBSI-9) has three 3-BIs, illuminations C, D, and E. Illumination C comprises LEDs with peaks at 430, 590, and 700 nm; illumination D comprises LEDs with peaks at 450, 565, and 630 nm; and  illumination E comprises LEDs with peaks at 470, 505, and 645 nm. These results indicate that LEDs with peaks at 470, 565, and 700 nm are most frequently used in MBSIs as the blue, green, and red LEDs, respectively. In particular, the LED with a peak at 470 nm can detect differences in the short wavelength region of SR following 0.2% Lugol staining. In addition, LEDs with peaks at 565 and 700 nm can detect differences in the long wavelength region of SR.  Tables 1 to 4 show the discrimination performance of 0.2% Lugol stained specimen images generated under MBSI-6, MBSI-9, CWL, and VOLUME 8, 2020 NBI; and Fig. 15 shows the mean accuracy, sensitivity, and specificity of each illumination type. These results show that the discrimination performance was higher for the designed MBSI than for CWL and NBI. NBI discriminates cancerous lesions based on differences in the short wavelength region of SR due to 0.2% Lugol stain. However, the proposed MBSI system can discriminate the cancerous area with higher accuracy because it exploits the difference in the long wavelength region as well as the short wavelength region of SR. Although increasing the number of 3-BIs in the MBSI raised the cancer discrimination performance (Tables 1 and 2), it hinders the real-time property. Namely, the relationship between discrimination performance and the real-time property is a tradeoff. For this reason, it is necessary to determine the optimal number of 3-BIs.   In this study, we did not design MBSIs based on data acquired in vivo. Because the color of resected tissue differs from that observed in vivo [28], as a next step it is necessary to acquire spectral information of esophageal specimens stained by low-concentration Lugol solution in the clinical situation.

B. MBSI DESIGN
After acquisition of such spectral information, we can easily apply the design method used in the present study to redesign the MBSIs. Fig. 16 shows the discrimination performance of the 1.0% Lugol stained specimen. Tables 5 to 8 list the results in detail.  TABLE 4. Discrimination results (accuracy, sensitivity, and specificity) for images of 0.2% Lugol stained specimens generated under illumination used in narrowband imaging (NBI).

TABLE 5.
Discrimination results (accuracy, sensitivity, and specificity) for images of 1.0% Lugol stained specimens generated under MBSI with six bands (MBSI-6). Pseudo-RGB images of the 1.0% Lugol stained specimens were used for both training and test data. Among the four illuminations (MBSI-6, MBSI-9, CWL, and NBI), the performance of our illuminations (MBSI-6 and MBSI-9) was higher than the conventional illuminations (CWL and NBI), being >90% in both cases. In contrast, Fig. 17 shows the discrimination performance of the specimen with no Lugol stain, and Tables 9 to 12 list the results in detail. Pseudo-RGB images of this specimen were used for both training and test data. Four illuminations could discriminate the cancerous area of the unstained specimens, with mean accuracy of 64%-70%. Although we designed the MBSI-6 and MBSI-9 illuminations for a low concentration of Lugol stain, these illuminations were effective for the high-concentration stain and unstained specimens as well as for the 0.2% Lugol stained specimen. Fig. 18 shows the profiles of the white reference and specimen images with respect to the horizontal direction. The angle was gradually changed so that the right side of the image was more distant from the camera after the rotation, and the left side more near. Note that when the white plate was 5 cm farther away from the reference position, the pixel values of the profile decreased, which was also the case for the specimen images. However, changing the angle had   little effect on the profile. Although the overall pixel values decreased with changing distance, the shapes of the profiles were maintained. Fig. 19 shows the change in the discrimination results for the 0.2% Lugol stained specimen with variation of distance and angle between the camera and the specimen. The squares, circles, and triangles indicate results for the specimen image with uniform illumination, with the image placed at the reference position, and with the image 5 cm away from the reference position, respectively. The performance of illuminations MBSI-6 and MBSI-9 were decreased when multiplied by illumination non-uniformity. The performance of MBSI-9 was higher than MBSI-6 in all cases. Moreover, changes in distance and angle had little influence on performance because all sample data had been scaled [see (4)]. VOLUME 8, 2020

D. FUTURE WORK FOR ENDOSCOPIC IMPLEMENTATION
As a first step, we have conducted a minimum feasibility study of our proposed method. It is necessary to address the following in future work before the method can be applied to practical use.
For robust discrimination, we need to build an imaging system and perform discrimination with a greater number of specimens. Here we employed SVM as a supervised learning method for classifying normal and cancerous areas, which was confirmed by a medical specialist. Ideally the classification would be performed without annotation data. Therefore, we will aim to use unsupervised learning methods for this classification.  In addition, it is necessary to investigate whether the designed MBSI is effective for in vivo data. We will investigate differences of images between in vivo and resected tissue using an endoscope. Although the color of tissue varies between in vivo and resected tissue, we expect that the design technique described here will be suitable for use.
When our method is applied to endoscopy, the optical system must be modified from that shown in Fig. 4. Light from the optimal LED lights should be guided using a light guide channel in the probe, as in the conventional technique. In vivo measurement also had specular reflection from tissue surface as well as resected tissue. Therefore, our system requires a method reducing the reflection, such as cross-Nicol alignment of polarization plates. In addition, to observe in vivo tissue clearly, our system requires a high-speed camera with high spatial resolution and wide dynamic range.
In the present system, multiple RGB images were acquired in a time sequence. It would be difficult to image the same target area using this system because an endoscope cannot be maintained in a fixed position. A high-speed camera synchronized with our switching illuminations would be necessary to reduce movement in the target area during acquisition of the RGB images. Moreover, the optimal light intensity for illumination must be selected according to the frame rate because the exposure time will become shorter with a highspeed camera. In short, as the speed of illumination light switching and the shutter speed of camera become faster, captured images become darker. Therefore, we need to enhance the illumination light intensity, maintaining suitable pixel values according to the shutter speed. In addition, to acquire the same area of multiband images, it is necessary to align numerous RGB images. As it is preferable that our system acquires multiband images concurrently, a snapshot-type system might be suitable.

V. CONCLUSION
To discriminate between normal and cancerous tissue with MBI under conditions of low-concentration Lugol staining, we proposed an endoscopic system with a 3-BI switching function. We predicted the effectiveness of the proposed endoscopic system through image generation simulation based on HSI of resected specimens of esophageal cancer.
To validate the effectiveness of the system, we generated pseudo-RGB images under MBSI-6, MBSI-9, CWL, and NBI, and tested the discrimination of cancerous areas on the generated pseudo-RGB images. The discrimination performance was the highest under MBSIs.
The present paper has presented the proof of concept of the proposed method; however, further studies including experiments with more specimens and modification of the system must be carried out before the method can be applied clinically.