Pharmaceutical Blister Package Identification Based on Induced Deep Learning

Prescription dispensing accuracy is of paramount importance for all hospitals. However, human errors are inevitable due to multiple reasons, such as fatigue, stress, heavy workload, lack of effective verification measures, mismanagement. Such human errors pose serious safety and health concerns on the part of patients and may as well lead to a series of medical disputes. Based on induced deep learning, this paper proposes a real-time Blister Package Identification System (BPIS) to assist pharmacists’ drug verification and dispensing. Under the guidance of the induction strategy, image preprocessing is introduced to form a standardized image containing the front and back side of the blister package, which is subsequently sent to CNN-based object identification network for feature extraction and identification. This preprocessing method allows the identification system to promote the deep learning system to focus on feature learning to obtain more information about the appearance of the package ruling out confounding factors such as background noise, size, shape or positioning. In addition, this article collects and establishes an image dataset of adult lozenges. Under this dataset, this paper verifies the enhancement of Induced Deep Learning (IDL) on YOLO v2, ResNet, and SENet. By optimizing the deep learning identification network with the help of the embedded technology and a two-side extraction mechanism, a real-time BPIS is built. Long-term tests in hospitals prove the effectiveness of the proposed system.


I. INTRODUCTION
Pharmacy station is an essential department in every hospital, and its core duty is to dispense the prescribed drugs to patients [1]. Accurate dispensing in line with medical norms is the benchmark of pharmacy work. Any mistake would cause serious consequences, sometimes even resulting in irretrievable consequences [1]. To improve the dispensing accuracy, caution and technical tools are necessary in pharmacists' work. Some hospitals use bar code dispensing system [2]. When pharmacists get the prescription, they would first scan the bar code on the patient's medicine bag, and then dispense the drug according to the information displayed on the screen, such as the name, storage location, code, appearance.
Although most hospitals have optimized their pharmacy work in terms of technology and management, dispensing errors are still often spotted in actual work [2]. According The associate editor coordinating the review of this manuscript and approving it for publication was Xi Peng . to incomplete statistics, the global annual loss caused by dispensing errors is enormous [2]. In the United States alone, about 8,000 people die from dispensing errors each year, causing about 20 billion US dollars in loss [3]. Literature review and hospitals visits revealed that heavy workload [4] was the main reason for the error. The lack of reasonable and effective verification methods made the problem worse. Take Taipei MacKay Memorial Hospital as an example, during the peak time, pharmacists are required to dispense 2 medications a minute on average. Faced with such intense work, it is difficult for pharmacists to effectively evaluate the doctor's prescriptions in a short period of time or to verify the drugs, which in turn may lead to the reduction of prescription review and the occurrence of drug-dispensing errors. The main purpose of this paper is to build an effective drug identification system by using computer vision technology to assist pharmacists in drug verification and reduce drug dispensing errors. That is to say, based on computer vision and deep learning, this paper proposes a real-time identification system for drug dispensing to assist pharmacists' drug verification efficiently and accurately.
Drugs under discussion in this paper refer to blister package drugs. The drugs are packed in rectangular plastic packages to form a row with blister package, one of the most common and widely used ways of packaging. The shape of the pill and the texture of the package can be seen on the front of the package, and the drug information and patterns can be seen on the back. Generally, the identification method is to use OCR technology to identify the drug information on the back, and then classify and identify it. However, there are some practical issues in daily practice: the printed text on the back is often very small, slightly distorted, reflective or printed with a light color. Under such circumstances, OCR technology often fails to effectively recognize the text and related information. In view of these issues, this paper integrates computer vision and deep learning to realize rapid identification of information on the blister package.
Long-term observation and experiment show that blister package identification mainly faces the following problems: (1) Comparatively limited samples: due to the hospital control of adult lozenges and limited time and manpower, this study only takes 72 pictures for the front and back sides of each blister package as the training and testing dataset.
(2) Similarity in blister package appearance: adult lozenges are mostly packed with silver aluminum foil, which makes the appearance very similar; for drugs of the same manufacturer or the same function, the appearance varies only slightly in the printed text and the manufacturer logo. Visually, it is very easy to confuse them. This is also one of the main reasons that pharmacists choose the wrong drug [5].
(3) Large number of blister package drugs: there are hundreds of blister package drugs in the adult lozenge area of most hospitals. For safety reason, the classifier must be able to effectively distinguish each blister package. This brings a big challenge to the classification model.
(4) Higher identification requirement: Compared with other object identification, the accuracy requirement of drug identification is extremely high, reaching almost 100%. Therefore, the design requirements for algorithm and device structure are also very high.
To solve the above problems, this paper proposes Induced Deep Learning (IDL) to improve identification accuracy and constructs a real-time Blister Package Identification System (BPIS) for drug verification by adopting an embedded technology.
Compared with our early work [6], the improvements of this paper are: 1) a real-time BPIS is constructed by adopting an embedded technology which can optimizes the deep learning network. Two years of tests and optimization in hospitals prove that it is stable and effective. 2) Long-time tests and detailed improvements are made to the induction strategy, and the stability of identification has been significantly improved. The main contributions of this article are as follows: (1) Induced deep learning is proposed and applied in blister package identification.
The so-called induction refers to the artificial guidance that allows the deep learning system to learn features that improve the identification effect. Human experience and cognition are integrated into deep learning to make the feature search of deep learning develop in the direction of human expectation and thus avoid a series of problems caused by relying on a large number of training data. In other words, it allows the deep learning system to focus primarily on the learning of important features.
The deep learning derived from it is called IDL. IDL is an application-oriented deep learning strategy. For specific problems, it provides guidance for deep learning network to enhance the differentiation of features.
Based on the above concepts, this paper designs an Induced Deep Learning Identification System (IDLIS) for blister package, as shown in Figure 1. Firstly, the front and back FIGURE 1. Induced deep learning for blister package identification. VOLUME 9, 2021 images of blister package are extracted from the camera; Then, the background is removed, the images are normalized and re-oriented into a fixed upright 448 × 224 image only containing drug information; Thirdly, the re-oriented front and back images are juxtaposed to obtain a re-oriented two-sided image (RTI) containing the front and back information. Finally, the image is sent to the deep learning network to identify and obtain the name of the drug. Since the original image has been extracted, the deep learning network can focus more on the learning and identification of important features, thereby reducing the pressure of deep learning and improving the accuracy and stability of identification.
(2) A real-time BPIS is designed and applied. In this paper, a lightweight real-time BPIS is realized by designing a two-side extraction mechanism and optimizing the deep learning network under induced strategy. The system can not only identify the blister package in real time, help pharmacists to verify the medicine, but also can be used for blister package similarity tests, which can be used as a reference to safeguard the drug and cabinet purchasing for medical institutions.

II. RELATED WORK
In response to the dispensing-error problem, technical methods were used along with strengthened pharmacy management and pharmacist training. At present, the main methods adopted are bar code drug delivery system [7], wireless drug detection cabinet [8], automatic drug cabinet [9] etc. The bar code drug delivery system [7] is mainly used to verify the relationship between drugs and patients at nursing ends to ensure that the drugs are accurately dispensed to the right patients. The drawback of this system is that the pharmacist must scan the bar codes on the drug bags, and then manually verify them one by one. The process is cumbersome and time-consuming. To solve this problem, RFID-based wireless detection drug cabinets have been developed [8], which attach RFID electronic tags to each drug, and infrared sensors are installed on each drug cabinet. When the pharmacist opens a cabinet that does not match with the prescription, the system will automatically warn the pharmacist. This system is used together with a bar code system. However, this requires modification of the original drug cabinet. To reduce the workload and simplify the work of pharmacists automatic drug cabinets appeared in recent years [9], which automatically accomplish the drug taking, checking and dispensing. The high cost of purchase and maintenance makes it only feasible for large medical institutions. For small and medium-sized hospitals, there is an urgent need for a fully automatic and economical drug identification system to assist pharmacists in verifying drugs, reducing dispensing errors and work intensity.
Presently, there are three types of drug identification: tablet, drug package and blister package. Tablet identification relies exclusively on appearance attributes such as color, shape and imprints (if any). The identification methods are mainly based on traditional machine learning and deep learning. Neto et al. [10] proposed the feature descriptor CoforDes based on the shape and color of the tablet, and achieved considerable results. António Cunha et al. [11] proposed features based on shape, dimensions and colors to help elderly people identify tablets. Zeng et al. [12] proposed a mobile tablet identification system based on deep learning, which has received widespread attention. Delgado et al. [13] optimized the deep learning network for faster and better identification. Chang et al. [14] proposed a wearable tablet identification system based on deep learning, which is convenient for daily use. Drug package identification refers to displaying the name of the drug by identifying the outer package of the drug. It is used for automatic drug sorting and helping the visually impaired to take drugs. Xia et al. [15] used the SVM method to identify drugs, which is used for drug verification in smart drug vending machines. Vasavi et al. [16] used the invariant feature to realize the identification of the medicine box, which is used to help the visually impaired to take the medicine accurately. Ribeiro et al. [17] integrated bar code identification, text identification, and feature matching technologies to realize a mobile tablet identification to help the visually impaired. Blister package identification refers to the identification of the drug name through the front texture and the back text of the blister package. In the early days, it was mainly realized by methods based on traditional machine learning. Li et al. [18] adopted the method of Shape Template Matching. In recent years, methods based on deep learning have been mainly used. For example, Chung et al. [19] used deep learning-based methods to realize blister package identification with occlusion, and Chung et al. [20] proposed an end-to-end rapid identification method, which has achieved better results. For better results, Chung et al. [21] just built a more compact structure based on the one-side image of the blister package to realize the rapid identification at the expense of slight reduction in identification accuracy.
CNN-based deep learning methods are used at present in place of the early object identification. VGG [22] carried out a more detailed design on the basis of AlexNet. The most obvious change is that the increased depth significantly improved the identification effect. The identification rate decreased instead of rising because of the gradient problem when the VGG network tried to increase the depth further. In order to solve this problem, He et al. [23] and others proposed ResNet, which added a short cut connection in the network to alleviate the gradient dissipation, and achieved great success. Hu et al. [24] proposed the Squeeze and Excitation structure from the perspective of feature enhancement, which highlights important features, suppresses features that are not useful for the current task, and achieves obvious results. Clustering methods based on CNN, such as Deep Clustering with Sample-Assignment Invariance Prior [25] and structured AutoEncoders for Subspace Clustering [26] outperforms most of state-of-the-art clustering approaches in object identification. Undoubtedly, Transformer [27] is a very popular deep learning model in recent years and promotes the progress of object identification.
Based on the CNN network, the deep learning framework for object detection has also undergone continuous development, such as GoogLeNet [28], R-CNN [29], Fast R-CNN [30], Faster R-CNN [31], etc. R-CNN is the first successful CNN-based object detection method. It only uses CNN to obtain features. The classifier uses SVM and needs to be trained separately which slows down detection speed. Later, Fast R-CNN and Faster R-CNN optimized the design on the basis of R-CNN, and their speed and accuracy have been significantly improved. However the speed still cannot meet the requirement for real-time detection. Redmon et al. proposed YOLO [32] in 2015, which adopts an end-to-end structure with high-speed and identification capability, but a slight decrease in accuracy. SSD [33] combined the advantages of YOLO and VGG, the accuracy has been improved significantly, while the speed is slightly reduced. YOLO v2 [34], YOLO v3 [35], YOLO v4 [36] have increased not only the detection speed, but also the accuracy, and become the current mainstream object identification method.

III. PHARMACY DRUG DISPENSING FRAMEWORK
Structurally, the pharmacy drug dispensing system is mainly composed of the doctor end, the toll end, the client end, the cloud server, the management end, and the dispensing end, as shown Figure 2. The doctor interface inputs the diagnosis and prescription for subsequent processing. The toll interface is operated by the toll collector for patient registration and payment. The client interface sends patients information such as notifications and precautions for drug taking. The cloud server serves as the center of data exchange, connecting various parts to realize the transmission of information and data storage, which enables easy query, use and analysis. The management end is the center of system, used for the monitoring, management, query, analysis and maintenance of the entire system. The drug dispensing end is the core of the entire dispensing system, used for the verification of doctor's prescriptions, the confirmation and verification of drugs.
Drug verification ensures that the dispensed drugs are consistent with the doctor's prescription, which is of vital importance for the pharmacy. It is necessary to adopt technical means to realize the automatic verification of prescriptions to reduce the heavy workload. Automatic verification uses artificial intelligence technology to automatically identify drugs and compare them with the doctor's prescription, and ensures that the drug taken by the pharmacist is exactly the same as the prescription. This article tries to build a real-time drug identification system by combining computer vision and artificial intelligence to help pharmacists check the drugs quickly and reduce dispensing errors.

IV. BLISTER PACKAGE IDENTIFICATION SYSTEM BASED ON INDUCED DEEP LEARNING A. INDUCED DEEP LEARNING
Deep learning, in essence, is a target oriented way to achieve the approximation of the objective function by using large numbers of data. Therefore, at present, deep learning relies heavily on a large number of training data and powerful computing resources.
Inspired by the concept of induction in biology, this paper introduces the concept of induction in the deep learning to tackle the above problems. Induction enables deep learning to learn the features more efficiently under the guidance of human experience. That is, it integrates human experience and cognition into deep learning, so that the feature search of deep learning will develop in the direction expected by human beings, and avoids a series of problems caused by relying only on large amounts of training data. For specific problems, it provides accurate guidance, such as information source selection, data preprocessing, feature extraction and so on. The aim is to promote the development of deep learning in an expected direction. Take the final examination as an example, the instruction of a teacher before an examination serves like an induction, which may save a student's time and energy but is conducive to achieving satisfactory results. Therefore, in essence, induced deep learning is an application-oriented optimization strategy.
The purpose of induction is to improve the accuracy of blister package identification. Specifically, it helps deep learning to solve the related problems: background interference, varying sizes, positioning, lighting, and limited blister package information, so as to improve the accuracy of identification.

B. BLISTER PACKAGE IDENTIFICATION BASED ON INDUCTION 1) IDENTIFICATION FRAMEWORK
Blister package identification is essentially object identification in which deep learning is supposed to be effective. However, the difficulty in this case is that it requires extremely high accuracy, approximately 100%. Any dispensing error may lead to very serious consequences.
Through long-term observation of the process of pharmacists' work and on-site interviews with them, we found that the main factors affecting blister package identification are: 1) Background interference caused by lighting or object interference, etc.; 2) Problems caused by different sizes; 3) Viewing problems caused by the change of camera positions; 4) Identification difficulties caused by the limited information on the two sides of a blister package.
To address the above problems, this article uses the induction strategy and proposes to embed preprocessing on the basis of the deep learning. In this way, deep learning can focus more on the learning of features. The blister package identification framework is shown in Figure 1. The framework consists of two parts: the preprocessing re-orientation and juxtaposition module and the object identification network. The preprocessing module is mainly used to reduce the impact of the above four types of problems and provide a better basis for the identification network.

2) RE-ORIENTATION AND JUXTAPOSITION
The front and back images captured by the camera are re-orientated and juxtaposed to get better input for deep learning network, thus to improve the identification accuracy. Generally speaking, there are two main shapes of blister packages: rectangle and quadrilateral with an arc edge. Digital image processing technology is used to extract the blister package from the original image for re-orientation and juxtaposition. However, in the real scene, the original image may appear in any position, viewing angle or lighting randomly as shown in Figure 3.
The re-orientation and juxtaposition method as shown in Figure 4 is proposed to re-orient and juxtapose the front and back images. There are three steps: background removal, corner location, re-orientation and juxtaposition.
Step 1 (Background Removal): This step identifies the boundary contour of the package from the given image and eliminates the interference of the background.
1) The front and back images are obtained by the camera, as shown in figure 5-a. At the same time, the color space of the image is transformed from BGR to gray; 2) Median blur is used to remove the noise in the image, and edge and contour detection are used to obtain the edge of the blister package. The results are shown in figure 5-b;   3) Convex hull operation is performed on the edge coordinates of the image obtained in the previous step, and the convex polygon is found from the contour coordinates and colored. The results are shown in figure 5-c; 4) The edge contours are processed to obtain the largest convex hull of the edges, and subsequently the largest convex hull is identified as the target as shown in Figure 5-d.
Step 2 (Corner Location): This step locates the four corners of the detected boundary contours for the subsequent processing and eliminates the interference of viewing angles in identification.
From the contour obtained in the first step, we choose at least three lines as side lines, and then the closest minimum quadrilateral and its corresponding corner coordinates can be obtained by geometric reasoning. The lines finding algorithm is shown in algorithm 1: Given the contours of the detected package, all the possible straight lines can be obtained by Hough transform, resulting in an array of lines, as shown in Fig. 6-a. Each identified line has two Hough transform parameters: ρ and θ, where ρ is the shortest distance between the straight line and origin, and θ is the angle between the line and the x axis, as shown in Fig.7. These lines will be numbered and sorted according to the number of votes. The smaller the number is, the more  With three straight sidelines detected by algorithm 1, the four corners can be located based on the procedure in algorithm 2.

3:
M ←The midpoint between two intersection points P 1 , P 4 .

5:
v ← The displacement vector from M to B. 6: In order to make the image regular in the third step, the four corners must be arranged clockwise or counterclockwise. Therefore, we first calculate the two intersections P1 and P4 (two green points in fig. 6-c) of the three known sidelines as the starting point and ending point, and then obtain the midpoint M (yellow point in fig. 6-c) between two intersections, the barycenter B (yellow point in fig. 6c) of the blister's contour, the displacement vector v from M to B as shown in fig. 6-c. Finally, P2 and P3 can be obtained with the displacement vector 2v from P1 and P4 as shown in Figure 6-d.
Step 3 (Re-Orientation and Juxtaposition): The main purpose is to eliminate the interference of different sized, obtain the standard RTIs, and provide more distinctive information. In other words, the blister package is re-oriented upright into a fixed size of 448 × 224 pixels, as illustrated in Fig. 8. Then we simply juxtapose the front and back images of the same package along their longer sides, forming a RTI with a resolution of 448 × 448 pixels.
Therefore, before juxtaposition, we must follow the re-orientation rule that the order of the points starts from the shorter line. The detection goes as follows: calculate the distance between P1 and P2, and P2 and P3 respectively. If the distance between P1 and P2 is relatively long, the order   of corner points starts at the longer side (in case the image in a horizontal lying position), as shown in figure 9-a. Once this happens, we loop all corner numbering sequences to the right to ensure that the corner sequence starts at the shorter line, as shown in Figure 9-b.
Once the images and the correct order are successfully obtained, we can use the image relationship between the four corners and the final template size (448 × 224 pixels) to find the perspective matrix; then use the perspective conversion function to convert the single-side image to the 448 × 224 pixel template, as shown in Figure 9-c.
After processing the single-side image, the image of the other side can be obtained with the same algorithm, and then the two pictures are juxtaposed to form the RTI as shown in Figure 10.

C. REALIZATION OF THE EMBEDDED BPIS
To realize real-time BPI, the embedded BPIS is constructed as shown in Figure 12. The system consists of three parts: two-side image capturing model, embedded computing module and identification display module. The two-side image capturing model uses two cameras to obtain the front and back images for next step re-orientation and juxtaposition; the embedded computing module completes the identification related calculation; and the display module displays the identification results.

1) DESIGN OF THE TWO-SIDE IMAGE CAPTURING MODEL
The two-side image capturing module obtains the front and back images. Its structure is shown in Fig.11. The module consists of a wooden cabinet shell, 8 board nails, a supporting board, a piece of transparent glass and two cameras. The two cameras are installed in the holes at the top of the wooden cabinet and the center of the board. The two cameras are facing each other, and the lenses are aimed at the transparent glass in the middle. When the drug is put on the glass, the front and back sides of the blister package can be photographed simultaneously by the upper and lower cameras, and are directly transmitted to the embedded computing module. In order to provide a stable light source for shooting, a light strip around the interior of the wooden cabinet is installed two centimeters above and below the transparent glass. In order to avoid reflection or noise in the background area of the image, black flannelette is pasted on the interior of the whole wooden cabinet and the peripheral areas of the wood and glass, leaving only a 30cm × 25cm transparent area in the middle, which is the same as the effective shooting range of the camera. The size of the whole module is about 40cm × 40cm × 40cm.

2) EMBEDDED PLATFORM BASED ON JETSON Tx2
The training stage of deep learning system is often implemented by a PC with high-end GPU, but in practice, embedded platform is often used. There are three main reasons: limited computing power required for real-time identification, limited space in the hospital pharmacy, and limited funds. Therefore, the embedded platform based on Jetson Tx2 is used can provide the required computing power, save space and money. Jetson Tx2 has 256 CUDA cores, 8g memory and 59.7 Gb/s bandwidth, which is enough for real-time blister package identification.
In practice, first, the PC equipped with NVIDIA GTX 1080 is used to train the deep learning for blister package identification, and then transfer the weight files generated in the training process to each dispensing table with Jetson Tx2 assembled. We can link the Jetson Tx2 and two Logitech BRIO cameras through USB interface, and get the RTIs. Then, the trained weight file is used to perform the identification, and the identified drug name is displayed on the screen.

3) OPTIMIZATION OF DEEP LEARNING NETWORK BASED ON EMBEDDED PLATFORM
In view of the limited computing resources in the embedded platform, the system in this paper uses the Tiny version of YOLO v2 (Tiny YOLO) proposed by Redmon et al. [32] as the object identification network. The network can recognize  more than 200 FPS of input images. The network structure is shown in Figure 12.
The Tiny YOLO network structure is simple and only contains 9 convolution layers. In practical applications, it is found that this simple structure collocation induces deep learning, which can realize blister package identification. In order to improve the identification speed, we have optimized Tiny YOLO. Table 1 shows the comparison between the original and optimum parameters of Tiny YOLO. Since the RTIs already contain enough identification information, we adjust the image size of the input network to a smaller 224 × 224 pixels to increase the execution speed and reduce the computation demand. Then, we let the deep learning network adjust randomly the saturation and exposure of the training image to increase the flexibility of the network when the input image changes with the lighting and position.
According to the design of the original Tiny YOLO, in the identification stage, the deep learning network will segment an input image according to a 13 × 13 grid, and predict five sizes of bounding boxes in each grid. Finally, with the confidence score returned by each grid, it will determine whether the grid contains target. However, in the induced deep learning system, the whole RTI must be learned and recognized because the interfering factors are eliminated in the re-orientation and juxtaposition step. The anchor size is adjusted from 13 × 13 to 7 × 7 (224 divided by 32) with the change of input image size from 416 × 416 pixels to 224 × 224 pixels. Based on the above change, we adjust the number of anchors for prediction from 5 to 1, and set the anchor prediction size to 7 × 7, so that the deep learning network only needs to predict one full-size bounding box for each grid in the real-time testing stage.
The last two parameters are training parameters: the original YOLO will randomly reduce the size of the bounding box by 20%, and adjust randomly the size of the input image from 320 × 320 to 608 × 608 for multi-scale training. We turn off these two data adjustment functions since they are unnecessary for the induced deep learning system, mainly because the bounding box area is the actual RTI and the size of the input image is always fixed.
The optimized Tiny YOLO network structure is shown in Figure 13. Compared with normal YOLO v2 or the original FIGURE 13. Optimized tiny YOLO network structure.  Tiny YOLO network, this optimized structure has better identification effect and requires less computation.

A. DATASET
The data used in this paper are all taken from the adult lozenge area of Taipei Mackay hospital, with a total of 250 kinds of drugs. 72 images of the front and back sides of each drug are taken, which results in a total of 36000 images. After selection, re-orientation and juxtaposition, 90000 images were obtained, including original images, tailored images and RTIs, as shown in Figure 14. The original images are mainly used to test the object detection algorithm such as YOLO; the tailored images are mainly used to test the object identification algorithm such as ResNet, and the RTIs are mainly used to test the effect of the induction method in different identification algorithms.
It should be noted that our current algorithm cannot judge whether the image is upright or up-side down in image reorientation. Therefore, when preparing the training dataset, for each blister package, four RTIs are re-oriented as shown in Figure 15. This covers all possibilities that may arise during testing phase.

B. IMPLEMENTATION DETAILS 1) BLISTER PACKAGE IDENTIFICATION ALGORITHM a: TEST PLATFORM AND SETTINGS
The test platform used in this paper consists of Intel (R) Core i7-7700K @ 4.2GHz CPU, GEFORCE GTX 1080 TI GPU, with a Windows 10 operating system. Tensorflow is used as the development framework. To test the improvement of induced deep learning in blister package identification, the mainstream object detection algorithm YOLO v2 and the  Comparative experiments are conducted to evaluate the effect of induced deep learning in blister package identification. As shown in Table 2, first of all, input the original front image, back image and RTI into YOLO, ResNet and SENet respectively to test the identification effect of IDL.

c: EXPERIMENT SETUP
To evaluate the identification effect accurately, we try our best to unify the training and testing rules of YOLO v2, ResNet and SENet networks, as shown in Table 3.
All data are adjusted to 224 × 224 pixels before sending to deep learning network for training or testing. During the training process, the data increment function of the network is turned off, and no pre-training model is used. The batch size used in all experiments is 8, that is, the model parameters will be adjusted after training every 8 images. The maximum number of epochs for each group of experiments is 100, which means that the network will stop training after seeing all training pictures 100 times. The model will store the weight data after each epoch.
Training and testing: Generally speaking, the evaluation of deep learning algorithm often divides the data sets into three independent categories: training set, verification set and test set. The training set is used to train multiple candidate models; the verification set is used to select the best model; and the test set is used to test the effect of the algorithm.
In our experiment, the data set is comparatively small. With limited front, back and RTI images, we do not have extra  data for verification set. Therefore, 54 images are randomly selected as the training set from 72 images in each category, and the rest 18 images are selected as the test set. The total training samples are 13500 and testing samples are 4500. Each test is repeated 4 times, and the average value of 4 test results is used as the final result. The results of the algorithm are evaluated by three criteria, Precision, Recall and F1-score.

2) EMBEDDED BLISTER PACKAGE IDENTIFICATION a: TEST PLATFORM AND SETTINGS
As shown in Figure 16, there are two stages in real-time blister package identification system, the training stage and identification stage. The training stage platform consists of Intel (R) Core i7-7700K @ 4.2GHz CPU, GEFORCE GTX 1080 TI GPU, with a Windows 10 operating system. Tensorflow is used as the development framework. The platform for testing uses Jetson Tx2 embedded development kit. In order to realize real-time identification with limited Jetson Tx2 resources, the optimized Tiny YOLO is used as the basic identification algorithm. The main reason is that Tiny YOLO is implemented by C language, which runs faster and occupies less resource. b: SYSTEM IMPLEMENTATION 13500 blister package images randomly selected from the RTI data are sent to the PC with NVIDIA GTX 1080 GPU for training to obtain multiple identification models. The remaining 4500 images are tested to obtain the best identification model. Then, the IDL processing and training model is transferred to Jetson Tx2 embedded platform. Finally, 250 blister packages are used for real-time identification.

C. ALGORITHM TEST RESULTS AND DISCUSSION
The test results of the algorithm are shown in Table 4, table  5 and table 6. The three tables record the model with the best identification effect (F1 score) and the least training epoch  from 100 epochs. Taking Precision, Recall and F1-score as criteria, they also record the training time, the number of epochs and the classification effect of the model on the test data. Table 4, table 5 and table 6 represent the experimental results of three deep learning networks respectively: YOLO v2, ResNet-101 and SE-ResNet-101.
In the conventional deep learning identification experiment in Experiment 1 and Experiment 2, it is a challenge to classify 250 drugs only by unprocessed front or back images, whether it is for object detection YOLO v2 or object classification ResNet and SENet.
For the front image, the deep learning network only recognizes the blister package by the shape of the pill and the texture of the package. These features are not enough for the deep learning network to obtain sufficient information. Therefore, in the experiment of YOLO V2, the best F1 score of the front image is 65.39%. ResNet effectively alleviates the problem of gradient disappearance and explosion in deep learning through residual function, and improves F1-score to 85.25% in the front image experiment. By adjusting the weight of features with the loss function, SENet makes the network more dependent on important features and reduces the attention to unimportant features, and achieves a better F1 -score of 90.90%.
The back image contains the printed drug name, dosage, pattern and other labels, which provides more information than the front image. The F1-scores in the YOLO v2, ResNet and SENet experiment are 86.48%, 86.68%, and 96.23% respectively. The above experiments also show that there is still room for improvement.
In the inductive deep learning in Experiment 3, we train and test RTIs. The identification effect is greatly improved. F1-score for YOLO v2, ResNet, and SENet is increased to 99.78%, 99.79%, and 99.97% respectively. In addition, the three networks improve the identification to a higher level with shorter training time and fewer epochs.
These prove that induced deep learning can focus on the learning of relevant features with the help of re-orientation and juxtaposition, without being interfered by background noise, regardless of the image size, positioning or lighting. With the above characteristics, inductive deep learning can provide a practical and effective solution for the blister package identification problems, such as appearance similarity, fewer and a great variety of samples.

D. COMPARISON WITH OTHER STATE-OF-THE-ART ALGORITHMS
In order to evaluate the proposed algorithm, this paper compares the recognition rate of the proposed method with that of the commonly used pharmaceutical blister package identification method, as shown in Table 7. It can be seen from the table that the effect of directly using Yolo is 89.39%, relatively poor among all the tested methods; ResNet, SeNet, RIN [19] and Fast ROR [21] produces better results, higher than 90%, which may be satisfactory for ordinary object recognition. However, for demanding recognition rate applications such as drug recognition, further improvement is needed. The method proposed in this paper reaches 99.98% and outperforms the above five methods. This shows that the proposed algorithm can effectively improve the accuracy of drug identification.

E. EMBEDDED BLISTER PACKAGE IDENTIFICATION
The F1-score of the Tiny YOLO identification model reaches 100% after 2 hours 34 minutes training, that is, at the 40th epoch. It successfully and completely recognizes 250 blister packages out of 4500 RTI samples in the database.
Then we transfer the Tiny YOLO identification model to the Jetson Tx2 embedded system. In the real-time testing stage, we use two cameras in the image capturing mechanism to capture images continuously and send them to the Re-oriented Two-side Template (RTT) for technical processing. This algorithm takes about 0.12 seconds (about 8.33 FPS) to generate a RTI on the Jetson Tx2 system, as shown in Figure 17. Finally, we send the RTIs to the optimized Tiny YOLO network and use the above transferred model for identification. The identification speed of this model can reach about 200 FPS. The identification speed of the complete sample extraction and identification system can reach about 6.23 FPS, with an accuracy of 100%, as shown in Table 8. After nearly two years of testing and practical application,  the system runs stably and the identification effect is satisfactory. So far, no false identification error caused by the system has been found.

VI. CONCLUSION
To reduce dispensing errors at pharmacy stations, this paper proposes a real-time blister package identification system based on induced deep learning. In practice, the difficulty of blister package identification is mainly caused the variety and similarity of blister package, and less available training samples. Drawing on induced strategy, this paper introduces an image preprocessing technology to extract the target, re-orient the front and back images and normalize the blister package image. It reduces the interference caused by background noise, different sizes and positions. The identification system can obtain more differentiating information about the blister package, and enable the deep learning system to focus more on feature learning. In order to test the effectiveness of the proposed algorithm, this paper builds a dataset of collected blister packages from hospitals, and tests the induced strategy on various classic object identification frameworks (YOLO v2, ResNet, SENet). The experimental results show that the method can effectively improve the identification effect. On this basis, a real-time blister package identification system is built by using embedded technology, optimizing identification network and constructing two-side imagine collection mechanism. The test results show that the system produces better and higher accuracy in a stably and fast manner, and does no need any additional modification on the part of the pharmacy. Therefore, it is suitable for small and medium-sized medical institutions. Since 1992, he has been with the Electrical Engineering Department, National Taiwan University of Science and Technology, Taiwan. He is currently a Professor and the Curriculum Chairman of the EE Department. His current research interests include automation, intelligent building, and interactive technology. He was granted a research award by the National Science Council, from 1994 to 2001. He was a recipient, with Stephane Lafortune and Feng Lin, of the IEEE 1994 George S. Axelby Outstanding Paper Award from the IEEE Control Systems Society, in 1994.
QIANG XIAO received the B.A. degree in English from the Wuhan University of Science and Technology, Wuhan, China. He is currently an Associate Professor with the School of Foreign Languages, Neijiang Normal University. His research interests include artificial intelligence and computer aided translation, Chinese-English translation: theory and practice.
JING-SYUAN WANG received the M.Sc. degree from the National Taiwan University of Science and Technology, Taipei City, Taiwan, in 2018. He is currently working in the National Taiwan University of Science and Technology as a Software Engineer, who specializes in machine learning, image processing, and embedded system development.