Identification of Partially Occluded Pharmaceutical Blister Packages

Medical dispensing refers to the in-office preparation and delivery of prescription drugs, which is mostly dispensed by the units of blister packages. The objective of the study is to design an image-based blister package identification solution, which is capable of identifying a fetched drug based on a pair of the two opposite camera images of the hand-held drug. To this aim, this paper proposes a deep learning based Hand-held Blister Identification network (HBIN) to identify partially occluded blister packages present in arbitrary positions and orientation with possibly cluttered backgrounds. The proposed HBIN is a two-stage network that contains Blister cropping network (BCN) followed by RTT identification network (RIN). The BCN subnetwork, an image to image translation deep learning network, is to crop both side contours of the hand-held drug, before the pair of cropped contours can be juxtaposed as a fixed sized and fixed orientation RTT (rectified two-sides template) for final identification in the RIN sub-network. A blister package dataset containing a total of 30,394 images based on 230 types, typically found in hospital dispensing stations, have been collected and labeled. With extensive test, the accuracy of the primitive primitive HBIN attains an F-score of more than 94.33% for testing data from similar backgrounds and an F-score of 79.80% for dissimilar backgrounds. Although still a prototype, the preliminary results show the feasibility of identifying blister packages during retrieval process without resorting to bar codes nor RFID tags.