Adaptive Speed Bump With Vehicle Identification for Intelligent Traffic Flow Control

This paper presents a mechatronic system for intelligent speed bump control. The idea is to make the road bump operate adaptively based on the identification or driving speed of the incoming vehicle. First, the vehicle speed is measured by two pressure sensors installed on the road with a fixed distance. An image-based dynamic license plate recognition technique is then developed for robust vehicle identification. The speed bump is released with a cushion mechanism if the vehicle motion is in a pre-defined speed range or the license plate is registered in the cloud database. With the objective of intelligent traffic flow control in transportation systems, the proposed technique is able to offer safer and more comfortable driving experiences. In the experiments, a miniature prototype system is implemented using 3D printing construction and embedded platform. The dynamic license plate recognition performed using real-world images has demonstrated the feasibility of our intelligent speed bump for the transportation infrastructure. It is able to achieve the identification rate of 96.67% with the real-time processing capability on a PC-based platform.


I. INTRODUCTION
Due to the advances of sensor technologies and the Internet of Things (IoT), intelligent systems have been widely available and adopted for many applications in the past few decades. Recent developments of embedded platforms for transportation systems also emphasizes the capability of intelligence in various aspects. Thus, it is important to design the embedded intelligent systems in a more creative way with mobility and connectivity for task execution. This is not only an emerging topic for academic investigation, but also has practical uses in real applications. In the intelligent system development, it is necessary to consider a number of specific characteristics. This might consist of customized design, sensor fusion, automatic processing and easy to use, especially for the outdoor traffic applications. With a high level system integration, the sensor data are used directly for decision making under the unmanned environment.
For intelligent transportation related works, there are various applications on traffic management, electronic tolling, security and surveillance, etc. In general, the traffic data are collected by the on-site sensing devices, and transmitted to The associate editor coordinating the review of this manuscript and approving it for publication was Chunsheng Zhu . the computing facilities via wired or wireless communications for processing. The derived information is then utilized to generate the control signals for subsequent actions, or provided for traffic analysis. In this work, a new concept of intelligent traffic control techniques using the IoT technology is proposed. The objective is to design and develop a sensor enabled mechatronics system which can integrate with the road transport infrastructure to improve the driving safety and comfort.
With the increasing number of vehicles on the road, speed bumps (or speed humps) are deployed on some specific road sections to reduce the driving speed [1]. They are commonly seen in many places to protect the pedestrians and drivers from the traffic accidents, especially in the neighborhoods of hospitals, schools, parking lots, or particular road junctions. The basic idea is to remind the drivers to pay more attentions to the surroundings and slow down the vehicle speed. However, the current design of speed bumps has many drawbacks in various application scenarios. For example, the speed bumps installed in the hospital districts will make the patients suffer great discomfort, and cause the delay of emergency treatment. Even in the general situations, speed bumps still make drivers and passengers uncomfortable under slow vehicle movement. Thus, there are image-based techniques developed exclusively for the speed bump detection [2]- [4]. However, it is more desirable to have an intelligent mechanism which enforces the speed bump to decelerate the incoming vehicles selectively based on additional information or other criteria.
The conventional speed bumps are made of concrete, asphalt or rubber, and fixed on the road surface. They use the vertical deflection to slow down the incoming traffic and improve the safety conditions. There also exist portable speed bumps with modular design which can be disassembled into a few pieces [5]. This type of speed bumps is usually adopted for temporal situations such as in the scene of a traffic accident. Although it is an efficient way to reduce the vehicle speed, the use of speed bumps is also controversial due to the noise, possible vehicle damage, and slowing down the emergency vehicles. In order to mitigate the impact of road bumps, the retractable design is proposed with the consideration of driving speed [6]. Nevertheless, it is still lack of vehicle identification to allow the unrestricted passing.
To identify a vehicle, the most common method is based on the recognition of its license plate. It is one important research topics and has been extensively investigated over the years [7]. In recent years, automatic license plate recognition techniques are available in many facilities, including small-scaled parking systems and the large-scaled electronic toll collection systems (ETCs). In addition to the use of license plates, there also exist other vehicle identification methods based on active sensing. RFID is one popular technique and can be used under various illumination and weather conditions [8], [9]. However, it requires the RFID tag pre-installed on the vehicle and performing close range scanning for verification. An illustration of the proposed intelligent speed bump system adopted in the transportation infrastructure. The vehicle speed is measured using two pressure sensors installed on the road. A camera is used for image acquisition and vehicle identification.
In this work, we present a prototype and implementation of intelligent speed bump. It is designed to make the road bump operate adaptively based on the vehicle identification or the driving speed [10]. As illustrated in Fig. 1, the vehicle speed is measured by the time span to activate two pressure sensors installed on the road with a fixed distance. An imagebased license plate recognition technique is then carried out to identify the approaching vehicle. The speed bump is released with a cushion mechanism if the vehicle motion is in a pre-defined speed range, or the license plate is registered in the database. The core idea of the adaptive bump system is the road bump which can operate adaptively based on the vehicle identification or the driving speed. Its objective is to allow the vehicles obeying the traffic speed regulation or certain types of vehicles such as ambulances not necessary to suffer from the road bump. In the experiment, we design and implement a prototype bump system using 3D printing construction and embedded control. The license plate recognition performed using real-world images has demonstrated the feasibility of the proposed intelligent speed bump concept for the transportation infrastructure.

II. RELATED WORK
In this paper, we propose a mechatronic system for intelligent speed bump control. Since the idea has not been explored before, the survey of the previous works is mainly limited to the individual components, instead of the application-oriented integration and development. There are two fundamental building blocks in the proposed system, including the design of an adjustable road bump mechanism and license plate recognition for moving vehicles.
To enforce a vehicle to reduce the moving speed, the speed bump is a commonly adopted traffic control facility. However, the abrupt ridge on the road surface has always made driving uncomfortable [11]. To deal with this problem, different types of bump modeling have been proposed in the past few years [12], [13]. This also includes the so-called speed hump, which allows to slow down the vehicles under faster speeds moderately [14]. Current modification of speed bumps can be categorized into the designs with electronic, hydraulic and mechanic structures [15], [16]. The electronic speed bumps utilize accelerometers to measure the vibration generated by the vehicle's tires rolling over. It is then used to control the retraction of the bump. The hydraulic and mechanical structures emphasize the methods to lift and control of the arch part of the speed bump.
More recently, the idea of using liquid for the speed bump implementation has been proposed. MAT Foundry presented a technique to adjust the hardness of viscous fluid inside the bump based on the vehicle speed [17]. The liquid bump is able to slow down the vehicles which exceed the speed limit, while the drivers who obey the traffic laws can travel without concern. With a similar purpose, Edeva proposed an Actibump which will be activated based on the vehicle speed [18]. It uses a radar to measure the speed of an approaching vehicle. The Actibump remains flat if the vehicle is within the speed limit. If the driving is too fast, the metal surface panel sinks several centimeters into the road obliquely. Nevertheless, these techniques do not take the identification of the vehicles into consideration for automated speed enforcement.
Although the literature survey on the advance of speed bump is not comprehensive in academic research, there are a number of patents over the years due to its importance in the practical applications [19], [20]. The conventional speed bumps are still widely used currently due to the advantages on the installation, cost, maintenance and durability, etc. To make the speed bump more intelligent, it is favorable to have various functions instead of the only deceleration purpose. An interesting practice is to combine the vehicle identification or classification with the speed bump applications. In this regard, the vision-based technique is currently the most popular approach. It is commonly used for the license plate recognition task in the intelligent transportation applications [21].
While the recognition of license plate can be carried out using either mobile or static cameras, most application scenarios only require to adopt a static camera for image acquisition. The license plate recognition process generally contains three stages, and takes a still image as input [22]. In the first stage, the location of the license plate is detected in the input image. The traditional methods such as wavelet transform [23], vector quantization [24] and Gaussian mixture model [25], are based on the local information. With the license plate location, the characters are then extracted in the second stage. A commonly adopted technique is to find the connected components in the binarized image. The last stage is to recognize the individual license plate characters. Various character recognition algorithms such as support vector machine (SVM) [26], template matching [27] and neural networks [28] can be performed for classification.
More recently, the deep learning based approaches are used for the license plate recognition task. Zhang et al. presented a two-stage license plate localization method based on the joint of CNN and RNN [29]. Although the proposed algorithm is able to achieve a high detection rate in unconstrained scenes, the character recognition for the license plates is not provided. In [30], Kong et al. applied the federated learning approach for license plate recognition on mobile devices. To deal with the privacy issue, the model training is performed locally, which results in limited resources for computation.

III. SYSTEM OVERVIEW
Our proposed system development for the intelligent speed bump consists of a prototype of the road bump design and the license plate recognition algorithm for dynamic vehicle image sequences. A miniature system is constructed using 3D printing to simulate the real-world application scenario where the camera and the speed bump are installed on the road.   on the left-hand side, and the driver circuit and other hardware equipment for sensing and perception are shown on the righthand side. The simulation environment setup of the prototype system is illustrated in Fig. 3. It consists of an embedded computing platform, a driver circuit, pressure sensors, an industrial camera, and a 3D printed bump structure. The license plate recognition algorithm is executed on the embedded platform for real-time processing, and the release/lock of the road bump is controlled by the driver circuit. Fig. 4 illustrates the 3D CAD model of the proposed speed bump in a modular design. Inspired by the speed bump energy harvester proposed by Wang et al. [31], there are four springs used to support the bump structure in their resting positions. When a vehicle is passing through, the springs will be compressed and restored if the speed bump is not activated. As shown in Fig. 5(a), an electromagnetic sucker is installed on the road cross-section and adjacent to the speed bump module on both sides. An iron shard is fixed on each side of the bump module. These two pairs of electromagnetic sucker and iron shard are used to control the activation of the bump according to the results of license plate recognition and speed detection. Two pressure sensors attached on the road and the driver circuit is shown in Fig. 5(b). The vehicle speed is then derived based on the activation time difference of the pressure sensors with a fixed separation distance.
In the simulation environment, the camera is placed in front of the speed bump for video sequence acquisition. The images are transmitted to a laptop computer for display and PC-based testing. To reduce the computation on the embedded platform for real-time license plate recognition, an adaptive processing VOLUME 10, 2022 strategy is carried out using the imaging geometry and camera settings in the environment. More specifically, the constraints such as the relationship between the vehicle distance and the camera's field-of-view (FOV) are adopted to restrict the region of interest for image processing. After the license plate recognition algorithms (detailed in the following section) are performed, the recognition result is compared with the license plate database for identification. In the real application scenarios, a traffic sign will be placed before the intelligent speed bump by about 100 -200 meters for notification. This is based on an approximate distance for the license plate recognition.

IV. DYNAMIC LICENSE PLATE RECOGNITION
Most of license plate recognition techniques in the literature take static images as input for processing. They are suitable for the parking lot applications where the vehicles commonly remain static for license plate detection and recognition. In the proposed system, we aim to identify the vehicle while it is in motion. Due to the dynamic nature of traffic scenes, it is a key requirement for the operation of adaptive bumps. Since the temporal change is involved, several key factors, such as the image quality, robustness and fast processing, need to be considered. As the flowchart illustrated in Figure 6, a fourstage algorithm for dynamic scene recognition is proposed. It consists of image sequence acquisition and pre-processing, license plate localization, character region segmentation, and character recognition.
The image sequence is captured for license plate recognition when a vehicle approaches the speed bump. Since the camera position and orientation are fixed with respect to the incoming traffic, a subimage region is extracted as the region of interest (ROI) for the subsequent processing. A background subtraction approach is adopted for vehicle detection. First, the background modeling is performed without the dynamic objects. The pixel-wise subtraction between the input image and the background model is then used to identify the vehicle region.
Let B t (x, y) and I t (x, y) represent the background model and the input image at time t, respectively. The initial background image is created by taking the average of the previous input images with static scenes, i.e., where N denotes the number of image frames for modeling. It will be updated over time if no foreground object is detected in the current frame. To make the background image more robust under the illumination change of the outdoor scene, we use a mixture model which takes the weighted sum of the previous background and the current input. The new background image is given by where the parameter λ = N /(N + 1) is the weighting for the previous background for update. The foreground subtraction is then derived by the image difference, i.e., at time t. The foreground image F t (x, y) is finally calculated by where a pixel is assigned to the foreground if the intensity difference is greater than a threshold τ . For the image containing a vehicle, the license plate localization is the first step prior to the recognition task. To locate the license plate, the Sobel edge detector is used to extract the features of the characters. In general, the characters are arranged horizontally in the license plates. Thus, a horizontal Sobel mask is more efficient to detect the vertical edges of the characters, and provides better character extraction results. To ensure that all license plate characters form a connected component, the morphological operations with dilation and erosion are applied on the resulting vertical edge image. A pre-defined aspect ratio is then used to extract the license plate region. After the license plate region is extracted, the segmentation of characters and subsequent processing are carried out on the binarized image. The separation of character and non-character regions from the image binarization algorithm is essential for the recognition task. This work modifies the adaptive binarization technique proposed by Sauvola and Pietikäinen [32]. An image is partitioned into several subregions, and the suitable threshold is derived to perform the image binarization individually. First, the adaptive threshold T (x, y) is calculated by where µ(x, y), σ (x, y) are the mean and standard deviation of the image area, respectively. It is used to determine a suitable threshold to binarize the image adaptively. The parameter R is the range of the standard deviation, and κ is a coefficient determined empirically (κ = 0.5 is used in the experiments). As shown in Fig. 7, it provides noticeable improvements compared to the binary images obtained using a fixed threshold and the conventional Otsu's method [33]. The next step is to find the candidate characters using the license plate detection result. One important restriction for the region extraction is the fixed number of characters appeared in the license plate. Since the connected components found in the image do not necessarily represent the correct characters, additional constraints are required for the character region selection. The rules we adopt to extract the bounding box are tabulated in Table 1. It consists of the aspect ratios of characters and the relationship between the characters and the plate. Fig. 8 illustrates an example of detection result with six characters. If an incorrect number of characters is found, a further processing stage to merge or split the connected components is carried out. An example with noisy license plate and false character extraction is shown in Fig. 9. This issue can be resolved using the inherent character locations in the central part of the license plate. A standard character region can be derived by sorting the candidate regions based on their size. The non-text region of the license plate can be filtered out by the height of the standard character. Finally, the image is binarized with an adaptive threshold, and the potential character regions can be derived.
For the dynamic license plate detection with a consecutive image input, the previous frame can provide a candidate region to restrict the search range. The license plate can be localized quickly by an adaptive tracking strategy. Once the license plate is detected and tracked successfully, there will be no need to perform the vehicle detection step. The implementation of our adaptive license plate localization is based on the tracking result of the character segmentation. If the candidate region in the current image contains characters, only a fixed search range in the neighborhood is carried out for license plate tracking in the next frame.
In addition to the detection of license plates, we propose an adaptive character labeling technique to automatically trace the characters in consecutive image frames. During the character segmentation step, the regions under processing can be clearly identified. Thus, we are able to predict the possible locations of missing characters using the temporal correlation. Fig. 10 illustrates an example of the initial character locations detected with six connected components and successfully labeled. The center position of each bounding box marked in red is used to represent the character for tracking. The recognition rates of individual characters are increased when the candidate regions are identified more accurately.
Due to the upward-looking camera setting, the center positions of the characters in the image sequence will change when the vehicle is approaching. To facilitate the tracking, the following procedure is carried out to estimate the character locations in the succeeding frame.
Step1: Identify the individual character regions, update the initial center points, and label the character sequence. Step2: If all character regions are identified in the succeeding frame, update the center points. Step3: If the center points fall within the character regions in the succeeding image, update the center points and go to Step2.   Our character recognition technique is built on top of the open source tool ''Tesseract OCR'' developed by Smith [34]. The algorithm is based on template matching and suitable for our real-time processing requirement. In addition to the fast recognition, the technique is able to build text samples easily, and supports character recognition for multiple languages. In our implementation, the character samples can be selftrained. It is based on the availability of known license plate characters. To reduce the computation for character classification, the features are extracted based on the polygon model. As illustrated in Fig. 11(a), the feature vector is approximated by the horizontal and vertical positions, orientation, and length of the character. The polygon features are then used for character recognition by template matching as shown in Fig. 11(b).
Based on the recognition reliability ranges from 0 to 100 of Tesseract OCR, our character recognition scoring mechanism is given in Table 2. In the experiments, we find the highest score from the 36 candidate characters (A-Z, 0-9) as the final recognition result. There are some common recognition errors in the license plate characters, such as 0 and D, 2 and Z, 1 and I, and 8 and B. A post-processing stage with the horizontal and vertical projections is carried out on these characters. The prior knowledge of local analysis as shown in Fig. 12 is utilized to distinguish the similar characters. The identification is verified by the spatial features on the corners and sides of the characters.

V. IMPLEMENTATION AND EXPERIMENTAL RESULTS
In the experiments, a prototype intelligent speed bump system is implemented using an embedded platform. It includes a   vision-based vehicle detection and license plate recognition system, a speed sensing and hardware control system, and a structural design of the speed bump. The miniature road bump created by 3D printing is shown in Fig. 13. There are four springs installed below the road bump. When the electromagnetic suckers mounted on the sides of the bump structure are disabled, the springs will be compressed and stretched to their resting positions. We adopt two pressure sensors for vehicle speed measurement and two electromagnetic suckers for the activation of the bump. These devices are connected to and controlled by an Arduino embedded board. In the current implementation, dynamic license plate recognition is carried out on the vehicles in the outdoor scenes, and the activation of  the bump is performed on our prototype system. The access control signal is then passed to the adaptive bump module for operation.
For the license plate detection and recognition, a PointGrey USB 3.0 camera Flea 3 (FL3-U3-32S2C/M-CS) is adopted. It provides the frame rate at 60 fps using a CMOS sensor with the pixel size of 2.5 µm. A fixed 6 mm lens is attached to the camera for image acquisition. The image sequences are captured by a camera placed on the road and in front of the speed bump. It is fixed at the middle of a lane to allow the incoming traffic. The working range for image acquisition is about 40 meters away from the camera. This is equivalent to approximately 2 seconds of travel time at the driving speed of 72 km/hr. It provides 60 at least image frames for license plate recognition. In addition, the response time of 0.5 second is reserved for the road bump control. Thus, the region-ofinterest of 600 × 400 pixels is extracted from the original image (with the resolution 1600 × 900) corresponding to the closest distance for vehicle detection.
Several video sequences are acquired for the outdoor experiment, and some images containing the license plate with 6 characters are shown in Fig. 14(a). The vehicles are driving at the speed of about 10 -30 km/hr on campus. We collect a total of 31 video sequences, with each corresponding to a different vehicle for license plate recognition. Fig. 14(b) shows the result of vehicle detection using background subtraction with morphological operations for noise removal. A structure element of 11 × 5 is used to perform the dilation and erosion, and ensure the completeness of the license plate. It is then followed by the license plate localization. The adaptive tracking increases the localization efficiency and avoids the false detection of incorrect connected components. To incorporate the dynamic information for license plate recognition, six array registers are established, with each of them corresponding to 36 candidate characters. The character recognition confidences obtained from the consecutive image frames are accumulated for each array register. A voting scheme is then carried out to derive the final recognition result. Our algorithms are tested on two embedded platforms (Raspberry Pi and Banana Pi) and a    Table 3 tabulates the hardware specifications for performance evaluation. The parameters such as processing time and image resolution are tabulated in Table 4. It shows the accuracy of 96.67% for the dynamic license plate recognition.

VI. CONCLUSION
In this paper, we propose an intelligent speed bump system with the capabilities of vehicle identification and speed detection for the intelligent traffic flow control. A mechatronic bump system is designed and implemented using 3D printing construction and embedded platform. The vehicle speed is measured by the activation two pressure sensors installed on the road with a fixed distance. An image-based dynamic license plate recognition technique is developed with adaptive tracking and character labeling using temporal information for robust classification. If the vehicle speed is within a pre-defined range or the license plate has the privilege for quick access, the road bump will be disabled to provide safer and more comfortable driving experiences. In the experimental results carried out on the embedded platform and the outdoor scenes, the feasibility of the proposed intelligent speed bump system has been demonstrated. In the future work, the information system will be established with a database for license plate retrieval and matching. He is the author of more than 170 journal and conference papers and has written two book chapters. He also holds 11 U.S. patents and nine Taiwan patents. His research interests include robotics, computer vision, machine learning, and image processing. He is a Senior Member of OSA and a fellow of IET. He is an Organizing Committee Member and a Program Committee Member of more than 50 international conferences. He was a recipient of the Outstanding Robotics Engineer Award from Robotics Society of Taiwan, the Outstanding Academic-Industry Cooperation Award from Taiwan Association of System and Science and Engineering, and the Excellent Research Award from National Chung Cheng University.
CHENG-YU HO received the B.S. degree in electrical engineering from the National Yunlin University of Science and Technology and the M.S. degree in electrical engineering from National Chung Cheng University, Taiwan. His research interests include image processing, computer vision, robotics, and mechatronics.