Object Detection System in Self-Driving Cars by Calibrated Camera and LiDAR Sensor data-Implementation of the Hybrid PointPillar Method

The object detection system in a self-driving car is a challenging part in the product development as all the decision making is depended on the objects detected by the LiDAR Sensor and the Camera, which then need to be fused by using sensor fusion techniques to process the data for object detection. A deep neural network is required for object detection which takes the camera data, LiDAR data, and calibrated data as the input and produces the object detection results bounded by 3D bounding boxes in camera data, LiDAR data as output. Using a neural network, the point cloud data (PCD) is converted into a 2D grid format in PointPillars method. KITTI training dataset, COCO training dataset is used to train and test the model which provides us a pytorch model, KITTI testing dataset is used to test the outputs of the trained model.