Improved semantic segmentation for robotic applications with hierarchical conditional random fields

Conventional approaches to semantic segmentation are inappropriate for robotic applications, as they focus on pixel-level performance and give little significance to spurious object detections. This paper presents a region-based conditional random field model for semantic segmentation that focuses on object-level performance, recognising that in a robotics context, false object detections can have costly consequences. We show how optimising at the semantic region-level results in significantly fewer false positive object detections than conventional approaches. We further show how both object and pixel-level performance can be improved over conventional methods by combining region random fields with dense pixel random fields in a hierarchical manner. An object-aware performance metric is introduced that heavily penalises false positive and false negative object detections, as appropriate for robotic applications. Our approach is evaluated on the challenging NYU v2 and Pascal VOC datasets, outperforming comparable conventional methods in terms of object and pixel-level performance.