Deep Stereo Fusion: Combining Multiple Disparity Hypotheses with Deep-Learning | IEEE Conference Publication | IEEE Xplore

Deep Stereo Fusion: Combining Multiple Disparity Hypotheses with Deep-Learning


Abstract:

Stereo matching is a popular technique to infer depth from two or more images and wealth of methods have been proposed to deal with this problem. Despite these efforts, f...Show More

Abstract:

Stereo matching is a popular technique to infer depth from two or more images and wealth of methods have been proposed to deal with this problem. Despite these efforts, finding accurate stereo correspondences is still an open problem. The strengths and weaknesses of existing methods are often complementary and in this paper, motivated by recent trends in this field, we exploit this fact by proposing Deep Stereo Fusion, a Convolutional Neural Network capable of combining the output of multiple stereo algorithms in order to obtain more accurate result with respect to each input disparity map. Deep Stereo Fusion process a 3D features vector, encoding both spatial and cross-algorithm information, in order to select the best disparity hypothesis among those proposed by the single stereo matchers. To the best of our knowledge, our proposal is the first i) to leverage on deep learning and ii) able to predict the optimal disparity assignments by taking only as input cue the disparity maps. This second feature makes our method suitable for deployment even when other cues (e.g., confidence) are not available such as when dealing with disparity maps provided by off-the-shelf 3D sensors. We thoroughly evaluate our proposal on the KITTI stereo benchmark with respect state-of-the-art in this field.
Date of Conference: 25-28 October 2016
Date Added to IEEE Xplore: 19 December 2016
ISBN Information:
Conference Location: Stanford, CA, USA
Citations are not available for this document.

Cites in Papers - |

Cites in Papers - IEEE (15)

Select All
1.
Erik Sandström, Kevin Ta, Luc Van Gool, Martin R. Oswald, "UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM", 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pp.4539-4550, 2023.
2.
James Okae, Bohan Li, Juan Du, Yueming Hu, "Robust Scale-Aware Stereo Matching Network", IEEE Transactions on Artificial Intelligence, vol.3, no.2, pp.244-253, 2022.
3.
Matteo Poggi, Fabio Tosi, Filippo Aleotti, Stefano Mattoccia, "Leveraging a weakly adversarial paradigm for joint learning of disparity and confidence estimation", 2020 25th International Conference on Pattern Recognition (ICPR), pp.270-277, 2021.
4.
Matteo Poggi, Fabio Tosi, Stefano Mattoccia, "Good Cues to Learn From Scratch a Confidence Measure for Passive Depth Sensors", IEEE Sensors Journal, vol.20, no.22, pp.13533-13541, 2020.
5.
Matteo Poggi, Gianluca Agresti, Fabio Tosi, Pietro Zanuttigh, Stefano Mattoccia, "Confidence Estimation for ToF and Stereo Sensors and Its Application to Depth Data Fusion", IEEE Sensors Journal, vol.20, no.3, pp.1411-1421, 2020.
6.
Max Mehltretter, Christian Heipke, "CNN-Based Cost Volume Analysis as Confidence Measure for Dense Matching", 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp.2070-2079, 2019.
7.
Can Pu, Robert B. Fisher, "UDFNET: Unsupervised Disparity Fusion with Adversarial Networks", 2019 IEEE International Conference on Image Processing (ICIP), pp.1765-1769, 2019.
8.
Maxime Ferrera, Alexandre Boulch, Julien Moras, "Fast Stereo Disparity Maps Refinement By Fusion of Data-Based And Model-Based Estimations", 2019 International Conference on 3D Vision (3DV), pp.9-17, 2019.
9.
Fabio Tosi, Matteo Poggi, Stefano Mattoccia, "Leveraging Confident Points for Accurate Depth Refinement on Embedded Systems", 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp.158-167, 2019.
10.
Hae-Gon Jeon, Jaesik Park, Gyeongmin Choe, Jinsun Park, Yunsu Bok, Yu-Wing Tai, In So Kweon, "Depth from a Light Field Image with Learning-Based Matching Costs", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.41, no.2, pp.297-310, 2019.
11.
Konstantinos Batsos, Changjiang Cai, Philippos Mordohai, "CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation", 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.2060-2069, 2018.
12.
Matteo Poggi, Fabio Tosi, Stefano Mattoccia, "Quantitative Evaluation of Confidence Measures in a Machine Learning World", 2017 IEEE International Conference on Computer Vision (ICCV), pp.5238-5247, 2017.
13.
Alessio Tonioni, Matteo Poggi, Stefano Mattoccia, Luigi Di Stefano, "Unsupervised Adaptation for Deep Stereo", 2017 IEEE International Conference on Computer Vision (ICCV), pp.1614-1622, 2017.
14.
Matteo Poggi, Stefano Mattoccia, "Learning to Predict Stereo Reliability Enforcing Local Consistency of Confidence Maps", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.4541-4550, 2017.
15.
Matteo Poggi, Stefano Mattoccia, "Learning a General-Purpose Confidence Measure Based on O(1) Features and a Smarter Aggregation Strategy for Semi Global Matching", 2016 Fourth International Conference on 3D Vision (3DV), pp.509-518, 2016.

Cites in Papers - Other Publishers (5)

1.
Erik Sandstrom, Martin R. Oswald, Suryansh Kumar, Silvan Weder, Fisher Yu, Cristian Sminchisescu, Luc Van Gool, "Learning Online Multi-sensor Depth Fusion", Computer Vision ? ECCV 2022, vol.13692, pp.87, 2022.
2.
Max Mehltretter, Christian Heipke, "Aleatoric uncertainty estimation for dense stereo matching via CNN-based cost volume analysis", ISPRS Journal of Photogrammetry and Remote Sensing, vol.171, pp.63, 2021.
3.
Matteo Poggi, Filippo Aleotti, Fabio Tosi, Giulio Zaccaroni, Stefano Mattoccia, "Self-adapting Confidence Estimation for Stereo", Computer Vision ? ECCV 2020, vol.12369, pp.715, 2020.
4.
Fabio Tosi, Matteo Poggi, Antonio Benincasa, Stefano Mattoccia, "Beyond Local Reasoning for Stereo Confidence Estimation with Deep Learning", Computer Vision ? ECCV 2018, vol.11210, pp.323, 2018.
5.
Johannes L. Schonberger, Sudipta N. Sinha, Marc Pollefeys, "Learning to Fuse Proposals from Multiple Scanline Optimizations in Semi-Global Matching", Computer Vision ? ECCV 2018, vol.11217, pp.758, 2018.

Contact IEEE to Subscribe

References

References is not available for this document.