Abstract:
Point clouds have been recently used in applications involving real-time capture and rendering of 3D objects. In a point cloud, for practical reasons, each point or voxel...Show MoreMetadata
Abstract:
Point clouds have been recently used in applications involving real-time capture and rendering of 3D objects. In a point cloud, for practical reasons, each point or voxel is usually associated with one single color along with other attributes. The region-adaptive hierarchical transform (RAHT) coder has been proposed for single-color point clouds. The cloud is usually captured by many cameras and the colors are averaged in some fashion to yield the point color. This approach may not be very realistic since, in real world objects, the reflected light may significantly change with the viewing angle, especially if specular surfaces are present. For that, we are interested in a more complete representation, the plenoptic point cloud, wherein every point has associated colors in all directions. Here, we propose a compression method for such a representation. Instead of encoding a continuous function, since there is only a finite number of cameras, it makes sense to compress as many colors per voxel as cameras, and to leave any intermediary color rendering interpolation to the decoder. Hence, each voxel is associated with a vector of color values, for each color component. We have here developed and evaluated four methods to expand the RAHT coder to encompass the multiple colors case. Experiments with synthetic data helped us to correlate specularity with the compression, since object specularity, at a given point in space, directly affects color disparity among the cameras, impacting the coder performance. Simulations were carried out using natural (captured) data and results are presented as rate-distortion curves that show that a combination of Kahunen-Loève transform and RAHT achieves the best performance.
Published in: IEEE Transactions on Image Processing ( Volume: 28, Issue: 3, March 2019)
Funding Agency:
Department of Electrical Engineering, Universidade de Brasilia, Brasília, Brazil
Gustavo Sandri (M’14) received the Engineer degree from the Universidade de Brasilia, Brazil, in 2013, the master’s degree in signal and image processing by a partnership between the Université de Bordeaux and ENSEIRB-MATMECA, Bordeaux, France, in 2013, and the M.Sc. degree in signal processing from the Universidade de Brasilia in 2016, where he is currently pursuing the Ph.D. degree. Since 2018, he has been a Lecturer of...Show More
Gustavo Sandri (M’14) received the Engineer degree from the Universidade de Brasilia, Brazil, in 2013, the master’s degree in signal and image processing by a partnership between the Université de Bordeaux and ENSEIRB-MATMECA, Bordeaux, France, in 2013, and the M.Sc. degree in signal processing from the Universidade de Brasilia in 2016, where he is currently pursuing the Ph.D. degree. Since 2018, he has been a Lecturer of...View more
Department of Electrical Engineering, Universidade de Brasilia, Brasília, Brazil
Ricardo L. de Queiroz (M’88–SM’99–F’17) received the Engineer degree from the Universidade de Brasilia, Brazil, in 1987, the M.Sc. degree from Universidade Estadual de Campinas, Brazil, in 1990, and the Ph.D. degree from The University of Texas at Arlington, in 1994, all in electrical engineering.
From 1990 to 1991, he was with the DSP Research Group, Universidade de Brasilia, as a Research Associate. He joined Xerox Corpo...Show More
Ricardo L. de Queiroz (M’88–SM’99–F’17) received the Engineer degree from the Universidade de Brasilia, Brazil, in 1987, the M.Sc. degree from Universidade Estadual de Campinas, Brazil, in 1990, and the Ph.D. degree from The University of Texas at Arlington, in 1994, all in electrical engineering.
From 1990 to 1991, he was with the DSP Research Group, Universidade de Brasilia, as a Research Associate. He joined Xerox Corpo...View more
Google, Seattle, WA, USA
Philip A. Chou (M’81–SM’00–F’03) received the B.S.E. degree in electrical engineering and computer science from Princeton University, Princeton, NJ, USA, and the M.S. degree in electrical engineering and computer science from the University of California at Berkeley, Berkeley, in 1980 and 1983, respectively, and the Ph.D. degree in electrical engineering from Stanford University in 1988. From 1988 to 1990, he was a member...Show More
Philip A. Chou (M’81–SM’00–F’03) received the B.S.E. degree in electrical engineering and computer science from Princeton University, Princeton, NJ, USA, and the M.S. degree in electrical engineering and computer science from the University of California at Berkeley, Berkeley, in 1980 and 1983, respectively, and the Ph.D. degree in electrical engineering from Stanford University in 1988. From 1988 to 1990, he was a member...View more
Contents Department of Electrical Engineering, Universidade de Brasilia, Brasília, Brazil
Gustavo Sandri (M’14) received the Engineer degree from the Universidade de Brasilia, Brazil, in 2013, the master’s degree in signal and image processing by a partnership between the Université de Bordeaux and ENSEIRB-MATMECA, Bordeaux, France, in 2013, and the M.Sc. degree in signal processing from the Universidade de Brasilia in 2016, where he is currently pursuing the Ph.D. degree. Since 2018, he has been a Lecturer of electronics with the Instituto Federal de Brasilia.
Gustavo Sandri (M’14) received the Engineer degree from the Universidade de Brasilia, Brazil, in 2013, the master’s degree in signal and image processing by a partnership between the Université de Bordeaux and ENSEIRB-MATMECA, Bordeaux, France, in 2013, and the M.Sc. degree in signal processing from the Universidade de Brasilia in 2016, where he is currently pursuing the Ph.D. degree. Since 2018, he has been a Lecturer of electronics with the Instituto Federal de Brasilia.View more
Department of Electrical Engineering, Universidade de Brasilia, Brasília, Brazil
Ricardo L. de Queiroz (M’88–SM’99–F’17) received the Engineer degree from the Universidade de Brasilia, Brazil, in 1987, the M.Sc. degree from Universidade Estadual de Campinas, Brazil, in 1990, and the Ph.D. degree from The University of Texas at Arlington, in 1994, all in electrical engineering.
From 1990 to 1991, he was with the DSP Research Group, Universidade de Brasilia, as a Research Associate. He joined Xerox Corporation in 1994, where he was a member of the research staff until 2002. From 2000 to 2001, he was also an Adjunct Faculty with the Rochester Institute of Technology. He joined the Electrical Engineering Department, Universidade de Brasilia, in 2003. In 2010, he became a Full (Titular) Professor with the Computer Science Department, Universidade de Brasilia. Since 2015, he has been a Visiting Professor with the University of Washington, Seattle.
Dr. de Queiroz has published extensively in Journals and conferences and contributed chapters to books as well. He also holds 46 issued patents. He is a past Elected Member of the IEEE Signal Processing Society’s Multimedia Signal Processing (MMSP) and the Image, Video and Multidimensional Signal Processing Technical Committees. He is a an Editor for the IEEE Transactions on Image Processing and a past Editor for the EURASIP Journal on Image and Video Processing, the IEEE Signal Processing Letters, and the IEEE Transactions on Circuits and Systems for Video Technology. He has been appointed an IEEE Signal Processing Society Distinguished Lecturer from 2011 to 2012.
His research interests include image and video compression, point cloud compression, multirate signal processing, and color imaging. He is a member of the Brazilian Telecommunications Society. He has been actively involved with IEEE Signal Processing Society chapters in Brazil and in U.S. He was the General Chair of ISCAS’2011, MMSP’2009, and SBrT’2012. He was also part of the organizing committee of many SPS flagship conferences.
Ricardo L. de Queiroz (M’88–SM’99–F’17) received the Engineer degree from the Universidade de Brasilia, Brazil, in 1987, the M.Sc. degree from Universidade Estadual de Campinas, Brazil, in 1990, and the Ph.D. degree from The University of Texas at Arlington, in 1994, all in electrical engineering.
From 1990 to 1991, he was with the DSP Research Group, Universidade de Brasilia, as a Research Associate. He joined Xerox Corporation in 1994, where he was a member of the research staff until 2002. From 2000 to 2001, he was also an Adjunct Faculty with the Rochester Institute of Technology. He joined the Electrical Engineering Department, Universidade de Brasilia, in 2003. In 2010, he became a Full (Titular) Professor with the Computer Science Department, Universidade de Brasilia. Since 2015, he has been a Visiting Professor with the University of Washington, Seattle.
Dr. de Queiroz has published extensively in Journals and conferences and contributed chapters to books as well. He also holds 46 issued patents. He is a past Elected Member of the IEEE Signal Processing Society’s Multimedia Signal Processing (MMSP) and the Image, Video and Multidimensional Signal Processing Technical Committees. He is a an Editor for the IEEE Transactions on Image Processing and a past Editor for the EURASIP Journal on Image and Video Processing, the IEEE Signal Processing Letters, and the IEEE Transactions on Circuits and Systems for Video Technology. He has been appointed an IEEE Signal Processing Society Distinguished Lecturer from 2011 to 2012.
His research interests include image and video compression, point cloud compression, multirate signal processing, and color imaging. He is a member of the Brazilian Telecommunications Society. He has been actively involved with IEEE Signal Processing Society chapters in Brazil and in U.S. He was the General Chair of ISCAS’2011, MMSP’2009, and SBrT’2012. He was also part of the organizing committee of many SPS flagship conferences.View more
Google, Seattle, WA, USA
Philip A. Chou (M’81–SM’00–F’03) received the B.S.E. degree in electrical engineering and computer science from Princeton University, Princeton, NJ, USA, and the M.S. degree in electrical engineering and computer science from the University of California at Berkeley, Berkeley, in 1980 and 1983, respectively, and the Ph.D. degree in electrical engineering from Stanford University in 1988. From 1988 to 1990, he was a member of Technical Staff at AT & T Bell Laboratories, Murray Hill, NJ, USA. From 1990 to 1996, he was a member of Research Staff, Xerox Palo Alto Research Center, Palo Alto, CA, USA. In 1997, he was a Manager of the Compression Group, VXtreme, Mountain View, CA, USA, an Internet video startup before it was acquired by Microsoft. From 1998 to 2016, he was a Principal Researcher with Microsoft Research, Redmond, WA, USA, where he was managing the Communication and Collaboration Systems Research Group from 2004 to 2011. He served as a Consulting Associate Professor with Stanford University from 1994 to 1995, an Affiliate Associate Professor with the University of Washington from 1998 to 2009, and has been an Adjunct Professor with the Chinese University of Hong Kong since 2006. He was with a startup, 8i.com, where he led the effort to compress and communicate volumetric media, popularly known as holograms, for virtual and augmented reality. In 2018, he joined the Google Daydream Group as a Research Scientist.
Philip A. Chou (M’81–SM’00–F’03) received the B.S.E. degree in electrical engineering and computer science from Princeton University, Princeton, NJ, USA, and the M.S. degree in electrical engineering and computer science from the University of California at Berkeley, Berkeley, in 1980 and 1983, respectively, and the Ph.D. degree in electrical engineering from Stanford University in 1988. From 1988 to 1990, he was a member of Technical Staff at AT & T Bell Laboratories, Murray Hill, NJ, USA. From 1990 to 1996, he was a member of Research Staff, Xerox Palo Alto Research Center, Palo Alto, CA, USA. In 1997, he was a Manager of the Compression Group, VXtreme, Mountain View, CA, USA, an Internet video startup before it was acquired by Microsoft. From 1998 to 2016, he was a Principal Researcher with Microsoft Research, Redmond, WA, USA, where he was managing the Communication and Collaboration Systems Research Group from 2004 to 2011. He served as a Consulting Associate Professor with Stanford University from 1994 to 1995, an Affiliate Associate Professor with the University of Washington from 1998 to 2009, and has been an Adjunct Professor with the Chinese University of Hong Kong since 2006. He was with a startup, 8i.com, where he led the effort to compress and communicate volumetric media, popularly known as holograms, for virtual and augmented reality. In 2018, he joined the Google Daydream Group as a Research Scientist.View more
