Abstract:
Micro-expression recognition aims to infer genuine emotions that people try to conceal from facial video clips. It is a very challenging task because micro-expressions ha...Show MoreMetadata
Abstract:
Micro-expression recognition aims to infer genuine emotions that people try to conceal from facial video clips. It is a very challenging task because micro-expressions have a very low intensity and short duration, which makes micro-expressions difficult to observe. Recently, researchers have designed various spatiotemporal descriptors to describe micro-expressions. It is notable that for better capturing the low-intensity facial muscle movement, a fixed spatial division grid, 8× 8 for example, is commonly used to partition the facial images into a few facial blocks before extracting descriptors. However, it is hard to choose an ideal division grid for different micro-expression samples because the division grids affect the discriminative ability of spatiotemporal descriptors to distinguish micro-expressions. To address this problem, in this paper, we design a hierarchical spatial division scheme for spatiotemporal descriptor extraction. By using the proposed scheme, it would not be a problem to determine which division grid is most suitable regarding different micro-expression samples. Furthermore, we propose a kernelized group sparse learning (KGSL) model to process hierarchical scheme based spatiotemporal descriptors such that they are more effective for micro-expression recognition tasks. To evaluate the performance of the proposed micro-expression recognition method consisting of the hierarchical scheme based spatiotemporal descriptors and KGSL, extensive experiments are conducted on two public micro-expression databases: CASME II and SMIC. Compared with many recent state-of-the-art approaches, our method achieves more promising recognition results.
Published in: IEEE Transactions on Multimedia ( Volume: 20, Issue: 11, November 2018)
Funding Agency:
Center for Machine Vision and Signal Analysis, Faulty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
Yuan Zong received the B.S. and M.S. degrees in electronics
engineering from Nanjing Normal University, Nanjing, China, in 2011 and 2014, respectively, and currently working
toward the Ph.D. degree at the Key Laboratory of Child Development and Learning Science of Ministry of Education,
School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China. From 2016 to 2017, he was
working as a V...Show More
Yuan Zong received the B.S. and M.S. degrees in electronics
engineering from Nanjing Normal University, Nanjing, China, in 2011 and 2014, respectively, and currently working
toward the Ph.D. degree at the Key Laboratory of Child Development and Learning Science of Ministry of Education,
School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China. From 2016 to 2017, he was
working as a V...View more
Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
Wenming Zheng (M’08) received the B.S. degree in computer
science from Fuzhou University, Fuzhou, China, in 1997, the M.S. degree in computer science from Huaqiao University,
Quanzhou, China, in 2001, and the Ph.D. degree in signal processing from the Southeast University, Nanjing, China, in
2004. Since 2004, he has been with the Research Center for Learning Science, Southeast University. He is currently a
Professor w...Show More
Wenming Zheng (M’08) received the B.S. degree in computer
science from Fuzhou University, Fuzhou, China, in 1997, the M.S. degree in computer science from Huaqiao University,
Quanzhou, China, in 2001, and the Ph.D. degree in signal processing from the Southeast University, Nanjing, China, in
2004. Since 2004, he has been with the Research Center for Learning Science, Southeast University. He is currently a
Professor w...View more
Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China
Xiaohua Huang received the B.S. degree in communication engineering
from Huaqiao University, Quanzhou, Fujian, China, in 2006, and the Ph.D. degree in computer science and engineering
from the University of Oulu, Oulu, Finland in 2014. He has been a scientist researcher in the Center for Machine
Vision and Signal Analysis (CMVS) at University of Oulu since 2015, and has been a senior researcher in the CMVS and a
resea...Show More
Xiaohua Huang received the B.S. degree in communication engineering
from Huaqiao University, Quanzhou, Fujian, China, in 2006, and the Ph.D. degree in computer science and engineering
from the University of Oulu, Oulu, Finland in 2014. He has been a scientist researcher in the Center for Machine
Vision and Signal Analysis (CMVS) at University of Oulu since 2015, and has been a senior researcher in the CMVS and a
resea...View more
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
Zhen Cui (M’14) received the B.S. degree from Shandong Normal
University, Jinan, China, the M.S. degree from Sun Yat-sen University, Guangzhou, China, and the Ph.D. degree from
the, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, in 2004, 2006, and 2014,
respectively. He was a Research Fellow with the Department of Electrical and Computer Engineering, National University
of Singapore, f...Show More
Zhen Cui (M’14) received the B.S. degree from Shandong Normal
University, Jinan, China, the M.S. degree from Sun Yat-sen University, Guangzhou, China, and the Ph.D. degree from
the, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, in 2004, 2006, and 2014,
respectively. He was a Research Fellow with the Department of Electrical and Computer Engineering, National University
of Singapore, f...View more
Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
Guoying Zhao (SM’12) received the Ph.D. degree in computer
science from the Chinese Academy of Sciences, Beijing, China, in 2005. In 2011, she was selected to the highly
competitive academy research fellow position. She was a Nokia Visiting Professor, in 2016. She is currently a
Professor with the Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland, where she has
been a Senior Researcher s...Show More
Guoying Zhao (SM’12) received the Ph.D. degree in computer
science from the Chinese Academy of Sciences, Beijing, China, in 2005. In 2011, she was selected to the highly
competitive academy research fellow position. She was a Nokia Visiting Professor, in 2016. She is currently a
Professor with the Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland, where she has
been a Senior Researcher s...View more
Contents Center for Machine Vision and Signal Analysis, Faulty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
Yuan Zong received the B.S. and M.S. degrees in electronics
engineering from Nanjing Normal University, Nanjing, China, in 2011 and 2014, respectively, and currently working
toward the Ph.D. degree at the Key Laboratory of Child Development and Learning Science of Ministry of Education,
School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China. From 2016 to 2017, he was
working as a Visiting Student with the Center for Machine Vision and Signal Analysis, University of Oulu, Oulu,
Finland. His research interests include affective computing, pattern recognition, and computer vision.
Yuan Zong received the B.S. and M.S. degrees in electronics
engineering from Nanjing Normal University, Nanjing, China, in 2011 and 2014, respectively, and currently working
toward the Ph.D. degree at the Key Laboratory of Child Development and Learning Science of Ministry of Education,
School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China. From 2016 to 2017, he was
working as a Visiting Student with the Center for Machine Vision and Signal Analysis, University of Oulu, Oulu,
Finland. His research interests include affective computing, pattern recognition, and computer vision.View more
Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
Wenming Zheng (M’08) received the B.S. degree in computer
science from Fuzhou University, Fuzhou, China, in 1997, the M.S. degree in computer science from Huaqiao University,
Quanzhou, China, in 2001, and the Ph.D. degree in signal processing from the Southeast University, Nanjing, China, in
2004. Since 2004, he has been with the Research Center for Learning Science, Southeast University. He is currently a
Professor with the School of Biological Sciences and Medical Engineering, Southeast University, and the Key Laboratory
of Child Development and Learning Science of Ministry of Education, Southeast University. His research interests
include neural computation, pattern recognition, machine learning, and computer vision. He is an Associate Editor for
the IEEE Transactions on Affective Computing and Neurocomputing, and a Member of the
Associate Editors-in-Chief of The Visual Computer.
Wenming Zheng (M’08) received the B.S. degree in computer
science from Fuzhou University, Fuzhou, China, in 1997, the M.S. degree in computer science from Huaqiao University,
Quanzhou, China, in 2001, and the Ph.D. degree in signal processing from the Southeast University, Nanjing, China, in
2004. Since 2004, he has been with the Research Center for Learning Science, Southeast University. He is currently a
Professor with the School of Biological Sciences and Medical Engineering, Southeast University, and the Key Laboratory
of Child Development and Learning Science of Ministry of Education, Southeast University. His research interests
include neural computation, pattern recognition, machine learning, and computer vision. He is an Associate Editor for
the IEEE Transactions on Affective Computing and Neurocomputing, and a Member of the
Associate Editors-in-Chief of The Visual Computer.View more
Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China
Xiaohua Huang received the B.S. degree in communication engineering
from Huaqiao University, Quanzhou, Fujian, China, in 2006, and the Ph.D. degree in computer science and engineering
from the University of Oulu, Oulu, Finland in 2014. He has been a scientist researcher in the Center for Machine
Vision and Signal Analysis (CMVS) at University of Oulu since 2015, and has been a senior researcher in the CMVS and a
researcher in R&D team of Sentido Enterprises OY, Finland, since 2017. He has authored or co-authored more than 30
papers in journals and conferences, and has served as a reviewer for journals and conferences. He was awarded Jorma
Ollila Grant by Nokia Foundation. His current research interests include facial expression recognition,
micro-expression analysis, group-level emotion recognition, multi-modal emotion recognition and texture
classification.
Xiaohua Huang received the B.S. degree in communication engineering
from Huaqiao University, Quanzhou, Fujian, China, in 2006, and the Ph.D. degree in computer science and engineering
from the University of Oulu, Oulu, Finland in 2014. He has been a scientist researcher in the Center for Machine
Vision and Signal Analysis (CMVS) at University of Oulu since 2015, and has been a senior researcher in the CMVS and a
researcher in R&D team of Sentido Enterprises OY, Finland, since 2017. He has authored or co-authored more than 30
papers in journals and conferences, and has served as a reviewer for journals and conferences. He was awarded Jorma
Ollila Grant by Nokia Foundation. His current research interests include facial expression recognition,
micro-expression analysis, group-level emotion recognition, multi-modal emotion recognition and texture
classification.View more
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
Zhen Cui (M’14) received the B.S. degree from Shandong Normal
University, Jinan, China, the M.S. degree from Sun Yat-sen University, Guangzhou, China, and the Ph.D. degree from
the, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, in 2004, 2006, and 2014,
respectively. He was a Research Fellow with the Department of Electrical and Computer Engineering, National University
of Singapore, from 2014 to 2015. He also spent half a year as a Research Assistant with Nanyang Technological
University, from June 2012 to December 2012. He is currently a Professor with Nanjing University of Science and
Technology, Nanjing, China. His research interests include computer vision, pattern recognition, and machine learning,
especially focusing on deep learning, manifold learning, sparse coding, face detection/alignment/recognition, object
tracking, image super resolution, emotion analysis, etc.
Zhen Cui (M’14) received the B.S. degree from Shandong Normal
University, Jinan, China, the M.S. degree from Sun Yat-sen University, Guangzhou, China, and the Ph.D. degree from
the, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, in 2004, 2006, and 2014,
respectively. He was a Research Fellow with the Department of Electrical and Computer Engineering, National University
of Singapore, from 2014 to 2015. He also spent half a year as a Research Assistant with Nanyang Technological
University, from June 2012 to December 2012. He is currently a Professor with Nanjing University of Science and
Technology, Nanjing, China. His research interests include computer vision, pattern recognition, and machine learning,
especially focusing on deep learning, manifold learning, sparse coding, face detection/alignment/recognition, object
tracking, image super resolution, emotion analysis, etc.View more
Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
Guoying Zhao (SM’12) received the Ph.D. degree in computer
science from the Chinese Academy of Sciences, Beijing, China, in 2005. In 2011, she was selected to the highly
competitive academy research fellow position. She was a Nokia Visiting Professor, in 2016. She is currently a
Professor with the Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland, where she has
been a Senior Researcher since 2005 and an Associate Professor since 2014. She has authored or coauthored more than
170 papers in journals and conferences. She has authored/edited three books and seven special issues in journals. Her
research has been reported by Finnish TV programs, newspapers, and MIT Technology Review. Her current research
interests include image and video descriptors, facial-expression and microexpression recognition, gait analysis,
dynamictexture recognition, human motion analysis, and person identification. She is a Copublicity Chair for FG2018.
She was Area Chairs for several conferences. She was a Co-Chair of many international workshops at ECCV, ICCV, CVPR,
ACCV, and BMVC. She has lectured tutorials at ICPR 2006, ICCV 2009, and SCIA 2013. She is an Associate Editor for
Pattern Recognition, theIEEE Transactions on Circuits and Systems for Video Technology, and
Image and Vision Computing journals. Her papers have currently more than 7200 citations in Google
Scholar (h-index 36).
Guoying Zhao (SM’12) received the Ph.D. degree in computer
science from the Chinese Academy of Sciences, Beijing, China, in 2005. In 2011, she was selected to the highly
competitive academy research fellow position. She was a Nokia Visiting Professor, in 2016. She is currently a
Professor with the Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland, where she has
been a Senior Researcher since 2005 and an Associate Professor since 2014. She has authored or coauthored more than
170 papers in journals and conferences. She has authored/edited three books and seven special issues in journals. Her
research has been reported by Finnish TV programs, newspapers, and MIT Technology Review. Her current research
interests include image and video descriptors, facial-expression and microexpression recognition, gait analysis,
dynamictexture recognition, human motion analysis, and person identification. She is a Copublicity Chair for FG2018.
She was Area Chairs for several conferences. She was a Co-Chair of many international workshops at ECCV, ICCV, CVPR,
ACCV, and BMVC. She has lectured tutorials at ICPR 2006, ICCV 2009, and SCIA 2013. She is an Associate Editor for
Pattern Recognition, theIEEE Transactions on Circuits and Systems for Video Technology, and
Image and Vision Computing journals. Her papers have currently more than 7200 citations in Google
Scholar (h-index 36).View more
