Conceptual Attention in StyleGAN

Attention is an important function in human cognition. Since the human brain can only process a limited amount of information at a time, attention selects and focuses on just the important information among all the perceived information. Recent years have seen the advent of methods that imitate the human attention mechanism by means of deep learning, one of which is the multi-head attention proposed for Transformer. Humans can consider a single object from multiple perspectives, so multi-head attention linearly maps the input to multiple latent spaces and computes the attention for each of them. However, when humans pay attention to a single object from various aspects, there is an explicit context in the brain (e.g., a concept), which they pay attention to and accordingly perceive in a top-down manner—and multi-head attention cannot do this. We have therefore developed Conceptual Attention (CA), which can more explicitly compute the attention for each concept. The aim of CA is to achieve the same concept-by-concept attentional mechanism as the human brain by using style vectors learned by StyleGAN as concepts. Our experiments demonstrate that introducing CA into the StyleGAN2 discriminator and then fine-tuning it improves the quality of the generated images for large, small, and noisy data. We also visualize the results of attention to show that the proposed method can capture the concepts and qualitatively clarify how the concepts are reflected in the generated images.