A UI Layout Generation Method Based on Shared Feature Extraction

The generative model that can produce different UI layouts based on text input is a valuable tool for assisting UI design. It can significantly reduce the time and cost of manually designing UI interfaces and is also an important task in cross-modal generation, such as text-to-image conversion. We first use an object detection network to capture layout information in the interface and generate constrained text from images. At the same time, we share the features learned in the intermediate process with the interface generation network to enhance the consistency between constrained text and the final interface. Then, we use a two-stage Stack GAN to convert constrained text into UI layouts. The encoder in the GAN network directly uses the feature extraction layer from the object detection network, saving training time and sharing feature extraction from images. Our proposed shared feature extraction method, compared to a network that does not share features, can improve content consistency by approximately 14% without changing network complexity and depth. Finally, on the Rico dataset, we achieved an average IS of 4.36, an average FID of 28.26, and an average R precision of 89.82%.