P2A: A Tool for Converting Pixels to Animated Mobile Application User Interfaces

Developing mobile applications is typically a labor-intensive process in which software engineers manually re-implement in code screen designs, inter-screen transitions, and in-screen animations developed by user interface and user experience experts. Other engineering domains have used computer vision techniques to automate human perception and manual data entry tasks. The P2A tool adopts computer vision techniques for developing animated mobile applications. P2A infers from mobile application screen designs the user interface portion of an application's source code and other assets that are ready to be compiled and executed on a mobile phone. Among others, inferred mobile applications contain inter-screen transitions and in-screen animations. In our experiments on screenshots of 30 highly-ranked third-party Android applications, the P2A-generated application user interfaces exhibited high pixel-to-pixel similarity with their input screenshots. P2A took an average of 26 seconds to infer in-screen animations.