Prodirect Manipulation: Bidirectional Programming for the Masses

Software interfaces today generally fall at either end of a spectrum. On one end are programmable systems, which allow expert users (i.e. programmers) to write software artifacts that describe complex abstractions, but programs are disconnected from their eventual output. On the other end are domain-specific graphical user interfaces (GUIs), which allow end users (i.e. non-programmers) to easily create varied content but present insurmountable walls when a desired feature is not built-in. Both programmatic and direct manipulation have distinct strengths, but users must typically choose one over the other or use some ad-hoc combination of systems. Our goal, put simply, is to bridge this divide. We envision novel software systems that tightly couple programmatic and direct manipulation - a combination we dub prodirect manipulation - for a variety of use cases. This will require advances in a broad range of software engineering disciplines, from program analysis and program synthesis technology to user interface design and evaluation. In this extended abstract, we propose two general strategies - real-time program synthesis and domain-specific synthesis of general-purpose programs - that may prove fruitful for overcoming the technical challenges. We also discuss metrics that will be important in evaluating the usability and utility of prodirect manipulation systems.