Estimating the efficiency of collaborative problem-solving, with applications to chip design

We present a statistical framework to address questions that arise in general problems involving collaboration of several contributors. One instance of this problem occurs in the complex process of designing ultralarge-scale-integration (ULSI) semiconductor chips. In these processes, computer-aided design tools are treated as “black boxes.” In most cases, the automated design tools operate on designs and successfully complete a specified task to create designs that satisfy specified design criteria. In other cases involving complex designs, however, the tools are unable to create designs that satisfy the specified criteria. In both situations, the performance of the tools can be enhanced with systematic external intervention that is implemented with some supplemental algorithm. This algorithm can either be fully automated or be implemented by hand, relying on formally describable human expertise. In this intervention, the supplemental algorithm and the automated design tool take turns to move the design from one configuration to another until either the task is complete or further improvements are not possible or necessary. In such a setting, a number of questions arise about how to measure the effectiveness of the external intervention. One question, for example, is whether the external intervention consistently assists the progress of the automated program. This situation is an instance of a general problem that we address in this paper. As an example, we apply the statistical framework to the problem of routing a functional unit of the IBM POWER4 microprocessor.