A framework for analyzing levels of analysis issues in studies of e-collaboration

There has been a proliferation of competing explanations regarding the inconsistent results reported by the e-collaboration literature since its inception. This study advances another possible explanation by investigating the range of multilevel issues that can be encountered in research on the use of synchronous or asynchronous group support systems. We introduce concepts of levels of analysis from the management literature and then examine all empirical studies of e-collaboration from seven information systems journals for the period 1999-2003. We identified a total of 54 studies of e-collaboration in these journals, and after excluding 18 nonconforming studies - those that were primarily conceptual, qualitative, or exploratory only-we analyzed the levels of analysis issues in the remaining 36 empirical studies. Based on our analysis and classification of these studies into six different clusters according to their levels of analysis, we found that a majority of these studies contain one or more problems of levels incongruence that cast doubts on the validity of their results. It is indeed possible that these methodological problems are in part responsible for the inconsistent results reported in this literature, especially since researchers' frequent decisions to analyze data at the individual level - even when the theory was formulated at the group level and when the research setting featured individuals working in groups -may very well have artificially inflated the authors' chances of finding statistically significant results. Based on our discussion of levels of analysis concepts, we hope to provide guidance to empirical researchers who study e-ollaboration.