Agile and other trends in software engineering

Successfully developing and delivering multi-year, multi-person software projects remains a highly challenging task. Software engineering researchers have spent considerable energy investigating ways to improve this situation by developing various processes, techniques, and tools over the last five decades. Understanding trends in the current state of the practice is crucial to identifying the challenges that software engineers face today, the changes their organizations are tackling, and how these challenges and changes impact industrial software production. This paper reports survey results from 99 software engineering developers and managers regarding their choice of process, technique, and tools, as well as their impressions as to the contributing factors towards project success or failure. In particular, the paper includes a focus on trends in adoption of agile practices. The data reinforces some known challenge areas such as the need for more effective communication. It also shows that there is a clear trend towards agile adoption across organizations, albeit traditional processes are still firmly entrenched in some areas. Scrum is the most common agile process in use, but it often gets adapted differently by different organizations, each according to their need. On the other hand, extreme programming was found to be almost non-existent in practice. The data also reveals that inadequate or unclear requirements are still the major problem that dogs the software industry. Similarly, a lack of rigorous time and cost estimation is commonplace. Together these two alone may explain the staggeringly high rate of software failures and delays. Finally, tool usage in industry varies quite widely; but there are certain areas (such as version control and unit testing) witnessing almost universal tool adoption.