By Topic

Software Engineering, IEEE Transactions on

Issue 6 • Date June 2013

Filter Results

Displaying Results 1 - 10 of 10
  • In Memoriam - David Notkin (1953-2013)

    Page(s): 742 - 743
    Save to Project icon | Request Permissions | PDF file iconPDF (86 KB)  
    Freely Available from IEEE
  • A fluid model for layered queueing networks

    Page(s): 744 - 756
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (842 KB) |  | HTML iconHTML  

    Layered queueing networks are a useful tool for the performance modeling and prediction of software systems that exhibit complex characteristics such as multiple tiers of service, fork/join interactions, and asynchronous communication. These features generally result in nonproduct form behavior for which particularly efficient approximations based on mean value analysis (MVA) have been devised. This paper reconsiders the accuracy of such techniques by providing an interpretation of layered queueing networks as fluid models. Mediated by an automatic translation into a stochastic process algebra, PEPA, a network is associated with a set of ordinary differential equations (ODEs) whose size is insensitive to the population levels in the system under consideration. A substantial numerical assessment demonstrates that this approach significantly improves the quality of the approximation for typical performance indices such as utilization, throughput, and response time. Furthermore, backed by established theoretical results of asymptotic convergence, the error trend shows monotonic decrease with larger population sizes-a behavior which is found to be in sharp contrast with that of approximate mean value analysis, which instead tends to increase. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A large-scale empirical study of just-in-time quality assurance

    Page(s): 757 - 773
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2137 KB) |  | HTML iconHTML  

    Defect prediction models are a well-known technique for identifying defect-prone files or packages such that practitioners can allocate their quality assurance efforts (e.g., testing and code reviews). However, once the critical files or packages have been identified, developers still need to spend considerable time drilling down to the functions or even code snippets that should be reviewed or tested. This makes the approach too time consuming and impractical for large software systems. Instead, we consider defect prediction models that focus on identifying defect-prone (“risky”) software changes instead of files or packages. We refer to this type of quality assurance activity as “Just-In-Time Quality Assurance,” because developers can review and test these risky changes while they are still fresh in their minds (i.e., at check-in time). To build a change risk model, we use a wide range of factors based on the characteristics of a software change, such as the number of added lines, and developer experience. A large-scale study of six open source and five commercial projects from multiple domains shows that our models can predict whether or not a change will lead to a defect with an average accuracy of 68 percent and an average recall of 64 percent. Furthermore, when considering the effort needed to review changes, we find that using only 20 percent of the effort it would take to inspect all changes, we can identify 35 percent of all defect-inducing changes. Our findings indicate that “Just-In-Time Quality Assurance” may provide an effort-reducing way to focus on the most risky changes and thus reduce the costs of developing high-quality software. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Abstracting runtime heaps for program understanding

    Page(s): 774 - 786
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (722 KB) |  | HTML iconHTML  

    Modern programming environments provide extensive support for inspecting, analyzing, and testing programs based on the algorithmic structure of a program. Unfortunately, support for inspecting and understanding runtime data structures during execution is typically much more limited. This paper provides a general purpose technique for abstracting and summarizing entire runtime heaps. We describe the abstract heap model and the associated algorithms for transforming a concrete heap dump into the corresponding abstract model as well as algorithms for merging, comparing, and computing changes between abstract models. The abstract model is designed to emphasize high-level concepts about heap-based data structures, such as shape and size, as well as relationships between heap structures, such as sharing and connectivity. We demonstrate the utility and computational tractability of the abstract heap model by building a memory profiler. We use this tool to identify, pinpoint, and correct sources of memory bloat for programs from DaCapo. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software

    Page(s): 787 - 805
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2310 KB) |  | HTML iconHTML  

    Testing provides a primary means for assuring software in safety-critical systems. To demonstrate, particularly to a certification authority, that sufficient testing has been performed, it is necessary to achieve the test coverage levels recommended or mandated by safety standards and industry guidelines. Mutation testing provides an alternative or complementary method of measuring test sufficiency, but has not been widely adopted in the safety-critical industry. In this study, we provide an empirical evaluation of the application of mutation testing to airborne software systems which have already satisfied the coverage requirements for certification. Specifically, we apply mutation testing to safety-critical software developed using high-integrity subsets of C and Ada, identify the most effective mutant types, and analyze the root causes of failures in test cases. Our findings show how mutation testing could be effective where traditional structural coverage analysis and manual peer review have failed. They also show that several testing issues have origins beyond the test activity, and this suggests improvements to the requirements definition and coding process. Our study also examines the relationship between program characteristics and mutation survival and considers how program size can provide a means for targeting test areas most likely to have dormant faults. Industry feedback is also provided, particularly on how mutation testing can be integrated into a typical verification life cycle of airborne software. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Event Logs for the Analysis of Software Failures: A Rule-Based Approach

    Page(s): 806 - 821
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1957 KB) |  | HTML iconHTML  

    Event logs have been widely used over the last three decades to analyze the failure behavior of a variety of systems. Nevertheless, the implementation of the logging mechanism lacks a systematic approach and collected logs are often inaccurate at reporting software failures: This is a threat to the validity of log-based failure analysis. This paper analyzes the limitations of current logging mechanisms and proposes a rule-based approach to make logs effective to analyze software failures. The approach leverages artifacts produced at system design time and puts forth a set of rules to formalize the placement of the logging instructions within the source code. The validity of the approach, with respect to traditional logging mechanisms, is shown by means of around 12,500 software fault injection experiments into real-world systems. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Local versus Global Lessons for Defect Prediction and Effort Estimation

    Page(s): 822 - 834
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2335 KB) |  | HTML iconHTML  

    Existing research is unclear on how to generate lessons learned for defect prediction and effort estimation. Should we seek lessons that are global to multiple projects or just local to particular projects? This paper aims to comparatively evaluate local versus global lessons learned for effort estimation and defect prediction. We applied automated clustering tools to effort and defect datasets from the PROMISE repository. Rule learners generated lessons learned from all the data, from local projects, or just from each cluster. The results indicate that the lessons learned after combining small parts of different data sources (i.e., the clusters) were superior to either generalizations formed over all the data or local lessons formed from particular projects. We conclude that when researchers attempt to draw lessons from some historical data source, they should 1) ignore any existing local divisions into multiple sources, 2) cluster across all available data, then 3) restrict the learning of lessons to the clusters from other sources that are nearest to the test data. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis

    Page(s): 835 - 856
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4443 KB) |  | HTML iconHTML  

    This paper provides a systematic meta-analysis of 27 studies that investigate the impact of Test-Driven Development (TDD) on external code quality and productivity. The results indicate that, in general, TDD has a small positive effect on quality but little to no discernible effect on productivity. However, subgroup analysis has found both the quality improvement and the productivity drop to be much larger in industrial studies in comparison with academic studies. A larger drop of productivity was found in studies where the difference in test effort between the TDD and the control group's process was significant. A larger improvement in quality was also found in the academic studies when the difference in test effort is substantial; however, no conclusion could be derived regarding the industrial studies due to the lack of data. Finally, the influence of developer experience and task size as moderator variables was investigated, and a statistically significant positive correlation was found between task size and the magnitude of the improvement in quality. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Using Timed Automata for Modeling Distributed Systems with Clocks: Challenges and Solutions

    Page(s): 857 - 868
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (887 KB) |  | HTML iconHTML  

    The application of model checking for the formal verification of distributed embedded systems requires the adoption of techniques for realistically modeling the temporal behavior of such systems. This paper discusses how to model with timed automata the different types of relationships that may be found among the computer clocks of a distributed system, namely, ideal clocks, drifting clocks, and synchronized clocks. For each kind of relationship, a suitable modeling pattern is thoroughly described and formally verified. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • What Industry Needs from Architectural Languages: A Survey

    Page(s): 869 - 891
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (8093 KB) |  | HTML iconHTML  

    Many times we are faced with the proliferation of definitions, concepts, languages, and tools in certain (research) topics. But often there is a gap between what is provided by existing technologies and what is needed by their users. The strengths, limitations, and needs of the available technologies can be dubious. The same applies to software architectures, and specifically to languages designed to represent architectural models. Tens of different architectural languages have been introduced by the research and industrial communities in the last two decades. However, it is unclear if they fulfill the user's perceived needs in architectural description. As a way to plan for next generation languages for architectural description, this study analyzes practitioners' perceived strengths, limitations, and needs associated with existing languages for software architecture modeling in industry. We run a survey by interviewing 48 practitioners from 40 different IT companies in 15 countries. Each participant is asked to fill in a questionnaire of 51 questions. By analyzing the data collected through this study, we have concluded that 1) while practitioners are generally satisfied with the design capabilities provided by the languages they use, they are dissatisfied with the architectural language analysis features and their abilities to define extra-functional properties; 2) architectural languages used in practice mostly originate from industrial development instead of from academic research; 3) more formality and better usability are required of an architectural language. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

The IEEE Transactions on Software Engineering is interested in well-defined theoretical results and empirical studies that have potential impact on the construction, analysis, or management of software. The scope of this Transactions ranges from the mechanisms through the development of principles to the application of those principles to specific environments. Specific topic areas include: a) development and maintenance methods and models, e.g., techniques and principles for the specification, design, and implementation of software systems, including notations and process models; b) assessment methods, e.g., software tests and validation, reliability models, test and diagnosis procedures, software redundancy and design for error control, and the measurements and evaluation of various aspects of the process and product; c) software project management, e.g., productivity factors, cost models, schedule and organizational issues, standards; d) tools and environments, e.g., specific tools, integrated tool environments including the associated architectures, databases, and parallel and distributed processing issues; e) system issues, e.g., hardware-software trade-off; and f) state-of-the-art surveys that provide a synthesis and comprehensive review of the historical development of one particular area of interest.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Matthew B. Dwyer
Dept. Computer Science and Engineering
256 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0115 USA
tseeicdwyer@computer.org