By Topic

Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION, 2007. TAICPART-MUTATION 2007

Date 10-14 Sept. 2007

Filter Results

Displaying Results 1 - 25 of 44
  • Testing: Academic and Industrial Conference Practice and Research Techniques - Cover

    Page(s): c1
    Save to Project icon | Request Permissions | PDF file iconPDF (117 KB)  
    Freely Available from IEEE
  • Testing: Academic and Industrial Conference Practice and Research Techniques - Title page

    Page(s): i - iii
    Save to Project icon | Request Permissions | PDF file iconPDF (60 KB)  
    Freely Available from IEEE
  • Testing: Academic and Industrial Conference Practice and Research Techniques - Table of contents

    Page(s): v - vii
    Save to Project icon | Request Permissions | PDF file iconPDF (49 KB)  
    Freely Available from IEEE
  • Preface

    Page(s): ix
    Save to Project icon | Request Permissions | PDF file iconPDF (61 KB)  
    Freely Available from IEEE
  • Organization

    Page(s): xiii
    Save to Project icon | Request Permissions | PDF file iconPDF (24 KB)  
    Freely Available from IEEE
  • Program Committee

    Page(s): xiv
    Save to Project icon | Request Permissions | PDF file iconPDF (28 KB)  
    Freely Available from IEEE
  • Keynote speakers

    Page(s): xv
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (117 KB)  

    Provides an abstract for each of the keynote presentations and a brief professional biography of each presenter. The complete presentations were not made available for publication as part of the conference proceedings. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Kiasan/KUnit: Automatic Test Case Generation and Analysis Feedback for Open Object-oriented Systems

    Page(s): 3 - 12
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (441 KB) |  | HTML iconHTML  

    We demonstrate how a static analysis feedback and unit test case generation framework, KUnit, built on the Bogor/Kiasan symbolic execution engine provides an effective unit test case generation for sequential heap-intensive Java programs (whose computation structures are incomplete - open systems). KUnit leverages method contract information to better deal with open object-oriented systems and to support automatic mock object creation. To facilitate application to realistic software, KUnit allows the scope/cost of the analysis and test case generation to be controlled via notions of heap configuration coverage. In a broad experimental study on 23 Java data structure modules, we show that it is able to: (a) achieve 100% feasible branch coverage on almost all methods by using only small heap configurations; (b) improve on competing tools for coverage achieved; size of test suites; and time to generate test suites. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Abstracting Symbolic Execution with String Analysis

    Page(s): 13 - 22
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (297 KB) |  | HTML iconHTML  

    Forward symbolic execution is a technique for program analysis that explores the execution paths of a program by maintaining a symbolic representation of the program state. Traditionally, applications of this technique have focused on symbolically representing only primitive data types, while more recent extensions have expanded to reference types. We demonstrate the ability to symbolically execute a commonly used library class, specifically a string, at an abstract level. By abstracting away the implementation details of strings using finite-state automata, symbolic execution can scale to more complex programs. This technique can be applied to programs which generate complicated strings, such as SQL database queries. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Improving Evolutionary Testing in the Presence of Function-Assigned Flags

    Page(s): 23 - 34
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (508 KB) |  | HTML iconHTML  

    Evolutionary structural testing, an approach to automatically generating relevant unit test cases, encounters difficulties when the tested software contains boolean variables. This issue, known as the flag problem, has been studied by many researchers. However, previous work does not address the issue of function-assigned flags which constitutes a special type of the flag problem that often occurs in the context of object-orientation. This paper elaborates on a new approach to the flag problem that can also handle function-assigned flags while being applicable to the conventional flag problem, as well. It relies on a code transformation that leads to an improved fitness landscape which provides better guidance to the evolutionary search. We present four case studies including a fitness landscape analysis and empirical results. The results show that the suggested code transformation improves evolutionary structural testing in the presence of function-assigned flags. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Testing Techniques - Are they of any Practical Use?

    Page(s): 35 - 39
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (525 KB) |  | HTML iconHTML  

    Testing techniques are taught in universities and on many industrial training courses yet there is evidence to suggest that, in the corporate world, testers often find it difficult to apply the techniques and instead rely on domain knowledge and experience when designing tests. This is a shame as testing techniques are very effective at reducing the cost of testing by providing better coverage with less effort. When applying techniques it is important to be pragmatic and understand where they can add value to your test process. A business focused approach, based on risk, is essential. Commercial strength tools must be considered to support the test process although home grown solutions should not be ruled out in some situations. An appropriate investment in effective testing will yield dividends in the longer term and so further research is required to understand why testing techniques are not much more widely adopted throughout industry. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Test Methods and Tools for ERP Implementations

    Page(s): 40 - 46
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (302 KB) |  | HTML iconHTML  

    Enterprise resource planning (ERP) systems represent a huge market in the commercial arena. Products from suppliers such as SAP, Oracle and more recently, Microsoft, dominate the market. Testing in these projects is a significant effort but is hardly supported by methods and tools other than those provided by the suppliers themselves. Experience shows that testing in these projects is critical, but often neglected. Recent 'lessons learned' work by the author indicates that a benefits-, risk- and coverage-based test approach could significantly reduce the risk of failures. Some foundation/visionary work to define ERP-specific test methods and tools has already been performed by the author. However, the approach needs more research, rigor and proving in a commercial environment. Academic and commercial partners are sought to refine, develop and prove these methods and tools. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Early UML Model Testing using TTCN-3 and the UML Testing Profile

    Page(s): 47 - 54
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (455 KB) |  | HTML iconHTML  

    Motorola has been driving the use of model-driven engineering for a number of years with good successes. Now with its drive to deploy the Unified Modelling Language (UML) version 2 for more effective development, Motorola has developed capabilities to enable the early testing of UML design models. To ensure commonality the standard testing specification and execution frameworks TTCN-3 and the UML testing profile were chosen. In doing so, a number of tools have been developed, extending both commercial and internal tools to provide an effective test capability for the early functional testing of UML design models. The same toolset can be used to test the same design on a target system after code generation. The toolset is being used by several different product groups within Motorola, and the paper reports some experience and findings, including areas where TTCN-3 can be extended. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • State-based Testing is Functional Testing

    Page(s): 55 - 66
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (353 KB) |  | HTML iconHTML  

    Empirical studies report unsatisfactory fault detection of state-based methods in class testing and advocate the use of functional methods to complement state-based testing. In this paper, we take the view that the modest fault detection of state-based class testing reported in the literature is actually due to the inappropriate state diagram used. We show that functional testing of a class can be reduced to state-based testing, provided that the right state model is produced. We present a strategy for constructing a state diagram for a class method, based on a domain partition derived through functional techniques. We also describe a method for deriving test sequences from the resulting state diagrams, essentially a variant of the W-method. The paper also reports results from an experimental evaluation of the proposed approach, based on mutants generated by Mu- Java. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Test Effort Estimation Models Based on Test Specifications

    Page(s): 67 - 71
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (343 KB) |  | HTML iconHTML  

    In competitive markets, test teams should be able to estimate the required effort to perform their test activities on the schedule. In this paper, we present a test execution and a test automation effort estimation model and their use for test selection. All these models are based on the test specifications written in a controlled natural language. This characteristic may allow us to have better estimation accuracy, even for tests never executed before. The evaluation of these models and some additional benefits to industry are also discussed in the paper. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On Test Data Generation of Object-Oriented Software

    Page(s): 72 - 76
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (224 KB) |  | HTML iconHTML  

    Nowadays, object-oriented (OO) languages are widely used in the development of many different kinds of applications. However, testing those applications is still very expensive and time-consuming for the software community. The automation of this task would therefore be highly desirable. Although automatic testing of procedural software has been studied in depth for many years, comparatively little work has been done about OO software. Different techniques exist. However, the most promising one is probably to model the task as a search problem. This paper explains why automatic testing of OO software is more difficult than procedural software. These difficulties provide strong challenges to the natural computation and software engineering communities. A brief review of the literature of the subject follows. The issues of the current state-of-art of the field are then outlined. Finally, some open research problems are discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Automatic Test-Data Generation: An Immunological Approach

    Page(s): 77 - 81
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (343 KB) |  | HTML iconHTML  

    In previous research, we presented an approach to automatically generate test-data for object-oriented software exploiting a genetic algorithm (GA) to achieve high levels of data-flow coverage. The experimental results from testing six Java classes helped us identify a number of problematic test targets, and suggest that in the future full data-flow coverage with a reasonable computational cost may be possible if we overcome these obstacles. To this end, the investigation of artificial immune system (AIS) algorithms was chosen. This paper provides a brief summary of our previous work and an introduction to both human and artificial immune system. We then suggest a framework for the application of AIS algorithms to the problem of automated testing, followed by some thoughts on why and how these algorithms can be beneficial in our effort to improve the performance of our previously implemented GA. Finally, our preliminary results from a proof-of-concept implementation are presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • How good are your testers? An assessment of testing ability

    Page(s): 82 - 88
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (310 KB) |  | HTML iconHTML  

    During our previous research conducted in the Sheffield Software Engineering Observatory [11], we found that test first programmers spent a higher percentage of their time testing than those testing after coding. However as the team allocation was based on subjects ' academic records and their preference, it was unclear if they were simply better testers. Thus this paper proposes two questionnaires to assess the testing ability of subjects, in order to reveal the factors that contribute to the previous findings. Preliminary results show that the testing ability of subjects, as measured by the survey, varies based on their professional skill level. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the Accuracy of Spectrum-based Fault Localization

    Page(s): 89 - 98
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (422 KB) |  | HTML iconHTML  

    Spectrum-based fault localization shortens the test- diagnose-repair cycle by reducing the debugging effort. As a light-weight automated diagnosis technique it can easily be integrated with existing testing schemes. However, as no model of the system is taken into account, its diagnostic accuracy is inherently limited. Using the Siemens Set benchmark, we investigate this diagnostic accuracy as a function of several parameters (such as quality and quantity of the program spectra collected during the execution of the system), some of which directly relate to test design. Our results indicate that the superior performance of a particular similarity coefficient, used to analyze the program spectra, is largely independent of test design. Furthermore, near- optimal diagnostic accuracy (exonerating about 80% of the blocks of code on average) is already obtained for low-quality error observations and limited numbers of test cases. The influence of the number of test cases is of primary importance for continuous (embedded) processing applications, where only limited observation horizons can be maintained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Software Fault Prediction using Language Processing

    Page(s): 99 - 110
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (439 KB) |  | HTML iconHTML  

    Accurate prediction of faulty modules reduces the cost of software development and evolution. Two case studies with a language-processing based fault prediction measure are presented. The measure, refereed to as a QALP score, makes use of techniques from information retrieval to judge software quality. The QALP score has been shown to correlate with human judgements of software quality. The two case studies consider the measure's application to fault prediction using two programs (one open source, one proprietary). Linear mixed-effects regression models are used to identify relationships between defects and QALP score. Results, while complex, show that little correlation exists in the first case study, while statistically significant correlations exists in the second. In this second study the QALP score is helpful in predicting faults in modules (files) with its usefulness growing as module size increases. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Modelling Dynamic Memory Management in Constraint-Based Testing

    Page(s): 111 - 120
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (683 KB) |  | HTML iconHTML  

    Constraint-Based Testing (CBT) is the process of generating test cases against a testing objective by using constraint solving techniques. When programs contain dynamic memory allocation and loops, constraint reasoning becomes challenging as new variables and new constraints are created during the test data generation process. In this paper, we address this problem by proposing a new constraint model of C programs based on operators that model dynamic memory management. These operators apply deduction rules on abstract states of the memory allowing so to enhance the constraint reasoning process that permits to generate test data for these programs. We illustrate our approach on structural testing of a complex program that contains dynamic memory allocation/deallocation, structures and loops. An implementation is in progress and first experimental results obtained on this program show the highly deductive potential of the approach. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Exploiting Constraint Solving History to Construct Interaction Test Suites

    Page(s): 121 - 132
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (416 KB) |  | HTML iconHTML  

    Researchers have explored the application of combinatorial interaction testing (CIT) methods to construct samples to drive systematic testing of software system configurations. Applying CIT to highly-configurable software systems is complicated by the fact that, in many such systems, there are constraints between specific configuration parameters that render certain combinations invalid. In recent work, automated constraint solving methods have been combined with search-based CIT methods to address this problem with promising results. In this paper, we observe that the pattern of computation in greedy CIT algorithms leads to sequences of constraint solving problems that are closely related to one another. We propose two techniques for exploiting the history of constraint solving: (1) using incremental algorithms that are present within available constraint solvers and (2) mining constraint solver data structures to extract information that can be used to reduce the CIT search space. We evaluate the cost-effectiveness of these reductions on four real-world highly-configurable software systems and on a population of synthetic examples that share the characteristics of those four systems. In combination our techniques reduce the cost of CIT in the presence of constraints to that of traditional unconstrained CIT methods without sacrificing the quality of solutions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Opportunities in System testing

    Page(s): 133
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (255 KB) |  | HTML iconHTML  

    Performing an effective system test requires an in-depth knowledge of the domain and system being tested and an effective test design method, coupled with the right test infrastructure. The current state-of-the-art does not respect the constraint that subject matter experts (SMEs) are in short supply, nor exploit the fact that similar systems have been built and tested in the past. Most current state-of-the-art methods do not leverage the various models and specifications and other design or implementation artifacts to simplify the test process. While there is a lot of focus on generating test inputs from specifications, inadequate attention has been paid to verifying test results. This paper tries to highlight some of the problems and research opportunities in this area. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Tabular Expression-Based Testing Strategies: A Comparison

    Page(s): 134
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (302 KB) |  | HTML iconHTML  

    Tabular expressions were proposed as a documentation tool that can be used to document software precisely and unambiguously. This paper explores the applications of four testing strategies in tabular expression-based specifications and further compares the strategies on a mathematical basis. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Case for GUI Testing Using Symbolic Execution Poster Abstract

    Page(s): 135
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (256 KB) |  | HTML iconHTML  

    A Graphical User Interface (GUI) consists of virtual objects (widgets) that are more intuitive to use, for example buttons, edit boxes, etc. While GUIs have become ubiquitous, testing them remains largely ad- hoc. In contrast with console applications where there is only one point of interaction (the command line), GUIs provide multiple points each of which might have different states. This structure makes GUI testing especially challenging because of its large input space. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.