Author Archives: gordon

EMSE: A detailed investigation of the effectiveness of whole test suite generation

  • [PDF] [DOI] J. M. Rojas, M. Vivanti, A. Arcuri, and G. Fraser, “A detailed investigation of the effectiveness of whole test suite generation,” Empirical Software Engineering, pp. 1-42, 2016.
    [Bibtex]
    @Article{emse16_effectiveness,
    author="Rojas, Jos{\'e} Miguel
    and Vivanti, Mattia
    and Arcuri, Andrea
    and Fraser, Gordon",
    title="A detailed investigation of the effectiveness of whole test suite generation",
    journal="Empirical Software Engineering",
    year="2016",
    pages="1--42",
    abstract="A common application of search-based software testing is to generate test cases for all goals defined by a coverage criterion (e.g., lines, branches, mutants). Rather than generating one test case at a time for each of these goals individually, whole test suite generation optimizes entire test suites towards satisfying all goals at the same time. There is evidence that the overall coverage achieved with this approach is superior to that of targeting individual coverage goals. Nevertheless, there remains some uncertainty on (a) whether the results generalize beyond branch coverage, (b) whether the whole test suite approach might be inferior to a more focused search for some particular coverage goals, and (c) whether generating whole test suites could be optimized by only targeting coverage goals not already covered. In this paper, we perform an in-depth analysis to study these questions. An empirical study on 100 Java classes using three different coverage criteria reveals that indeed there are some testing goals that are only covered by the traditional approach, although their number is only very small in comparison with those which are exclusively covered by the whole test suite approach. We find that keeping an archive of already covered goals along with the tests covering them and focusing the search on uncovered goals overcomes this small drawback on larger classes, leading to an improved overall effectiveness of whole test suite generation.",
    issn="1573-7616",
    doi="10.1007/s10664-015-9424-2",
    url="http://dx.doi.org/10.1007/s10664-015-9424-2"
    }

New 1.0.3 release

A new version 1.0.3 of EvoSuite has now been released. This is the version used at this year’s SBST tool competition.

Besides a bunch of bug fixes and performance improvements, we have also included test naming, where tests are not named with numbers (test1, test2, etc.) but based on what they cover (testFoo, testBarReturnsTrue, etc.) If you want to try this feature, run EvoSuite with the option -Dtest_naming_strategy=coverage.

Release data is available 1.0.3 release results.

ISSTA 2015: Automated Unit Test Generation during Software Development: A Controlled Experiment and Think-Aloud Observations

  • [PDF] J. M. Rojas, G. Fraser, and A. Arcuri, “Automated Unit Test Generation during Software Development: A Controlled Experiment and Think-Aloud Observations,” in Proceedings of the 2015 International Symposium on Software Testing and Analysis, 2015, pp. 338-349.
    [Bibtex]
    @inproceedings{ISSTA15_Study,
     author = {Jos{\'e} Miguel Rojas and Gordon Fraser and Andrea Arcuri},
     title = {Automated Unit Test Generation during Software Development: A Controlled Experiment and Think-Aloud Observations},
     booktitle = {Proceedings of the 2015 International Symposium on Software Testing and Analysis},
     series = {ISSTA '15},
     year = {2015},
     publisher = {ACM},
     pages={338--349},
    }

EvoSuite at the SBST 2015 competition

  • [PDF] G. Fraser and A. Arcuri, “EvoSuite at the SBST 2015 Tool Competition,” in 8th International Workshop on Search-Based Software Testing (SBST’15) at ICSE’15, 2015.
    [Bibtex]
    @inproceedings{SBST15_competition,
      author    = {Gordon Fraser and Andrea Arcuri},
      title     = {EvoSuite at the SBST 2015 Tool Competition},
      booktitle = {8th International Workshop on Search-Based Software Testing (SBST'15) at ICSE'15},
      year      = {2015},
      note = {To appear}
    }

TOSEM: Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study

  • [PDF] G. Fraser, M. Staats, P. McMinn, A. Arcuri, and F. Padberg, “Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 24, iss. 4, p. 23, 2015.
    [Bibtex]
    @article{TOSEM_userstudy,
    author = {Gordon Fraser and Matt Staats and Phil McMinn and Andrea Arcuri and Frank Padberg},
    title = {Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study},
    journal ={ACM Transactions on Software Engineering and Methodology (TOSEM)},
      volume={24},
      number={4},
      pages={23},
      year={2015},
      publisher={ACM}
    }

TOSEM: A Large Scale Evaluation of Automated Unit Test Generation Using EvoSuite

  • [PDF] G. Fraser and A. Arcuri, “A Large Scale Evaluation of Automated Unit Test Generation Using EvoSuite,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 24, iss. 2, p. 8, 2014.
    [Bibtex]
    @article{TOSEM_evaluation,
    author = {Gordon Fraser and Andrea Arcuri},
    title = {A Large Scale Evaluation of Automated Unit Test Generation Using EvoSuite},
    journal ={ACM Transactions on Software Engineering and Methodology (TOSEM)},
    year={2014},
    volume={24},
      number={2},
      pages={8},
      year={2014},
      publisher={ACM}
    }

ASE 2014: Automated Unit Test Generation for Classes with Environment Dependencies

  • [PDF] A. Arcuri, G. Fraser, and J. P. Galeotti, “Automated Unit Test Generation for Classes with Environment Dependencies,” in IEEE/ACM Int. Conference on Automated Software Engineering (ASE), New York, NY, USA, 2014.
    [Bibtex]
    @inproceedings{ase14_environment,
        AUTHOR="Andrea Arcuri and Gordon Fraser and Juan Pablo Galeotti",
        TITLE="Automated Unit Test Generation for Classes with Environment Dependencies",
        booktitle={IEEE/ACM Int. Conference on Automated Software Engineering (ASE)},
        pages="",
        year = {2014},
        pages = {79--90},
        numpages = {12},
        publisher = {ACM},
        address = {New York, NY, USA},
    }

ASE 2014: Continuous Test Generation: Enhancing Continuous Integration with Automated Test Generation

  • [PDF] J. Campos, A. Arcuri, G. Fraser, and R. Abreu, “Continuous Test Generation: Enhancing Continuous Integration with Automated Test Generation,” in IEEE/ACM Int. Conference on Automated Software Engineering (ASE), New York, NY, USA, 2014, pp. 55-66.
    [Bibtex]
    @inproceedings{ase14_ctg,
        AUTHOR="Jose Campos and Andrea Arcuri and Gordon Fraser and Rui Abreu",
        TITLE="Continuous Test Generation: Enhancing Continuous Integration with Automated Test Generation",
        booktitle={IEEE/ACM Int. Conference on Automated Software Engineering (ASE)},
        year = {2014},
        location = {Vasteras, Sweden},
        pages = {55--66},
        numpages = {12},
        publisher = {ACM},
        address = {New York, NY, USA},
    }