Author Archives: gordon

New 1.2.0 release

A new version 1.2.0 of EvoSuite has now been released. The main changes are:

  • Java 9 support for DSE (@ilebrero)
  • Updated Docker configuration (@mitchellolsthoorn)
  • Support for JUnit 5 output (@VoglSebastian)
  • Refactoring of the generics-related code (@VoglSebastian)
  • Java 9 support for Eclipse plugin
  • General refactorings and reformatting
  • Bug fixes for various crashes and minor issues

Release data is available 1.2.0 release results.

As usual, the source code release can be found on GitHub at https://github.com/EvoSuite/evosuite/releases/tag/v1.2.0.

Recent work on improving the fitness landscape

The fitness landscape faced by EvoSuite is challenging. Here are a couple of recent investigations to address this problem:

  • [PDF] S. Vogl, S. Schweikl, and G. Fraser, “Encoding the certainty of boolean variables to improve the guidance for search-based test generation,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2021, pp. 1088-1096.
    [Bibtex]
    @inproceedings{GECCO21_Fitness,
      title={Encoding the certainty of boolean variables to improve the guidance for search-based test generation},
      author={Vogl, Sebastian and Schweikl, Sebastian and Fraser, Gordon},
      booktitle={Proceedings of the Genetic and Evolutionary Computation Conference},
      pages={1088--1096},
      year={2021}
    }
  • [PDF] [DOI] Y. Lin, Y. S. Ong, J. Sun, G. Fraser, and J. S. Dong, “Graph-Based Seed Object Synthesis for Search-Based Unit Testing,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, New York, NY, USA, 2021, p. 1068–1080.
    [Bibtex]
    @inproceedings{FSE21_ObjectSeed,
    author = {Lin, Yun and Ong, You Sheng and Sun, Jun and Fraser, Gordon and Dong, Jin Song},
    title = {Graph-Based Seed Object Synthesis for Search-Based Unit Testing},
    year = {2021},
    isbn = {9781450385626},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3468264.3468619},
    doi = {10.1145/3468264.3468619},
    abstract = {Search-based software testing (SBST) generates tests using search algorithms guided
    by measurements gauging how far a test case is away from exercising a coverage goal.
    The effectiveness of SBST largely depends on the continuity and monotonicity of the
    fitness landscape decided by these measurements and the search operators. Unfortunately,
    the fitness landscape is challenging when the function under test takes object inputs,
    as classical measurement hardly provide guidance for constructing legitimate object
    inputs. To overcome this problem, we propose test seeds, i.e., test code skeletons
    of legitimate objects which enable the use of classical measurements. Given a target
    branch in a function under test, we first statically analyze the function to build
    an object construction graph that captures the relation between the operands of the
    target method and the states of their relevant object inputs. Based on the graph,
    we synthesize test template code where each "slot" is a mutation point for the search
    algorithm. This approach can be seamlessly integrated with existing SBST algorithms,
    and we implemented EvoObj on top of EvoSuite. Our experiments show that EvoObj outperforms
    EvoSuite with statistical significance on 2750 methods over 103 open source Java projects
    using state-of-the-art SBST algorithms.},
    booktitle = {Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
    pages = {1068–1080},
    numpages = {13},
    keywords = {object oriented, code synthesis, software testing, search-based},
    location = {Athens, Greece},
    series = {ESEC/FSE 2021}
    }
  • [PDF] Y. Lin, J. Sun, G. Fraser, Z. Xiu, T. Liu, and J. S. Dong, “Recovering fitness gradients for interprocedural Boolean flags in search-based testing,” in Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2020, pp. 440-451.
    [Bibtex]
    @inproceedings{ISSTA20_Gradient,
      title={Recovering fitness gradients for interprocedural Boolean flags in search-based testing},
      author={Lin, Yun and Sun, Jun and Fraser, Gordon and Xiu, Ziheng and Liu, Ting and Dong, Jin Song},
      booktitle={Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis},
      pages={440--451},
      year={2020}
    }

Recent SBST tool competition results

EvoSuite participated in the various instances of the SBST tool competition (https://sbst21.github.io/tools/) over the recent years, here is a list of all the results that have been missing on the webpage so far. In all these cases, EvoSuite achieved the highest overall score in the competition:

  • [PDF] S. Vogl, S. Schweikl, G. Fraser, A. Arcuri, J. Campos, and A. Panichella, “EVOSUITE at the SBST 2021 Tool Competition,” in 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST), 2021, pp. 28-29.
    [Bibtex]
    @inproceedings{SBST21_competition,
      title={EVOSUITE at the SBST 2021 Tool Competition},
      author={Vogl, Sebastian and Schweikl, Sebastian and Fraser, Gordon and Arcuri, Andrea and Campos, Jose and Panichella, Annibale},
      booktitle={2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST)},
      pages={28--29},
      year={2021},
      organization={IEEE}
    }
  • [PDF] A. Panichella, J. Campos, and G. Fraser, “EvoSuite at the SBST 2020 Tool Competition,” in Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, 2020, pp. 549-552.
    [Bibtex]
    @inproceedings{SBST20_competition,
      title={EvoSuite at the SBST 2020 Tool Competition},
      author={Panichella, Annibale and Campos, Jos{\'e} and Fraser, Gordon},
      booktitle={Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops},
      pages={549--552},
      year={2020}
    }
  • [PDF] J. Campos, A. Panichella, and G. Fraser, “EvoSuiTE at the SBST 2019 tool competition,” in 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST), 2019, pp. 29-32.
    [Bibtex]
    @inproceedings{SBST19_competition,
      title={EvoSuiTE at the SBST 2019 tool competition},
      author={Campos, Jos{\'e} and Panichella, Annibale and Fraser, Gordon},
      booktitle={2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST)},
      pages={29--32},
      year={2019},
      organization={IEEE}
    }
  • [PDF] [DOI] G. Fraser, J. M. Rojas, and A. Arcuri, “EvoSuite at the SBST 2018 Tool Competition,” in Proceedings of the 11th International Workshop on Search-Based Software Testing, New York, NY, USA, 2018, p. 34–37.
    [Bibtex]
    @inproceedings{SBST18_competition,
    author = {Fraser, Gordon and Rojas, Jos\'{e} Miguel and Arcuri, Andrea},
    title = {EvoSuite at the SBST 2018 Tool Competition},
    year = {2018},
    isbn = {9781450357418},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3194718.3194729},
    doi = {10.1145/3194718.3194729},
    abstract = {EvoSuite is a search-based tool that automatically generates executable unit tests
    for Java code (JUnit tests). This paper summarises the results and experiences of
    EvoSuite's participation at the sixth unit testing competition at SBST 2018, where
    EvoSuite achieved the highest overall score (687 points) for the fifth time in six
    editions of the competition.},
    booktitle = {Proceedings of the 11th International Workshop on Search-Based Software Testing},
    pages = {34–37},
    numpages = {4},
    location = {Gothenburg, Sweden},
    series = {SBST '18}
    }

New 1.1.0 release

A new version 1.1.0 of EvoSuite has now been released. The main changes are:

  • This release contains a major refactoring to improve the internal code quality, and specifically the handling of generics in the code base.
  • This release also adds Java 9+ support (with some caveats: DSE currently only works up to Java 8, and JEE support has been removed for now).
  • The default algorithm is now DynaMOSA instead of whole test suite generation.
  • The release also includes loads of bug fixes and improvements.

Release data is available 1.1.0 release results.

As usual, the source code release can be found on GitHub at https://github.com/EvoSuite/evosuite/releases/tag/v1.1.0.

New 1.0.6 release

A new version 1.0.6 of EvoSuite has now been released, and again contains a bunch of bug fixes. Many changes relate to the mocking infrastructure/Mockito integration as well as general crashes, and the search algorithms have seen some refactoring. Release data is available 1.0.6 release results.

This version has also been tested with different coverage analysis tools. For details, visit Measuring Code Coverage and the corresponding example projects.

EvoSuite wins the SBST 2017 tool competition

EvoSuite achieved the overall highest score of all competing tools at the SBST 2017 Unit Testing Tool Competition. For more details read the following paper:

  • [PDF] G. Fraser and A. Arcuri, “EvoSuite at the SBST 2017 Tool Competition,” in 10th International Workshop on Search-Based Software Testing (SBST’17) at ICSE’17, 2017, pp. 39-42.
    [Bibtex]
    @inproceedings{SBST17_competition,
      author    = {Gordon Fraser and Andrea Arcuri},
      title     = {EvoSuite at the SBST 2017 Tool Competition},
      booktitle = {10th International Workshop on Search-Based Software Testing (SBST'17) at ICSE'17},
      year      = {2017},
      pages     = {39--42},
    }

New 1.0.5 release

A new version 1.0.5 of EvoSuite has now been released, and contains a bunch of bug fixes and performance improvements. Release data is available 1.0.5 release results: Average branch coverage is up from 74% to 76%; the number of crashes should be substantially lower now.

ICST 2017: Private API Access and Functional Mocking in Automated Unit Test Generation

  • [PDF] A. Arcuri, G. Fraser, and R. Just, “Private API Access and Functional Mocking in Automated Unit Test Generation,” in ICST’11: Proceedings of the 10th International Conference on Software Testing, Verification and Validation, 2017.
    [Bibtex]
    @inproceedings{icst17_mocking,
        AUTHOR="A. Arcuri and G. Fraser and R. Just",
        TITLE={Private API Access and Functional Mocking in Automated Unit Test Generation},
      booktitle = {ICST'11: Proceedings of the 10th International Conference on Software Testing, Verification and Validation},
        YEAR="2017",
        note={To appear}
    }

SSBSE 2016: Java Enterprise Edition Support in Search-Based JUnit Test Generation

  • [PDF] A. Arcuri and G. Fraser, “Java Enterprise Edition Support in Search-Based JUnit Test Generation,” in International Symposium on Search Based Software Engineering, 2016, pp. 3-17.
    [Bibtex]
    @inproceedings{ssbse16_jee,
      title={Java Enterprise Edition Support in Search-Based JUnit Test Generation},
      author={Arcuri, Andrea and Fraser, Gordon},
      booktitle={International Symposium on Search Based Software Engineering},
      pages={3--17},
      year={2016},
      organization={Springer}
    }

EvoSuite Tutorials

The EvoSuite documentation on our webpage now includes 4 tutorials on using and extending various aspects of EvoSuite:

Some of this material has featured in the tutorials about EvoSuite and search-based testing we have given over the summer: