SF110 Corpus of Classes

About SF110

Research on software testing produces many innovative automated techniques, but because software testing is by necessity incomplete and approximate, any new technique faces the challenge of an empirical assessment. Scientific advance is typically demonstrated using a set of examples that represent a particular problem addressed by the technique. However, demonstrating scientific advance is not necessarily the same as demonstrating practical value: A technique that works well on small, artificial problems might not scale up to the complexity of real systems. Ideally, one would use large “real-world” case studies to minimize the threats to external validity when evaluating research tools. However, neither choosing such case studies nor applying research prototypes to them are trivial tasks. To address this problem, we originally introduced the SF110 corpus of classes:

  • [PDF] G. Fraser and A. Arcuri, “A Large Scale Evaluation of Automated Unit Test Generation Using EvoSuite,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 24, iss. 2, p. 8, 2014.
    [Bibtex]
    @article{TOSEM_evaluation,
    author = {Gordon Fraser and Andrea Arcuri},
    title = {A Large Scale Evaluation of Automated Unit Test Generation Using EvoSuite},
    journal ={ACM Transactions on Software Engineering and Methodology (TOSEM)},
    year={2014},
    volume={24},
      number={2},
      pages={8},
      year={2014},
      publisher={ACM}
    }

The SF100 corpus of classes is a statistically representative sample of 100 Java projects from SourceForge, which is a popular open source repository (more than 300,000 projects with more than two million registered users). Because SourceForge is home to many old and stale projects, we have extended SF100 with the 10 most popular projects, resulting in a revised corpus of classes, SF110.

We now also make a source version available, where we massaged all projects into a common build infrastructure. Unit tests are included for those projects that come with tests, and for each project we provide a test suite produced with EvoSuite.

Download

All projects selected for the case study are open source projects. To check the individual licenses please check the projects’ websites (see table below). To download SF110, choose one of the following options:

Project details

Project Name Short Name Website Source
Tullibee tullibee Website Source repository
A4J (Amazon AWS 4 Java) a4j Website Source repository
Genetic Algorithms in Java gaj Website Source repository
Remote Invocation Framework (RIF) rif Website Source repository
templateIt templateit Website Source repository
jNFe – Java NFe jnfe Website Source repository
sfm sfmis Website Source repository
gfarceGestionFac gfarcegestionfa Website Source repository
FalseLight falselight Website Source repository
DAWN water-simulator Website Source repository
imSMART Migration imsmart Website Source repository
DSA Chat Program dsachat Website Source repository
jdbacl jdbacl Website Source repository
JState omjstate Website Source repository
BeanBin beanbin Website Source repository
JoomlaTemplateDetails templatedetails Website Source repository
Inspirento inspirento Website Source repository
JSecurity jsecurity Website Source repository
JMCA – Java Method Cohesion Analyzer jmca Website Source repository
NekoMUD nekomud Website Source repository
Geo-Google – Free Address Standardizer geo-google Website Source repository
byuic byuic Website Source repository
Java Wiki Bot Framework jwbf Website Source repository
saxpath saxpath Website Source repository
JNI-InChI jni-inchi Website Source repository
JIPA jipa Website Source repository
Framework for Game Related Research gangup Website Source repository
greencow greencow Website Source repository
APBSmem apbsmem Website Source repository
BluePenguinMail bpmail Website Source repository
Xisemele xisemele Website Source repository
HTTP Analyzer httpanalyzer Website Source repository
JVC – Java View Controller javaviewcontrol Website Source repository
SBMLReader 2 Plugin for Cytoscape sbmlreader2 Website Source repository
Corina corina Website Source repository
SchemaSpy schemaspy Website Source repository
PetSoar – Open Source J2EE PetStore petsoar Website Source repository
javaBB javabullboard Website Source repository
Diffi diffi Website Source repository
Group Layout Engineer glengineer Website Source repository
Follow follow Website Source repository
asphodel asphodel Website Source repository
lilith lilith Website Source repository
summa summa Website Source repository
Lotus lotus Website Source repository
Nutzenportfolioanalyse nutzenportfolio Website Source repository
DVD Home Video Project dvd-homevideo Website Source repository
resources4j resources4j Website Source repository
Die Bierse diebierse Website Source repository
Pascal interpreter written in Java biff Website Source repository
Java Interactive Profiler jiprof Website Source repository
Lagoon lagoon Website Source repository
Shp2KML shp2kml Website Source repository
DB-Everywhere db-everywhere Website Source repository
Lavalamp lavalamp Website Source repository
jHandballMoves jhandballmoves Website Source repository
hft-bomberman hft-bomberman Website Source repository
FPS370 – First Person Shooter Template fps370 Website Source repository
MyGrid – Distributed Grid mygrid Website Source repository
Sugar : making Java sweeter sugar Website Source repository
framework for dynamic analysis and test noen Website Source repository
dom4j: flexible XML framework for Java dom4j Website Source repository
ObjectExplorer4J objectexplorer Website Source repository
JTailPlus jtailgui Website Source repository
GoToServers Simple FTP Client gsftp Website Source repository
openjms openjms Website Source repository
Google App Engine Application Manager gae-app-manager Website Source repository
Bible Study Tool biblestudytool Website Source repository
macaw lhamacaw Website Source repository
EchoDep Hub and Spoke echodep Website Source repository
Ext4J ext4j Website Source repository
Battlecry battlecry Website Source repository
Free Instant Messenger Server/Client 1.0 fim1 Website Source repository
FixSuite fixsuite Website Source repository
OpenHRE openhre Website Source repository
dash-framework dash-framework Website Source repository
Io Project io-project Website Source repository
CalorieCount caloriecount Website Source repository
The West Fortbattle Player twfbplayer Website Source repository
Wheel wheelwebtool Website Source repository
JavAthena javathena Website Source repository
Internet Protocol Address Calculator ipcalculator Website Source repository
xBus xbus Website Source repository
IFX Framework ifx-framework Website Source repository
SHOP shop Website Source repository
Advanced T-Robots Version 2J at-robots2-j Website Source repository
JAW-BR jaw-br Website Source repository
JOpenChart Toolkit jopenchart Website Source repository
JIGL (Java Image and Graphics Library) jiggler Website Source repository
dcParseArgs dcparseargs Website Source repository
Class Viewer classviewer Website Source repository
JCVI Java Common jcvi-javacommon Website Source repository
QuickServer – TCP server framework quickserver Website Source repository
JCLO: A Java Command Line Option package jclo Website Source repository
CellWars2009 celwars2009 Website Source repository
HEAL: Health Education Assets Library heal Website Source repository
Feudalism Game feudalismgame Website Source repository
Transmitter Locator trans-locator Website Source repository
Newzgrabber newzgrabber Website Source repository
JGAAP jgaap Website Source repository
SAP Netweaver Server Adapter for Eclipse sapnweclipse Website Source repository
SQuirreL SQL Client squirrel-sql Website Source repository
Sweet Home 3D sweethome3d Website Source repository
Vuze azureus Website Source repository
FreeMind freemind Website Source repository
Eclipse Checkstyle Plugin eclipse-cs Website Source repository
Weka Machine Learning Software Weka Website Source repository
Liferay Portal lportal Website Source repository
PDF Split and Merge pdfsam Website Source repository
Firebird firebird Website Source repository