Detection of Software Clones

This page is meant to be a general repository and information centre for Detection of Software Clones. As nearly all web pages, it is under permanent construction.

Tool comparison experiment

Time schedule

File format for submissions

Format 1

lineend   := '\n'
separator := '\t'
digit     := '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9'
digit0    := '0' | digit
number    := digit [digit0*]
filename1 := valid-filename
fromline1 := number
toline1   := number
filename2 := valid-filename
fromline2 := number
toline2   := number
clonetype := '0' | '1' | '2' | '3'
line      := filename1 separator fromline1 separator toline1
             filename2 separator fromline2 separator toline2
             separator clonetype lineend
file      := line [line*]

foo.c	12	56	bar.c	68	90	1
wom.c	34	50	bat.c	90	124	2
wom.c	69	100	bar.c	45	70	1
wom.c	69	100	foo.c	59	80	1
bar.c	45	70	foo.c	59	80	1

I have now created an Emacs mode for visualizing clone pairs in this "format 1" (or Clone Pair Format) mode. You can download it from below.

Format 2

lineend   := '\n'
separator := '\t'
marker    := '§'
digit     := '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9'
digit0    := '0' | digit
number    := digit [digit0*]
filename  := valid-filename
fromline  := number
toline    := number
clonetype := '0' | '1' | '2' | '3'
line      := filename separator fromline separator toline lineend
clonelist := marker separator clonetype newline line line*
file      := clonelist [clonelist*] marker newline

§	1
foo.c	12	56
bar.c	68	90
§	2
wom.c	34	50
bat.c	90	124
§	1
wom.c	69	100
bar.c	45	70
foo.c	59	80
§

• fromline/toline are both inclusive. I.e. fromline=n and toline=m represents m-n+1 lines of code.
• valid-filename should be the path to the file, starting from the system/project directory (e.g. spule/src/server/ClientConnection.java is a valid-filename of the project spule in our test experiments).
• As we're only interested in clones in the main system, I'll drop every clone where one code snippet contains a filename without the /src/ part in it, as this marks the main system.
• Please make sure that no clones are reported that are smaller than 6 lines of un-processed source code in size. I'll skip those when importing your data.
• The clones found in each system in our experiment should go into an individual single file. Please don't submit all clones for all systems in one file and please don't submit several files for one system (as we don't do cross-system clone detection in our experiment, it's no problem to separate clones of different systems into different files).

• Type 1 is an exact copy. No modifications (except for whitespace and comments) are allowed.
• Type 2 is a parametrized copy. Variable names and function calls have been renamed and/or types have been changed.
• Type 3 is a copy with further modification. Statements have been added or removed.

Resources for the main experiment

Concerning the C Programming Language projects, I have switched from using GNU/Linux/i386 as configuration to SunOS/spark. All sources for the C Programming Language have been tested with the Sun C Compiler's -Xc switch, which enforces strict ANSI C compliance.
Every C project has a file check.sh included in the archive which I used to test it. You can see the defines (e.g. HAVE_CONFIG_H) and include paths you have to use from those files. In addition you should define the following defines as the Sun C Compiler automatically defines them as well:

    __sun
    __unix
    __SUNPRO_C = 400
    __sparc
    __BUILTIN_VA_ARG_INCR
    __SVR4
    __STDC__
    

For the Java projects, I provide jar archives of classes our chosen projects depend on. All projects depend on rt.jar. The other jars are listed below for each project.

weltab.tar.gz	Code for Weltab (System 1 in the C Programming Language)
cook.tar.gz	Code for Cook (System 2 in the C Programming Language)
snns.tar.gz	Code for SNNS (System 3 in the C Programming Language)
postgresql.tar.gz	Code for Postgresql (System 4 in the C Programming Language)
netbeans-javadoc.tar.gz	Code for NetBeans Javadoc (System 1 in the Java Programming Language) Jar files of classes imported by NetBeans Javadoc: tools.jar and netbeans-support.jar.
eclipse-ant.tar.gz	Code for Eclipse Ant (System 2 in the Java Programming Language) Jar files of classes imported by Eclipse Ant: j2ee.jar, jakarta-oro-2.0.5.jar and log4j-core.jar.
eclipse-jdtcore.tar.gz	Code for Eclipse JDTCore (System 3 in the Java Programming Language) Jar files of classes imported by Eclipse JDTCore: ant.jar, jakarta-ant-1.4.1-optional.jar, resources.jar, runtime.jar and xerces.jar.
j2sdk1.4.0-javax-swing.tar.gz	Code for Java 2 SDK 1.4.0 Swing components (System 4 in the Java Programming Language)

Results

Please note that submission data as well as the source code for the evaluation program have already been superceded. So, I advise you to download the ISO and then the additional updates from below if you need them.

For English speaking audience, there's an intermediate results report available in the archive results.tar.gz.

Here's the SQL data with all candidates and references:

data_orig_2percent_clean.sql.gz	Original data after submission deadline. No mappings present.
data_orig_2percent_mapped.sql.gz	Original data after submission deadline. With mappings from candidates to references.
data_new_2percent_clean.sql.gz	Data includes Jens Krinke's voluntary submission which was received after the experiment's deadline. No mappings present.
data_new_2percent_mapped.sql.gz	Data includes Jens Krinke's voluntary submission which was received after the experiment's deadline. With mappings from candidates to references.

Source code

Emacs mode for CPF files

Documents

Changelog

Stefan Bellon (bellon@informatik.uni-stuttgart.de), Last modified: Fri Oct 8 16:37:45 CEST 2004