Results of the 2nd Rewriting Engines Competition

This page summarizes the presentation of the competition at WRLA'08. There, after introducing the procedure followed in the competition (slides), the five participating systems, namely ASF+SDF, Maude, Stratego/XT, TermWare and Tom, were introduced, and the results were discussed (the table below was used to present the results).

The table gives the user times (additional info in the links) for the different tests executed on each of the problems for each of the five participant systems. The machine was a 1GHz/1GB linux box.

The participants were asked to provide a tool to transform REC syntax into their corresponding syntax (compilers for Maude, Stratego/XT and Tom were provided). For each problem, in addition to the automatically generated solution, a second optimized version of the specification could be provided. Although no automatic trnaformation tool was provided for them, ASF+SDF and TermWare provided manually generated solutions for some of the problems.

Thus, in the table, for each of the problems, we give the original problem in REC syntax and the tests run, and the corresponding problems in the syntax of each the participant systems, both in the automatically generated and manually generated/optimized version, together with the times consumed in their computation (in milliseconds). The holes indicate that there is no result in such case, because it was not provided or, in some cases, because the computation terminated abruptly or didn't terminate in a reasonable amount of time.

(There are some links still missing in the table, e.g. the ASF-SDF problems were run and the times included, but the output files are not available yet, as some of the results for Tom.)

  ASF-SDF Maude Stratego/XT TermWare Tom
trs asfsdfbenchmark test 1 __ __ auto 7 auto 0 __  __  auto 177
test 2 __ 5402 2450 __ 3253
test 3 __ 7 0 __ 231
test 4 __ 7109 3330 __ 3400
test 5 __ 11 651320 __ __
test 6 __ 17609 __ __ __
test 1 manual 40 manual 11 manual __ manual 102 manual 108
test 2 430 9985 __ 86712 3236
test 3 0 8 __ 127 17
test 4 420 11960 __ 178269 3471
test 5 30 31 __ 154 16
test 6 490 40343 __ 150614 3795
factorial test 1 auto __ auto 1 auto __ __  __  auto __ 
test 2 __ __ __ __ __
test 1 manual 0 manual 0 manual __ manual __ manual __
test 2 ___ 0 __  __ __
fibonacci test 1 auto __ auto 1 auto 0 __   __ auto 17
test 1 manual __ manual 0 manual __ manual  4 manual 17 
garbagecollection test 1 auto __ auto 0 auto 0 __ __ auto 19
test 2 __ 0 0 __ 0
test 1 manual __ manual 0 manual __ manual 2 manual 19
test 2 __ 0 __ 0 0
rev test 1 auto __ auto 1 auto 770 __ __ auto 27
test 1 manual 0 manual 0 manual ___ manual 1 manual 18
ctrs bubblesort test 1 auto __ auto 4 auto 0 __ __ auto 35
test 2 __ 471 140 __ 218
test 3 __ 319 140 __ 184
test 1 manual 0 manual 4 manual __ manual 417 manual 35 
test 2 0 471 __ 67  218
test 3 60 319 __ __ 184 
closure test 1 auto __ auto __  auto __ __ __ auto  
test 2 __ __ __ __ __
test 1 manual __ manual __ manual __ __ __ manual __
test 2 __ __ __ __ __
fibfree test 1 auto __ auto 28 auto 20 __ __ auto 70
test 1 manual __ manual 28 manual __ manual 464 manual 65
hanoi test 1 auto __ auto 1 auto __ __ __ auto 44
test 1 manual 0 manual 1 manual __ manual 8 manual 45
mergesort test 1 auto __ auto 1 auto 0 __ __ auto  58
test 2 __ 50 __ __  26
test 3 __ 23599 __ __  614
test 1 manual __ manual 0 manual __ manual 62 manual 101
test 2 __ 0 __ 1228  30
test 3 __ 11 __ __  600
missionaries test 1 auto __ auto 40 auto 20 __ __ auto 102 
test 1 manual __ manual 7 manual __ manual 358 manual 102 
oddeven test 1 auto __ auto 192 auto 0 __  __ auto 18
test 2 __ 7 0 __ 235
test 3 __ ____ 0 __  1
test 1 manual 0 manual 0 manual __ manual 2 manual 21
test 2 0 0 __ 0 235 
test 3 0 0 __ 0  1
quicksort test 1 auto __ auto 2 auto __ __ __ auto 30
test 2 __ 113 __ __ 64
test 3 __ 495579 __ __ 11397
test 1 manual __ manual 0 manual __ manual 5 manual 30
test 2 __ 6 __ 22 64
test 3 __ 859 __ 2782 11397
modulo dart test 1 auto __ auto 114 auto 0 __ __   __ __
test 1 manual __ manual 0 manual __ manual 1339 __ __
logic3 test 1 auto __ auto 27 auto __ __ __ __ __
test 1 manual __ manual 27 manual __ manual __ __ __
permutations test 1 auto __ auto 1306 auto 0 __ __ __ __
test 1 manual __ manual 21 manual __ manual 19 __ __
tauthard test 1 auto __ auto 43 auto __ __ __ __ __
test 2 __ 278 __ __ __
test 3 __ 694 __ __ __
test 1 manual __ manual 43 manual __ manual __ __ __
test 2 __ 278 __ __ __
test 3 __ 694 __ __ __
cs sieve test 1 auto __ auto 4 auto __ __   __ __ __
test 2 __ 175 __ __ __
test 3 __ 235099 __ __ __
test 1 manual __ manual 4 manual __ manual 574 __ __
test 2 __ 175 __ __ __
test 3 __ 3332355 __ __ __