Skip to content

Tamarin Test Score Card

jeffdyer edited this page Mar 27, 2013 · 14 revisions

(On this page we keep track of test results and comment on classes of failures)

Test score

Date suite flags pass fail uxpass xfail
2013-03-27 ecma3 -c4 36816 27 465 731
Avoid boxing as3 -c4 7865 91 48 868
2013-03-27 ecma3 -i 36842 0 465 731
-c4 36706 138 469 727
as3 -i 7967 0 46 873
-c4 7847 109 41 875
2013-03-06 ecma3 -i 42772 709 0 0
as3 -i 7097 1212 0 0
2013-02-28 ecma3 -i 41424 2036 0 0
as3 -i 7026 1242 0 0

NOTE: uxpass = unexpected passes; xfail = expected fails

NOTE: The test harness was cleaned up as of 3/27 so the smaller number of ecma3 tests reflects that some tests are now excluded.

NOTE: -c4 implies -verify

Causes of failures

ecma3

  • Prototype confusion. e.g. Boolean.prototype vs Object.prototype
  • Unsealed objects
  • Unicode support
  • Differing error messages

as3

  • Namespace URI mangling: expected: www.ecma-international.org got: www$ecma$international$org
  • Interfaces after the first one in the implements list of are class are ignored
  • Differing error messages

Detailed test results

Here is the Google spreadsheet where we track test results: Spreadsheet of test results