[M3devel] Open CM3 regression tests
Olaf Wagner
wagner at elegosoft.com
Sun Jan 20 22:18:32 CET 2008
Hi again everybody,
I got a little bit side-tracked during recent days while trying
to set up a sensible interface for test status reporting
by the poor state of the CM3 WWW presentation. So I tried to improve
the presentation and structure of the site. It's still all simple
hand-crafted HTML and nothing very fancy, though I think it looks
better now than before. Let me know what you think. And I know,
the colours will be the most interesting discussion topic ;-)
But back to more important issues. There are still a number of
problems in the CM3 compiler regression tests that I'd like to
discuss and close: either adapt the test, fix the compiler, or
document the issue as intended or erroneous behaviour of CM3.
Though I've thrown in some names here and there, nobody should
hesitate to voice his opinion. Here's the extract of the test run
log:
CM3_TARGET=FreeBSD4
BINDISTMIN=/home/wagner/cm3-min-POSIX-FreeBSD4-5.4.0.tgz
=== 2008-01-19 23:43:24 run cm3 compiler and runtime regression test suite in /home/wagner/work/cm3-ws/luthien-2008-01-19-02-30-01 with lastok version
Critical Mass Modula-3 version d5.5.1
last updated: 2007-12-30
compiled: 2008-01-19 17:27:13
configuration: /home/wagner/work/cm3-inst/luthien/current/bin/cm3.cfg
--- building in FreeBSD4 ---
> no errors for p1 to p115
--- p116b --- default IEEE floating point tests from Xerox PARC
--- p1/p116b/stderr.pgm 2008-01-20 00:47:20.000000000 +0100
+++ ../src/p1/p116b/stderr.pgm 2008-01-13 00:55:55.000000000 +0100
@@ -17,7 +17,7 @@
Class (MaxFinite*ten) test OK
Finite (MaxFinite*ten) test OK
IsNaN (MaxFinite*ten) test OK
-** Class (zero/zero) test not OK: FALSE should be TRUE
+ Class (zero/zero) test OK
Finite (zero/zero) test OK
IsNaN (zero/zero) test OK
Sign (zero) test OK
@@ -58,7 +58,7 @@
Class MaxFinite*ten test OK
Finite MaxFinite*ten test OK
IsNaN MaxFinite*ten test OK
-** Class zero/zero test not OK: FALSE should be TRUE
+ Class zero/zero test OK
Finite zero/zero test OK
IsNaN zero/zero test OK
Sign (zero) test OK
> Some arithmetic problems. Perhaps Henning Thielemann could have
> a look at this?
--- p117 --- SUBARRAY (LOOPHOLE)
OK again
--- p155 --- operations on small sets
--- p1/p155/stderr.pgm 2008-01-20 00:48:16.000000000 +0100
+++ ../src/p1/p155/stderr.pgm 2008-01-13 00:55:56.000000000 +0100
@@ -2,7 +2,6 @@
check set q = {}
check set r = {a, b}
check set x = {a, b, e}
-************************ ERROR: check (NOT (p >= x)) failed
> This concerns the >= operation on sets. The language definition
> says:
>
> infix <=, >= (x,y: Ordinal) : BOOLEAN
> (x,y: Float) : BOOLEAN
> (x,y: ADDRESS) : BOOLEAN
> (x,y: Set) : BOOLEAN
>
> In the first three cases, <= returns TRUE if x is at most as large as
> y. In the last case, <= returns TRUE if every element of x is an
> element of y. In all cases, it is a static error if the type of x is
> not assignable to the type of y, or vice versa. The expression x >= y
> is equivalent to y <= x.
>
> So the implementation seems to be plainly wrong. I think the test
> for the previous operation <= only succeeds accidentally.
--- p156 --- operations on medium-sized sets
> OK
> until
--- p172 --- REAL vs. C's float
--- p1/p172/stdout.pgm 2008-01-20 00:49:10.000000000 +0100
+++ ../src/p1/p172/stdout.pgm 2003-03-08 23:36:48.000000000 +0100
@@ -0,0 +1 @@
+23 23
--- p1/p172/stderr.pgm 2008-01-20 00:49:10.000000000 +0100
+++ ../src/p1/p172/stderr.pgm 2003-03-08 23:36:48.000000000 +0100
@@ -1,5 +1,3 @@
-23.4 23
-************************ ERROR: 23.4 instead of 23
> result too exact?
> Perhaps Henning again? And Tony?
--- p173 --- LONGREAL vs. C's double
> OK again until
--- p185 --- REAL vs. C float
--- p1/p185/stderr.pgm 2008-01-20 00:49:32.000000000 +0100
+++ ../src/p1/p185/stderr.pgm 2003-03-08 23:36:50.000000000 +0100
@@ -1,4 +1,3 @@
-************************ ERROR: 53 instead of 24
> This is another effect of the precision problem. Expression evaluation
> seems always to be done with 53 bits of precision (double), and never
> with pure reals (23 bits). I assume it will be the same for C on
> all PC hardware at least. Should we consider this to be an error or
> just document the behaviour?
>
> I assume it cannot easily be changed, can it?
--- p186 --- case statement with large labels
> Just some output differences; I seem to have switched core dumps
> off in one environment...
>
> So we can probably ignore these ones.
--- r0/r001/stdout.build 2008-01-20 00:50:07.000000000 +0100
+++ ../src/r0/r001/stdout.build 2008-01-13 00:55:57.000000000 +0100
@@ -1,3 +1,3 @@
"../src/Main.m3", line 13: warning: potentially unhandled exception: Main.a
1 warning encountered
-Abort trap (core dumped)
+Abort trap
--- r002 --- stack overflow in the main thread
--- r0/r002/stdout.build 2008-01-20 00:50:09.000000000 +0100
+++ ../src/r0/r002/stdout.build 2008-01-13 00:55:57.000000000 +0100
@@ -1 +1 @@
-Bus error (core dumped)
+Bus error
--- r003 --- b3tests/b002 - improper size for an open array parameter
--- r0/r003/stdout.build 2008-01-20 00:50:11.000000000 +0100
+++ ../src/r0/r003/stdout.build 2008-01-13 00:55:57.000000000 +0100
@@ -1,4 +1,4 @@
"../src/Main.m3", line 11: warning: open array passed by value (x)
"../src/Main.m3", line 13: warning: large parameter passed by value (40 bytes) (x)
2 warnings encountered
-Abort trap (core dumped)
+Abort trap
--- r004 --- negative size for an open array
--- r0/r004/stdout.build 2008-01-20 00:50:13.000000000 +0100
+++ ../src/r0/r004/stdout.build 2008-01-13 00:55:57.000000000 +0100
@@ -1 +1 @@
-Abort trap (core dumped)
+Abort trap
> That were all the functional tests. Following is only error detection
> and handling:
>
> There seems to be a problem recognizing some recursive declarations:
--- e0/e020/stdout.build 2008-01-20 00:50:32.000000000 +0100
+++ ../src/e0/e020/stdout.build 2008-01-09 02:15:42.000000000 +0100
@@ -1 +1 @@
-Bus error (core dumped)
+Fatal Error: package build failed
--- e021 --- illegal recursive declaration
> OK until
--- e026 --- two types with the same brand
--- e0/e026/stdout.build 2008-01-20 00:50:38.000000000 +0100
+++ ../src/e0/e026/stdout.build 2003-03-08 23:36:13.000000000 +0100
@@ -0,0 +1,3 @@
+"../src/Main.m3", line 8: string: duplicate brand
+"../src/Main.m3", line 7: string: duplicate brand
+2 errors encountered
> This seems to be just different/missing error output.
>
> Another more obvious example of this is here:
--- e029 --- use type instead of value as an initializer
--- e0/e029/stdout.build 2008-01-20 00:50:41.000000000 +0100
+++ ../src/e0/e029/stdout.build 2008-01-09 02:15:44.000000000 +0100
@@ -1,11 +1,4 @@
"../src/Main.m3", line 24: warning: variable has type NULL (r)
-"../src/Main.m3", line 24: ** INTERNAL CG ERROR *** stack not empty, depth (1)
-"../src/Main.m3", line 25: ** INTERNAL CG ERROR *** stack not empty, depth (1)
-"../src/Main.m3", line 26: ** INTERNAL CG ERROR *** unaligned load_indirect type=11 s/a=0/8
-"../src/Main.m3", line 26: ** INTERNAL CG ERROR *** unaligned store_indirect type=11 s/a=0/8
-"../src/Main.m3", line 26: ** INTERNAL CG ERROR *** stack not empty, depth (1)
-"../src/Main.m3", line 26: ** INTERNAL CG ERROR *** stack not empty, depth (1)
-"../src/Main.m3", line 22: ** INTERNAL CG ERROR *** stack not empty, depth (1)
-"../src/Main.m3", line 22: 2 code generation errors
-8 errors and 1 warning encountered
+"../src/Main.m3", line 24: types are not assignable
+1 error and 1 warning encountered
Fatal Error: package build failed
> Just ignore it?
>
> For reference, here's the extract of the programs' stderr output:
>>> test_m3tests error extract:
FreeBSD4/p1/p116b/stderr.pgm:** Class (zero/zero) test not OK: FALSE should be TRUE
FreeBSD4/p1/p116b/stderr.pgm:** Class zero/zero test not OK: FALSE should be TRUE
FreeBSD4/p1/p155/stderr.pgm:************************ ERROR: check (NOT (p >= x)) failed
FreeBSD4/p1/p155/stderr.pgm:1 error(s) and 0 warning(s) detected
FreeBSD4/p1/p172/stderr.pgm:************************ ERROR: 23.4 instead of 23
FreeBSD4/p1/p172/stderr.pgm:1 error(s) and 0 warning(s) detected
FreeBSD4/p1/p185/stderr.pgm:************************ ERROR: 53 instead of 24
FreeBSD4/p1/p185/stderr.pgm:1 error(s) and 0 warning(s) detected
FreeBSD4/r0/r001/stderr.pgm:***
FreeBSD4/r0/r001/stderr.pgm:*** runtime error:
FreeBSD4/r0/r001/stderr.pgm:*** Unhandled exception: Main.a
FreeBSD4/r0/r001/stderr.pgm:*** file "../src/Main.m3", line 13
FreeBSD4/r0/r001/stderr.pgm:***
FreeBSD4/r0/r003/stderr.pgm:***
FreeBSD4/r0/r003/stderr.pgm:*** runtime error:
FreeBSD4/r0/r003/stderr.pgm:*** An open array had the wrong shape.
FreeBSD4/r0/r003/stderr.pgm:*** file "../src/Main.m3", line 19
FreeBSD4/r0/r003/stderr.pgm:***
FreeBSD4/r0/r004/stderr.pgm:***
FreeBSD4/r0/r004/stderr.pgm:*** runtime error:
FreeBSD4/r0/r004/stderr.pgm:*** An enumeration or subrange value was out of range.
FreeBSD4/r0/r004/stderr.pgm:*** file "../src/runtime/common/RTAllocator.m3", line 309
FreeBSD4/r0/r004/stderr.pgm:***
>>> 7 in test_m3tests 2008-01-19-02-30-01 /home/wagner/work/cm3-ws/luthien-2008-01-19-02-30-01
=== 2008-01-19 23:50:47 cm3 m3tests run done
Thanks in advance for your comments,
Olaf
--
Olaf Wagner -- elego Software Solutions GmbH
Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
More information about the M3devel
mailing list