[M3devel] Open CM3 regression tests
Tony Hosking
hosking at cs.purdue.edu
Sun Jan 20 22:38:16 CET 2008
The set operations are all coded as external C functions -- should be
easy enough to fix those. Or are there type issues too?
On Jan 20, 2008, at 4:18 PM, Olaf Wagner wrote:
> Hi again everybody,
>
> I got a little bit side-tracked during recent days while trying
> to set up a sensible interface for test status reporting
> by the poor state of the CM3 WWW presentation. So I tried to improve
> the presentation and structure of the site. It's still all simple
> hand-crafted HTML and nothing very fancy, though I think it looks
> better now than before. Let me know what you think. And I know,
> the colours will be the most interesting discussion topic ;-)
>
> But back to more important issues. There are still a number of
> problems in the CM3 compiler regression tests that I'd like to
> discuss and close: either adapt the test, fix the compiler, or
> document the issue as intended or erroneous behaviour of CM3.
> Though I've thrown in some names here and there, nobody should
> hesitate to voice his opinion. Here's the extract of the test run
> log:
>
> CM3_TARGET=FreeBSD4
> BINDISTMIN=/home/wagner/cm3-min-POSIX-FreeBSD4-5.4.0.tgz
> === 2008-01-19 23:43:24 run cm3 compiler and runtime regression
> test suite in /home/wagner/work/cm3-ws/luthien-2008-01-19-02-30-01
> with lastok version
> Critical Mass Modula-3 version d5.5.1
> last updated: 2007-12-30
> compiled: 2008-01-19 17:27:13
> configuration: /home/wagner/work/cm3-inst/luthien/current/bin/
> cm3.cfg
>
> --- building in FreeBSD4 ---
>
>> no errors for p1 to p115
>
> --- p116b --- default IEEE floating point tests from Xerox PARC
> --- p1/p116b/stderr.pgm 2008-01-20 00:47:20.000000000 +0100
> +++ ../src/p1/p116b/stderr.pgm 2008-01-13 00:55:55.000000000 +0100
> @@ -17,7 +17,7 @@
> Class (MaxFinite*ten) test OK
> Finite (MaxFinite*ten) test OK
> IsNaN (MaxFinite*ten) test OK
> -** Class (zero/zero) test not OK: FALSE should be TRUE
> + Class (zero/zero) test OK
> Finite (zero/zero) test OK
> IsNaN (zero/zero) test OK
> Sign (zero) test OK
> @@ -58,7 +58,7 @@
> Class MaxFinite*ten test OK
> Finite MaxFinite*ten test OK
> IsNaN MaxFinite*ten test OK
> -** Class zero/zero test not OK: FALSE should be TRUE
> + Class zero/zero test OK
> Finite zero/zero test OK
> IsNaN zero/zero test OK
> Sign (zero) test OK
>
>> Some arithmetic problems. Perhaps Henning Thielemann could have
>> a look at this?
>
> --- p117 --- SUBARRAY (LOOPHOLE)
> OK again
> --- p155 --- operations on small sets
> --- p1/p155/stderr.pgm 2008-01-20 00:48:16.000000000 +0100
> +++ ../src/p1/p155/stderr.pgm 2008-01-13 00:55:56.000000000 +0100
> @@ -2,7 +2,6 @@
> check set q = {}
> check set r = {a, b}
> check set x = {a, b, e}
> -************************ ERROR: check (NOT (p >= x)) failed
>
>> This concerns the >= operation on sets. The language definition
>> says:
>>
>> infix <=, >= (x,y: Ordinal) : BOOLEAN
>> (x,y: Float) : BOOLEAN
>> (x,y: ADDRESS) : BOOLEAN
>> (x,y: Set) : BOOLEAN
>>
>> In the first three cases, <= returns TRUE if x is at most as large as
>> y. In the last case, <= returns TRUE if every element of x is an
>> element of y. In all cases, it is a static error if the type of x is
>> not assignable to the type of y, or vice versa. The expression x >= y
>> is equivalent to y <= x.
>>
>> So the implementation seems to be plainly wrong. I think the test
>> for the previous operation <= only succeeds accidentally.
>
>
> --- p156 --- operations on medium-sized sets
>> OK
>> until
>
> --- p172 --- REAL vs. C's float
> --- p1/p172/stdout.pgm 2008-01-20 00:49:10.000000000 +0100
> +++ ../src/p1/p172/stdout.pgm 2003-03-08 23:36:48.000000000 +0100
> @@ -0,0 +1 @@
> +23 23
> --- p1/p172/stderr.pgm 2008-01-20 00:49:10.000000000 +0100
> +++ ../src/p1/p172/stderr.pgm 2003-03-08 23:36:48.000000000 +0100
> @@ -1,5 +1,3 @@
> -23.4 23
> -************************ ERROR: 23.4 instead of 23
>> result too exact?
>> Perhaps Henning again? And Tony?
>
> --- p173 --- LONGREAL vs. C's double
>> OK again until
>
> --- p185 --- REAL vs. C float
> --- p1/p185/stderr.pgm 2008-01-20 00:49:32.000000000 +0100
> +++ ../src/p1/p185/stderr.pgm 2003-03-08 23:36:50.000000000 +0100
> @@ -1,4 +1,3 @@
> -************************ ERROR: 53 instead of 24
>
>> This is another effect of the precision problem. Expression
>> evaluation
>> seems always to be done with 53 bits of precision (double), and never
>> with pure reals (23 bits). I assume it will be the same for C on
>> all PC hardware at least. Should we consider this to be an error or
>> just document the behaviour?
>>
>> I assume it cannot easily be changed, can it?
>
> --- p186 --- case statement with large labels
>> Just some output differences; I seem to have switched core dumps
>> off in one environment...
>>
>> So we can probably ignore these ones.
>
> --- r0/r001/stdout.build 2008-01-20 00:50:07.000000000 +0100
> +++ ../src/r0/r001/stdout.build 2008-01-13 00:55:57.000000000 +0100
> @@ -1,3 +1,3 @@
> "../src/Main.m3", line 13: warning: potentially unhandled
> exception: Main.a
> 1 warning encountered
> -Abort trap (core dumped)
> +Abort trap
> --- r002 --- stack overflow in the main thread
> --- r0/r002/stdout.build 2008-01-20 00:50:09.000000000 +0100
> +++ ../src/r0/r002/stdout.build 2008-01-13 00:55:57.000000000 +0100
> @@ -1 +1 @@
> -Bus error (core dumped)
> +Bus error
> --- r003 --- b3tests/b002 - improper size for an open array parameter
> --- r0/r003/stdout.build 2008-01-20 00:50:11.000000000 +0100
> +++ ../src/r0/r003/stdout.build 2008-01-13 00:55:57.000000000 +0100
> @@ -1,4 +1,4 @@
> "../src/Main.m3", line 11: warning: open array passed by value (x)
> "../src/Main.m3", line 13: warning: large parameter passed by
> value (40 bytes) (x)
> 2 warnings encountered
> -Abort trap (core dumped)
> +Abort trap
> --- r004 --- negative size for an open array
> --- r0/r004/stdout.build 2008-01-20 00:50:13.000000000 +0100
> +++ ../src/r0/r004/stdout.build 2008-01-13 00:55:57.000000000 +0100
> @@ -1 +1 @@
> -Abort trap (core dumped)
> +Abort trap
>
>> That were all the functional tests. Following is only error detection
>> and handling:
>>
>> There seems to be a problem recognizing some recursive declarations:
>
> --- e0/e020/stdout.build 2008-01-20 00:50:32.000000000 +0100
> +++ ../src/e0/e020/stdout.build 2008-01-09 02:15:42.000000000 +0100
> @@ -1 +1 @@
> -Bus error (core dumped)
> +Fatal Error: package build failed
>
> --- e021 --- illegal recursive declaration
>> OK until
>
> --- e026 --- two types with the same brand
> --- e0/e026/stdout.build 2008-01-20 00:50:38.000000000 +0100
> +++ ../src/e0/e026/stdout.build 2003-03-08 23:36:13.000000000 +0100
> @@ -0,0 +1,3 @@
> +"../src/Main.m3", line 8: string: duplicate brand
> +"../src/Main.m3", line 7: string: duplicate brand
> +2 errors encountered
>
>> This seems to be just different/missing error output.
>>
>> Another more obvious example of this is here:
>
> --- e029 --- use type instead of value as an initializer
> --- e0/e029/stdout.build 2008-01-20 00:50:41.000000000 +0100
> +++ ../src/e0/e029/stdout.build 2008-01-09 02:15:44.000000000 +0100
> @@ -1,11 +1,4 @@
> "../src/Main.m3", line 24: warning: variable has type NULL (r)
> -"../src/Main.m3", line 24: ** INTERNAL CG ERROR *** stack not
> empty, depth (1)
> -"../src/Main.m3", line 25: ** INTERNAL CG ERROR *** stack not
> empty, depth (1)
> -"../src/Main.m3", line 26: ** INTERNAL CG ERROR *** unaligned
> load_indirect type=11 s/a=0/8
> -"../src/Main.m3", line 26: ** INTERNAL CG ERROR *** unaligned
> store_indirect type=11 s/a=0/8
> -"../src/Main.m3", line 26: ** INTERNAL CG ERROR *** stack not
> empty, depth (1)
> -"../src/Main.m3", line 26: ** INTERNAL CG ERROR *** stack not
> empty, depth (1)
> -"../src/Main.m3", line 22: ** INTERNAL CG ERROR *** stack not
> empty, depth (1)
> -"../src/Main.m3", line 22: 2 code generation errors
> -8 errors and 1 warning encountered
> +"../src/Main.m3", line 24: types are not assignable
> +1 error and 1 warning encountered
> Fatal Error: package build failed
>
>> Just ignore it?
>>
>> For reference, here's the extract of the programs' stderr output:
>
>>>> test_m3tests error extract:
> FreeBSD4/p1/p116b/stderr.pgm:** Class (zero/zero) test not OK:
> FALSE should be TRUE
> FreeBSD4/p1/p116b/stderr.pgm:** Class zero/zero test not OK: FALSE
> should be TRUE
> FreeBSD4/p1/p155/stderr.pgm:************************ ERROR: check
> (NOT (p >= x)) failed
> FreeBSD4/p1/p155/stderr.pgm:1 error(s) and 0 warning(s) detected
> FreeBSD4/p1/p172/stderr.pgm:************************ ERROR: 23.4
> instead of 23
> FreeBSD4/p1/p172/stderr.pgm:1 error(s) and 0 warning(s) detected
> FreeBSD4/p1/p185/stderr.pgm:************************ ERROR: 53
> instead of 24
> FreeBSD4/p1/p185/stderr.pgm:1 error(s) and 0 warning(s) detected
> FreeBSD4/r0/r001/stderr.pgm:***
> FreeBSD4/r0/r001/stderr.pgm:*** runtime error:
> FreeBSD4/r0/r001/stderr.pgm:*** Unhandled exception: Main.a
> FreeBSD4/r0/r001/stderr.pgm:*** file "../src/Main.m3", line 13
> FreeBSD4/r0/r001/stderr.pgm:***
> FreeBSD4/r0/r003/stderr.pgm:***
> FreeBSD4/r0/r003/stderr.pgm:*** runtime error:
> FreeBSD4/r0/r003/stderr.pgm:*** An open array had the wrong shape.
> FreeBSD4/r0/r003/stderr.pgm:*** file "../src/Main.m3", line 19
> FreeBSD4/r0/r003/stderr.pgm:***
> FreeBSD4/r0/r004/stderr.pgm:***
> FreeBSD4/r0/r004/stderr.pgm:*** runtime error:
> FreeBSD4/r0/r004/stderr.pgm:*** An enumeration or subrange value
> was out of range.
> FreeBSD4/r0/r004/stderr.pgm:*** file "../src/runtime/common/
> RTAllocator.m3", line 309
> FreeBSD4/r0/r004/stderr.pgm:***
>>>> 7 in test_m3tests 2008-01-19-02-30-01 /home/wagner/work/cm3-ws/
>>>> luthien-2008-01-19-02-30-01
> === 2008-01-19 23:50:47 cm3 m3tests run done
>
> Thanks in advance for your comments,
>
> Olaf
> --
> Olaf Wagner -- elego Software Solutions GmbH
> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin,
> Germany
> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23
> 45 86 95
> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz:
> Berlin
> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr:
> DE163214194
More information about the M3devel
mailing list