[M3devel] recent problems?
Jay
jay.krell at cornell.edu
Tue Mar 17 08:22:50 CET 2009
Truncated again.
Maybe a good thing.
(But do try to wade through, there are a few "new questions" -- 1) that link suggests going further with the wrapping. What do folks think? There's so little left to do, and it would provide a little value, so I am sympathetic to it..but it should probably be automated, now that I think about the actual volume 2) Tony, can you implement Modua-3 on top of Linux kernel futex/clone? That might address Daniel's point, and give us portability across C runtimes, such as Linux/ARM/uclibc and Linux/ARM/glibc; I have a Linux/ARM/uclibc system.).
Daniel, ok, is the choice:
"some code with no checking"
vs.
"more code with a little checking"
and, depending on "more vs. some".
and "no vs. little", you could argue
that "some with no" is better?
Agreed, it's simple math, but I think in this case we
are better with "more code with a little checking".
"It depends on the coefficients."
Again, I am confident in my C programming..having
written thousands of lines in the past few months..
- Jay
From: jay.krell at cornell.edu
To: dabenavidesd at yahoo.es; hosking at cs.purdue.edu
CC: m3devel at elegosoft.com
Subject: RE: [M3devel] recent problems?
Date: Tue, 17 Mar 2009 07:10:35 +0000
Daniel, I do not understand.
We were already depending on the underlying code.
Now we also have thin C wrappers and a little of checking by the C compiler.
Where before we had no checking.
I will admit I feel little hesitation to add more C code.
I am really fluent and comfortable in C and am well aware of its dangers and how to avoid them. But..well..I need more justification than "because I can".
I should add, btw, that that link suggests going a little beyond what I have done.
I have a few ad-hoc methodologies/goals.
Mainly "remove structs" and "remove constant #defines" (or wrap them).
However, "remove all types" is not a goal.
int remains, among others.
Where there is something "as simple as"
int foo(int);
I have held back and just left the direct:
<*EXTERNAL*> PROCEDURE foo(int):int;
however that link suggests that /everything/ should be wrapped.
It is easy to do, just:
Interface.i3
<*EXTERNAL Interface__Foo*> PROCEDURE foo(int):int:
Interface.c:
int Interface__Foo(int a)
{
return Foo(a);
}
Usually this does nothing but introduce a jmp or an unoptimized prolog/call/epilog.
(It can also help in getting callstacks and debugging, though that is a flawed argument.)
However if foo is a macro, or some fancy inline function (more common these days), it could be a useful transform. (Look at how stat works..)
I am interested in what people think here -- on the idea of "wrapping everything".
There isn't a lot of this stuff left, so maybe we should go ahead and wrap it.
Hey, actually, this will help with the "remains" -- the time stuff esp.
I can hardcode time_t et. al. as long and go through thin wrappers.
Sometimes I strive to wrap at a higher level, but that isn't necessary.
This might be what you are alluding to, and has been on my mind a while anyway;
I am sympathetic to the idea of eliminating C runtime dependencies and making direct kernel calls, at least on Linux. It is common on Linux/ARM systems to have uclibc instead of glibc.
glibc is deemed "bloated" in some circles.
Our binaries were not and are not portable to these systems. If we don't make this change, we are headed toward platforms that are more than architecture_kernel and instead architecture_kernel_cruntime -- ARM_LINUX_GLIBC, ARM_LINUX_UCLIBC, but it's worse than actually, ARM_LINUX_OLDABI_GLIBC, ARM_LINUX_OLDABI_UCLIBC, ARM_LINUX_UCLIBC, ARM_LINUX_GLIBC. I'd like to imagine "oldabi" is old and irrelevant, but I have such a system -- oldabi/uclibc -- the Western Digital MyBook "World Edition" is a viable target for Modula-3, on my list..though things are advancing..I have an $80 PogoPlug on the way, it might be much more performant..also, you could probably statically link to "any" C runtime, given our light usage..
"dietlibc" is another option, though I had trouble using it..and it seems a little too non-mainstream for me. "newlib" is also an option. But:
This kind of devolves to:
"Hey Tony, can you look into the Linux kernel 'futex' and 'clone' stuff et. al. and implement the Modula-3 threading library on top of it?" (Or anyone else?)
That's not the whole story though. There'd still be stat, open, and socket stuff.
But I think threading the main difficulty.
I'm pretty sure this viable -- that the Modula-3 runtime is "thick enough", that anyone who maintains it (Tony; not necessarily me) is capable of implementing reasonable threading "on top of almost anything".
- Jay
Date: Mon, 16 Mar 2009 20:05:24 -0700
From: dabenavidesd at yahoo.es
Subject: RE: [M3devel] recent problems?
To: hosking at cs.purdue.edu; jay.krell at CORNELL.EDU
CC: m3devel at elegosoft.com
The problem is not your changes, the problem is that putting C code can't be checked by a type safe checking process: I do agree the goodness of your checks, which may get things better, but unfortunately you have seen the effects of the this changes, we can't have human checking to solve unhidden bugs.
Compile type checking allows to discard problems in Modula-3 code and at least a clue that something needs more checking in UNSAFE Modules or Interfaces.
You have changed that we depend no more on cloned platform dependent .i3 files, but on C code which it slef depends on unsafe C platform dependent thirdparty libraries. That is the problem is not now on the Modula-3 source code but in the actual platform. I can't agree that is safer than it was before, nor because of the language, or you, but of the thid party code. Now we have more code just that it's no more Modula-3, it's C but yes, is lesser code (well forgeting about C preprocessor).
I hope this is more clear of my ideas and any clue of what i am tryin to say here.
Thanks for your comments.
--- El lun, 16/3/09, Jay <jay.krell at cornell.edu> escribió:
De: Jay <jay.krell at cornell.edu>
Asunto: RE: [M3devel] recent problems?
Para: "Daniel (M3)" <dabenavidesd at yahoo.es>, "Tony" <hosking at cs.purdue.edu>
CC: "m3devel" <m3devel at elegosoft.com>
Fecha: lunes, 16 marzo, 2009 9:47
I'm making the system more safe, not less.
The previous methodology was tedious, error prone, and didn't get any sort of checking.
Whatever was declared in the *.i3 files was presumed correct.
The linkage from the next layer to this layer gets the same amount of checking as it used.
The link you point to shows people fixing things the same way as I am making them.
unix/*.i3 must use <*EXTERNAL*> to declare C functions.
That is what it did. That is what it does.
Extend this out slightly to m3core/libm3.
Same thing.
The difference is whether or not those C functions are implemented right there, once, plain to see that they match the <*EXTERNAL*> declarations, or whether they are in a per-platform /usr/include and source tree somewhere, with tedious and error prone and per-platform verification as to if they are correct. As well as copy/pasting where there the "API" might be a macro.
Really, I am making the system more safe, not less.
There is no loss in checking but there is more obvious correctness by human inspection in what must be unsafe, one way or another.
The result is also more portable.
- Jay
Date: Mon, 16 Mar 2009 17:14:35 -0700
From: dabenavidesd at yahoo.es
To: hosking at cs.purdue.edu; jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] recent problems?
Hi all:
well, I know more about the kind of work done recently, which I can not just say is unsafe, but putting a layer of C code that obviously is dependent of each Linux/Unix, I guess in the past this is not always true, but having each Linux it's own headers and different implementations makes more unstable the system.
Besides the type checking made by C Compiler is not checked at link time, in contrast of Modula-3 that type checks whenever interfaces are changed and checks on link time.
So if you put change in your C code doesn't mean that Modula-3 will catch common silly but common true errors, like buffer overflows that if not checked at run time will crash (almost always) at runtime of a safe program, and it will not be safe (hidden code is beyond the Modula-3 compiler).
I strongly support simplifying and factoring as much as possible, but try to use what is there in Modula-3, not re-implementing in another Language if possible at all. I know but even more, the idea of having linking with C code on the kernel system calls allows to protect from so called side effects of Kernel code. Memory protection is exhausted here.
But when using true library user space system calls, this puts in a dangerous side the untrusted code. Here I reclaim again not using more that true system calls. I see that this started to be harder with AMD64 or x86_64 current platforms, so further development may be needed here see http://edoofus.blogspot.com/2008/08/interesting-bug-unbreaking-cvsupamd64.html about ezm3 related problems, is this supposed to be handled in Cm3 like the same they did?
Jay and all seem to do the better job they can, but I don't want to miss the opportunity to say this if I may. I will tell you later more about the type system If this it is useful to be said.
Thanks in advance
--- El lun, 16/3/09, Jay <jay.krell at cornell.edu> escribió:
De: Jay <jay.krell at cornell.edu>
Asunto: Re: [M3devel] recent problems?
Para: "Tony" <hosking at cs.purdue.edu>
CC: "m3devel" <m3devel at elegosoft.com>
Fecha: lunes, 16 marzo, 2009 3:55
It's ok on LINUXLIBC6 with the "new headers" but
I believe PPC_DARWIN problem caused by them so
backed out pending debugging.
This one looks easy to solve, though my time
is scarser next week or two.
I should have maybe considered:
"new headers" used on a number of Linux systems already
but no Darwin systems yet
- Jay
----------------------------------------
> From: hosking at cs.purdue.edu
> To: jay.krell at cornell.edu
> Date: Tue, 17 Mar 2009 01:54:34 +1100
> CC:
m3devel at elegosoft.com
> Subject: Re: [M3devel] recent problems?
>
> This is worrying as it seems some level of instability
has been
> introduced by recent changes. mentor used to be fine.
>
> On 17 Mar 2009, at 00:16, Jay wrote:
>
>>
>> mklib seems ok now, having started over with a new checkout, new
>> build.
>> Strange considering what I had already tried, this problem had been
>> plaguing me over a week.
>>
>>
>> stubgen doesn't hang with 1.5, will try 1.7 again/later.
>>
>>
>> But formsedit still crashes during startup intermittently,
>> and Cygwin X apps hang at startup.
>> ugh.
>> I know, I probably caused whatever problem..
>>
>>
>> - Jay
>>
>>
>>
>> ----------------------------------------
>>> From:
jay.krell at cornell.edu
>>> To: hosking at cs.purdue.edu
>>> CC: m3devel at elegosoft.com
>>> Subject: RE: [M3devel]
recent problems?
>>> Date: Mon, 16 Mar 2009 08:57:40 +0000
>>>
>>>
>>> Thanks.
>>>
>>>
>>> Here is another one.
>>> I found this testing moving PPC_DARWIN to "my headers".
>>> formsedit crashes during startup roughly more than
>>> half the times I start up, but not every time.
>>> The stack is always the same.
>>> I'll test it without my change.
>>> I think it'll happen either way, but I'll see.
>>> Yes, it recurs without my change, though I double
>>> check that my change is really undone..
>>> I'll have to dig into this.
>>>
>>>
>>> How does one get the code from CVS corresponding to a
date?
>>> cvs upd -D ?
>>> I'll try that or something, see if I can find when
things
worked.
>>>
>>>
>>> I should also be able to try the mklib crashing scenario
>>> on other platforms. It normally doesn't run on other
platforms.
>>>
>>>
>>> stubgen..if I'm ambitious, I can run in a loop on another
platform.
>>> But I think just moving Cygwin off of pthreads will
"fix" it
>>> and provide a cleaner base (not due to pthreads, but the Cygwin
>>> stuff).
>>>
>>>
>>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>>> Reason: KERN_PROTECTION_FAILURE at address: 0x00000004
>>> [Switching to process 19583 thread 0x2643]
>>> 0x0031798c in ScrollerVBTClass__PaintViewWithShadows
>>> (M3_Ao3CED_v=0x1f1eef4) at
../src/lego/POSIX/Sc
>>> rollerVBTClass.m3:325
>>> 325 VBT.PaintTint(v, r,
v.shadow.bg);
>>> (gdb) bt
>>> #0 0x0031798c in ScrollerVBTClass__PaintViewWithShadows
>>> (M3_Ao3CED_v=0x1f1eef4) at ../src/lego/POSI
>>> X/ScrollerVBTClass.m3:325
>>> #1 0x0031766c in ScrollerVBTClass__PaintView
>>> (M3_Ao3CED_v=0x1f1eef4) at ../src/lego/POSIX/ScrollerV
>>> BTClass.m3:292
>>> #2 0x0031a460 in ScrollerVBTClass__Repaint (M3_Ao3CED_v=0x1f1eef4,
>>> M3_A0Kbc5_rgn=0xf0284778) at ../
>>> src/lego/POSIX/ScrollerVBTClass.m3:693
>>> #3 0x0031a590 in ScrollerVBTClass__Redisplay
>>> (M3_Ao3CED_v=0x1f1eef4) at ../src/lego/POSIX/ScrollerV
>>> BTClass.m3:706
>>> #4 0x005c2870 in VBTClass__Redisplay (M3_BFdKo9_v=0x1f1eef4) at
../
>>> src/vbt/VBTClass.m3:367
>>>
#5 0x005cf1c0 in VBTRep__Redisplay () at ../src/vbt/VBTRep.m3:652
>>> #6
0x005ce59c in VBTRep__UncoverRedisplay () at ../src/vbt/
>>> VBTRep.m3:601
>>> #7 0x005ce724 in VBTRep__RdApply (M3_EMTrVz_self=0x1d554c8) at ../
>>> src/vbt/VBTRep.m3:606
>>> #8 0x0132ff88 in ThreadPThread__RunThread (M3_ChNkUD_me=0x1e23940)
>>> at ../src/thread/PTHREAD/ThreadP
>>> Thread.m3:531
>>> #9 0x0132fb58 in ThreadPThread__ThreadBase
>>> (M3_AJWxb1_param=0x1e23940) at ../src/thread/PTHREAD/Thr
>>> eadPThread.m3:508
>>> #10 0x9002b908 in _pthread_body ()
>>> (gdb)
>>>
>>>
>>>
>>> - Jay
>>>
>>>
>>>
>>> ----------------------------------------
>>>> From: hosking at cs.purdue.edu
>>>> To:
jay.krell at cornell.edu
>>>> Date: Mon, 16 Mar 2009 08:10:05 +1100
>>>> CC:
m3devel at elegosoft.com
>>>> Subject: Re: [M3devel] recent problems?
>>>>
>>>> You might have structure sizes wrong and your stack will get
shifted
>>>> around as the command line changes length. You get some data
in a
>>>> structure wrong and then hit the wrong stack org and you might
see
>>>> problems like this.
>>>>
>>>> I am not running things totally off of the head, so if the
problem
>>>> was
>>>> introduced recently it might only now be seen.
>>>>
>>>> On 16 Mar 2009, at 06:15, Jay wrote:
>>>>
>>>>>
>>>>> I'm seeing some problems lately. Maybe it is just me.
>>>>> Some of them I've had over
a week (ie: before my
(most) recent
>>>>> changes),
>>>>>
some of them even if recently appeared I verified without
my (most)
>>>>> recent changes.
>>>>>
>>>>> On Cygwin 1.5, mklib always crashes. I haven't been
able to debug
>>>>> it.
>>>>> This has been over a week.
>>>>> I've had to stop using Cygwin for Modula-3 development
on this
>>>>> machine.
>>>>> I have a good repro but hadn't had luck yet debugging
it.
>>>>> Basically if I shorten the mklib command line (in the
response
>>>>> file) substantially, no crash.
>>>>> There is a cut off somewhere.
>>>>>
>>>>> On Cygwin 1.7 machine, stubgen in obliqparse usually but
not always
>>>>>
hangs.
>>>>> It hangs nowhere else.
>>>>> This I verified occured
without my recent changes.
>>>>> I only just in the past few days tried Cygwin 1.7.
>>>>> mklib works fine here.
>>>>> Of course since this is somewhat intermittent, maybe it
isn't a new
>>>>> problem.
>>>>>
>>>>> I've only tried Cygwin 1.5 and 1.7 on one machine
each, separate
>>>>> machines,
>>>>> though they do coexist ok on one machine.
>>>>> 1.7 seemed much faster (on an unimpressive machine),
though 1.7
>>>>> binares
>>>>> apparently don't run on 1.5 -- at least not if they
use assert().
>>>>> 1.7 is a recent past or future release, a long time in the
works,
>>>>> drops Win9x support, lots of good
sounding changes.
>>>>>
>>>>> I should run m3tests more, maybe they
will reveal the
crash or the
>>>>> hang as well.
>>>>>
>>>>> On LINUXLIBC6 on one machine stubgen was crashing at least
in one
>>>>> directory.
>>>>> But I think this stopped happening.
>>>>>
>>>>> Cygwin being broken pushes me to other platforms more,
probably a
>>>>> good thing.
>>>>> I'm loathe to debug the hang. When it occurs I
can't see any stacks
>>>>> in gdb. I can see in strace..
>>>>> I think we are waiting for all the threads to suspend
themselves.
>>>>> I'll probably try moving back to Win32 threads and see
if it clears
>>>>> up.
>>>>> I know that's not
great.
>>>>> There isn't a clear implied question and I know I
should/need
to
>>>>> look into these,
>>>>> but I also thought I shouldn't keep this to myself. I
also
>>>>> understand the philosophy
>>>>> of stopping changes when there are problems, to stop
further
>>>>> damage, but, of course,
>>>>> I'm very confident in my changes (always..), have been
testing them
>>>>> on other platforms,
>>>>> and looking them over extra extra since I know of these
other
>>>>> problems.
>>>>>
>>>>> I'm not sure any problem on Cygwin is worrisome
really.
>>>>> I think the underlying code is kind of gnarly and I
don't think
>>>>> anyone uses this platform, though people asked for
it.
>>>>> But I did have a crash on
LINUXLIBC6.
>>>>>
>>>>> - Jay
>>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20090317/3a81795d/attachment-0002.html>
More information about the M3devel
mailing list