[M3devel] cm3 command line interface and output; was: RE: www.opencm3.net/m3tests

[Tony Hosking]

Oops I meant *Olaf* !

On May 6, 2008, at 10:06 AM, Tony Hosking wrote:

> I'm am strongly with Jay on this one.
>
> On May 6, 2008, at 9:33 AM, Olaf Wagner wrote:
>
>> Quoting Jay <jayk123 at hotmail.com>:
>>
>>> When I say show cm3cg/as invocations, I mean in this new TARGET/ 
>>> cm3.log.
>>> As I said, I propose exactly the stdout/stderr you want, but then   
>>> some other output always, a log file.
>>> Also, about -commands, I have also claimed that it does work --   
>>> talking out of both sides of my mouth. The extra commands it  
>>> prints  are obviously harmless. I forget what doesn't print, maybe  
>>> exec vs.  q_exec? Also the '@' character wasn't working with  
>>> q_exec, but that  isn't necessarily relevant here..I forgot where  
>>> I use exec vs.  q_exec..
>>> And there plenty of other ways to debug than -commands and it is   
>>> "just" a debugging facility..
>>
>> I've got a rather definite opinion about the output behaviour of
>> cm3. It's like this, and I would like others to comment, too:
>>
>> o cm3 without any switches (-commands, -debug, -verbose) should
>>  not output any called commands, neither to stdout nor stderr.
>>  It may print what it is doing in an abstract, target independent
>>  (short) way. This output may refer to compilation units, interfaces,
>>  modules, and compilation phases, but not to target specific commands
>>  and files (unless there are errors, of course) Even then I'd like
>>  to abstract from differences.
>>
>> o cm3 -silent should suppress the normal output, except for warnings
>>  and error messages.
>>
>> o cm3 -commands should output exactly all invokations of externals
>>  commands so that they can be re-executed from the command line
>>  if necessary.
>>
>> I realize that it might not be this way in every aspect, so that
>> we may need to adapt the code here and there. But in my experience
>> it has worked this way quite well. Here are some points to consider:
>>
>> o We need consistent behaviour and output across all target  
>> platforms.
>>  This is not just a requirement of regression testing, but also of
>>  a consistent user interface. I don't want different behaviour on
>>  different platforms.
>>
>> o If -commands does not work as expected for some use case,
>>  we need to fix it.
>>
>> o If q_exec or whatever quake function is still deficient in this
>>  respect, we need to fix/adapt it. It should not be much effort.
>>
>> o I find the options -commands, -verbose, -debug, -silent a very
>>  good categorization of information types, intuitively understandable
>>  and usable in practive (as I've never had any problems with it, but
>>  have used them efficiently for years). So I don't see the point
>>  in changing anything, unless there is a better concept.
>>
>> o I don't think it is a good idea to write any log files by default
>>  (apart from the stdout and stderr streams).
>>
>> This is of course just my opinion, but unless there are others in
>> favour of changes (and I'd like some motivation for them) I'd insist
>> that we keep the existing design.
>>
>> Olaf
>>
>>> From: jayk123 at hotmail.comTo: wagner at elegosoft.com;  m3devel at elegosoft.comDate 
>>> : Tue, 6 May 2008 10:57:01 +0000Subject:  Re: [M3devel] www.opencm3.net/m3tests
>>>
>>>
>>> Olaf, I don't entirely agree.The logs should be fairly  
>>> informative  without intervention.Just not too verbose. I'd would  
>>> rather remove  the lines that say "compiling" and show the cm3cg  
>>> invocations.But  more than one line per source file is too much.- 
>>> commands doesn't  really work. Lots of bogus extra commands are  
>>> echoed (rm .tmp), and  not all the commands that are run are  
>>> echoed. I understand that  stability for tests is also a goal, but  
>>> information for the  interactive user and for post mortem  
>>> debugging via a log is also  valuable, and these are contrary to  
>>> stability-for-tests. Somehow  contradictory goals need to be met.  
>>> How about a compromise?How about  we always log more stuff to  
>>> TARGET/cm3.log? And then exactly what  you want to stdout/stderr? 
>>> While, as I said, more than one line per  file is overkill, as  
>>> part of this compromise I'd be willing to  include the cm3cg and  
>>> as invocation. Have folks used build.exe in  the Windows NT DDK? 
>>> It's model is close to what you get here.It has  three sets of  
>>> information:  to the console/stdout   build.log,  build.exe is a  
>>> wrapper over make, and this includes all the make  output -- all  
>>> the command lines run   build.err, a small subset of  console/ 
>>> stdout output My "design" here has been to avoid  over-verbosity.  
>>> Link and mt run on the order of once, or a small  number of times,  
>>> per directory, so showing the full command line  (response  
>>> files...) isn't so verbose -- vs. say something shown once  per  
>>> compiled file. There's another "problem" here that I have partly   
>>> fixed.Underlying errors are not shown on console/stdout/ 
>>> stderr.They  are often in like TARGET/m3core.lst. I changed stuff  
>>> to refer to  that file, after having hunted the errors down myself  
>>> slowly the  first time. We could change things to alway says "see   
>>> TARGET/cm3.log" for more information, or somesuch.  - Jay
>>>
>>>> Date: Tue, 6 May 2008 10:07:11 +0200> From:  
>>>> wagner at elegosoft.com>  To: m3devel at elegosoft.com> Subject: Re:  
>>>> [M3devel]  www.opencm3.net/m3tests> > Quoting Jay <jayk123 at hotmail.com 
>>>> >:> > >  The "problem" here, a really really small one, is just  
>>>> that the  link > > and mt commands got echoed.> > Olaf made them  
>>>> not echoed.  I then made them conditionally echoed.> > I made m3- 
>>>> sys\m3tests  define M3TESTS so that NT386.common doesn't echo.> >  
>>>> It's not a big  deal either way.> > Aha -- tests in other  
>>>> directories would have a  problem, and I think > > there are  
>>>> some.> >> > I like more echoing  usually, so the system explains  
>>>> what is going on,> > instead of the  vaguer "linking foo" sort of  
>>>> message.> > Granted, nobody bothers  watching gcc run assembler  
>>>> commands, so I > > guess it is just  quite gray.> > Jay, cm3  
>>>> needs to behave the same on all platforms.  By default> commands  
>>>> should _not_ be echoed to stdout or stderr;  that's what the> - 
>>>> commands switch is for: this gives you exactly  the commands the>  
>>>> compiler issues to invoke other tools.> > For  more verbose  
>>>> output, you may also depend on the -verbose or the>  -debug  
>>>> switches.> > Please adapt the configuration files that bey   
>>>> default all echoing> of commands is off.> > Olaf> > > I don't  
>>>> know  how to run the Tinderbox either yet, sorry.> > I tried.> >>  
>>>> > For  adding tests, well, there is m3-sys\m3tests.> > That is  
>>>> where a lot  of the tests are, but not all.> > I am not sure  
>>>> where tests belong.  I added a small number there.> > They are  
>>>> named with just a letter  and a number.> > The letters have some  
>>>> meaning, explained in the  m3makefile.> > "p" is programs that  
>>>> are run and their stdout/stderr  compared > > against expected> >  
>>>> "e" is programs build and the  errors from the compiler checked  
>>>> to be > > reasonable.> > I don't  think in general they are  
>>>> expected to successfully compile > > or  run, though the case of  
>>>> "warnings" may be unclear.> > "c" is  programs built so a human  
>>>> can look at the generated code.> > There  is another case I think  
>>>> for human checking.> > The numbers are just  001, 002, 003, etc.>  
>>>> > Each hundred tests are in a separate  directory, like p0\p001,  
>>>> > > p0\p002, p1\p100, p1\101, etc.> >> >  Something like this.> >  
>>>> Wherever I have details wrong, just look in  m3-sys\m3tests. It's  
>>>> > > pretty simple, obvious, and well  commented.> >> > The output  
>>>> is a little clearer if you have a  working diff.exe on the path.>  
>>>> > Then what you do is search for  "@@" in the output.> >> > -  
>>>> Jay> >> >> > Date: Tue, 6 May 2008  03:46:27 +0200From: dabenavidesd at yahoo.esTo 
>>>> : > >  m3devel at elegosoft.comSubject: [M3devel] www.opencm3.net/m3testsDear 
>>>>   > > developers:Does the recent tests on NT386 seem broken  
>>>> because a  > > recent change on the m3-sys tree, or is the HTML  
>>>> bad generated,  I > > mean can you check the last tests Sunday,  
>>>> May 4th (p001 to  p042) has > > a red background > >  http://www.opencm3.net/m3tests/m3tests-NT386-2008-04-23-13-30-57.html 
>>>> </pre></td></tr> <tr class="bgred tl"><td> p002 </td><td  
>>>> class="small" width="45%"> Text</td><td  
>>>> class="small"><pre>Comparing files P0\P002\stdout.build and ..\SRC 
>>>> \P0\P002\STDOUT.BUILD***** P0\P002\stdout.buildlink  
>>>> @_m3responsefile0.txt 2>&1 > pgm.lstmt /nologo /manifest  
>>>> pgm.exe.manifest /outputresource:pgm.exe;1***** ..\SRC 
>>>> \P0\P002\STDOUT.BUILD*****Comparing files P0\P002\stderr.build  
>>>> and ..\SRC\P0\P002\STDERR.BUILDFC: no differences  
>>>> encounteredComparing files P0\P002\stdout.pgm and ..\SRC 
>>>> \P0\P002\STDOUT.PGMFC: no differences encounteredComparing files  
>>>> P0\P002\stderr.pgm and ..\SRC\P0\P002\STDERR.PGMFC: no  
>>>> differences encounteredand almost the same pattern in the above  
>>>> tests.I would suggest if it is thinkable using NT386 variant with  
>>>> a complete dedicated machine/system, I can try to set up one this  
>>>> week and send the data back (the machine is behind a proxy), but  
>>>> I don't remember the mail explaining the set up, and also want to  
>>>> know if there is a chance of run the tests with one run  
>>>> script.Also what is the best natural way to put a new tests, and  
>>>> the standard name it should > > have?Thanks> >> >> > Enviado  
>>>> desde Correo Yahoo!La bandeja de entrada más inteligente.> > > >  
>>>> -- > Olaf Wagner -- elego Software Solutions GmbH> Gustav-Meyer- 
>>>> Allee 25 / Gebäude 12, 13355 Berlin, Germany> phone: +49 30 23 45  
>>>> 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95> http://www.elegosoft.com 
>>>>  | Geschäftsführer: Olaf Wagner | Sitz: Berlin> Handelregister:  
>>>> Amtsgericht Charlottenburg HRB 77719 | USt-IdNr:  DE163214194>
>>
>>
>>
>> -- 
>> Olaf Wagner -- elego Software Solutions GmbH
>>              Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin,  
>> Germany
>> phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23  
>> 45 86 95
>>  http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz:  
>> Berlin
>> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr:  
>> DE163214194
>>
>