[M3devel] my status on win32

[Jay]

Regarding temp, I find that cm3 doesn't delete the response files qk, qk_1, etc. so I have modified cm3.cfg/NT386/NT386GNU to write the response files into the output directory, so they will be reliably cleaned up at some point.
That /might/ divorce cm3 from all interactions with TEMP/TMP.
 
It was tricky, because mklib demands newlines in the response file, spaces aren't adequate, and arglist doesn't always write a response file, such as for short command lines, and the command lines are nested lists, which don't seem to flatten like the docs say, I could find no way to flatten them.
 
When I get back home I think I'll send all my diffs and hope for one big approval and then submit them.
Maybe I'll figure out the NT386GNU runtime crash before that.
 
 - Jay


Date: Wed, 16 Jan 2008 15:55:51 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.com; jayk123 at hotmail.comSubject: RE: [M3devel] my status on win32

Jay:
 
Thanks for your reply.  I think most of your goals seem reasonable and I concur that there are nasty problems introduced by spaces and short names.  It's a messy situation that neither you nor I created.  I was just relaying some of my prior experience in getting things to work.
 
Indeed, for me, I try to avoid spaces always.  That's one reason I root my cm3 at "C:\cm3" instead of in "C:\Program Files\cm3" or "C:\Documents and Settings\userhome\cm3".  I've noted that with the various Visual Studio and Visual C tools going back for some time now that after you install it, your environment vars for TEMP and TMP get changed to use short names without spaces, so I guess even Microsoft has trouble with spaces.
 
I also recall in the past with cm3 having trouble if my TEMP and TMP vars included spaces, so having Microsoft switch them to use short names without spaces seemed to solve that cm3 problem at least.
 
As for the VC installation, I too recall that on some prior versions it asked you whether you wanted to set the environment vars permanently or to use vcvars.  This time, when I installed Visual Studio 2008, it did not ask this question, so I guess it is version dependant.  I'm not at the machine right now where I did the install, but later tonight I'll check to see if the VS2008 install set any permanent environment vars.
 
Again, for me, I would like to be able to build everything as the first step in my series of tests.  As soon as you can give me a fix to make things build, I can move on to make more progress.
 
Thanks for your help.
 
Regards,
Randy>>> Jay <jayk123 at hotmail.com> 1/16/2008 2:13 PM >>>Hi Randy. I mostly agree and understand.Go ahead with your installer work. :) #2 is no longer needed. The cm3.cfg I use, that is checked in, that is in the newer distributions, just runs plain "cl" and links to plain "kernel32.dll", and doesn't pass any /I switches to cl => therefore it depends on %PATH%, %LIB%, and %INCLUDE%. As I was saying/rambling, we/I can probably teach cm3 a bit so as to not need vcvars.As I recall, but I need to check, when you install VC, there are some variables it sets unconditionally, and some it asks the user if they should be set. The ones it asks about are PATH, LIB, INCLUDE. The ones it sets unconditionally are the "root" of the install. PATH / LIB / INCLUDE are more invasive, more impactful, have more potential to break things. They are also easily derived from the "root". Similarly, the CM3 installer could ask the user if it is ok to modify their %PATH%, and do it in the registry and set the WM_INI_CHANGED or such message so that Explorer picks it up. Need to check MSDN on this. Forward slashes are fine for options. I take a dim view of these space, quoting, escaping issues.I have very mixed experience here. In SOME situations, your command lines pass through various layers of code, with various attempts to handle or ignore the various issues. The result is in my opinion an unpredictable mess.The problem is that people have confused "an array of strings" with "one flat string". At this point the confusion is widespread and deeply ingrained. On Unix I believe people just ignore the issue by not using spaces (or quotes?) on the command line, though escaping issues still remain for other characters. You just have to read the documenation on CreateProcess to realize how messed up this is.c:\program files\foo I believe is interpreted multiple ways until something works.It can be running c:\program with the parameter files\foo, or it can be running c:\program files\foo. Reliable interprocess communication should not be via command lines but via something more strongly typed such as RPC or shared memory or reading/writing a Pickle. I use the command line a lot, and I frequently copy paths from the command line and paste them into the file open dialog, which does not accept forward slashes nor nested double slashes. It should but oh well. Therefore I have goal before much longer to ensure that every path generated and probably reported by cm3 is in the "normal" form. If the user types in a forward slash, perhaps that should be reported back though. And forward slashes should be ok to enter and then get used internally. The need to configure short file names are gone, along with the need to quote them, by virtue of the dependency on environment variables. Paths with spaces can still come in other ways, but not via SYSTEM_CC in a "normal" cm3, for my newfangled revisionist version of "normal". :) Short names really stink, in so many ways.First, they aren't necessary even shorter, they are just of a limited character set.They can be up to twice as long as "long".Try this:  mkdir \1.1.1.  dir /x 1* Also, wildcard matching "always" (in FindFirstFile/FindNextFile) operates against both long and short, and since short are unpredcitable, wildcard matching therefore is. An example, probably, is: echo > foo.barecho > food.barfdir *.bar Will probably, but not necessarily, match both files, though you would only expect one match. As well, I don't believe short file names have always been backed up and restored -- for anyone that uses any backup softwre (not me!). So you determine your short name, you backup and restore, and it changes. Or more clearly, recreating files, might get a different name.It's a mess. I would discourage you from persisting short names ever.Names with spaces I discourage but not as strong.For example, in some contexts, there is just one name and there is no ambiguity.Actually in %PATH% there is no ambiguity because the sepArator is the semicolon.I think Win9x different though, sigh. I agree from my "min" distribution, you need to build the rest.I have to admit, I thought the "culture" on this list was to want to and actually build the system from source.I will put a "std" distribution up on birch this week hopefully and hopefully someone will put it on the web page. Maintaining the installer should not be a problem.As long as the licensing and cost and all is right. I have not fixed the build yet, sorry.  - Jay


Date: Wed, 16 Jan 2008 11:24:34 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.com; jayk123 at hotmail.comSubject: Re: [M3devel] my status on win32
Jay:
 
Not sure I've clued in to all of your dialog.  Sorry.
 
I have in the past used a free tool to build an installer package and I was thinking of using it again.  Since the tool is free, we could put all the work I do into the CVS repository so that others could also maintain it.
 
What I am looking to achieve is to have a way where a "normal" Windows user could download a program, run it, and have all of the cm3 packages installed without having to use the MIN distribution to build it up or any other tools to get the sources (like cvs).  You could achieve the same thing by having a big ZIP file also I suppose, but having the installer would also let us easily put something in the start menu to launch a command line with all the environment variables set properly via automatic call to vcvars et al.  When I get "Reactor" working, it too could be put into the start menu.  That just makes it fit in better with the Windows user culture I think.
 
The reason I wanted to rebuild everything is two-fold:  
1) ensure it can be done (i.e., test for "Broken #1"), and 
2) get all of the rest of the packages built so I could then use cm3 to build "reactor" and my own programs and test them to see if there are any "Broken #2" issues.  The MIN distribution is really insufficient to do much of anything except to build the rest of the packages you need.  My understanding is that I needed to use CVS to get the current sources, because if I went back to the source archive, it was old and known to be Broken #1 on Windows.
 
I'm glad that you believe the current "Broken #1" situation is easy to fix.  Let me know what to do and I'll give it another try.
 
My experience in the past has been that there are 3 conditions to getting cm3 to work properly on windows after it has been installed:
1) make sure your environment variables are set up by vcvars to point to the compiler/linker/libraries (of course if you used a non-Microsoft C compiler/linker you would need to use something other than vcvars);
2) make sure cm3.cfg has the right paths in place to find all the libraries and tools using DOS style short names (no embedded spaces); and
3) make sure C:\cm3\bin is in your path.
Of these, #2 & #3 are essentially one-time events, and #1 can be set up by a invoking a batch/command file via a shortcut, so no big deal for me.  You could even make #1 a one-time event by modifying the system-level (vice user-level) environment variables.  The old cm3install program seemed to do a decent job of #2 for me.
 
Also, I've noted some of your past dialog about accepting either '/' or '\' as a path separator.  I'm sure you are aware that on Windows '/' is used often for switches (arguments).  Plus Windows also allows spaces to be embedded in the command line, so whatever solution you propose needs to take into account that a valid command line might look like this:
C:\My Program Folder\My Program.exe /myArg1 /myArg2
In some situations, this command line would have to be quoted and rendered as:

"C:\My Program Folder\My Program.exe" /myArg1 /myArg2
and in places where the backslash is an escape character, I've seen "C:\\My Program Folder\\My Program.exe" /myArg1 /myArg2
 
Alas, I seem to have rambled a bit also.  Sorry.  
The immediate need for me now is to solve the Broken #1 condition, so please let me know how to fix it and proceed.
 
Regards,
Randy>>> Jay <jayk123 at hotmail.com> 1/16/2008 10:28 AM >>>Randy, yes and no and clarifications. I mention starting with 5.1.6 as a "worst" or "worse" case.The earlier you start, the harder things are -- for me, to ensure it will work. If you merely start with my 5.5.0, well, what are you rebuilding from source for?Heck, you can use any binary distribution, 4.1, 5.1.6, and just use it. They should work. The n steps you/I give are really actually geared for people who WANT to build from source, it is not what people HAVE to do.As well there have been can be source distributions, to cut out CVS. I'll look into the datefn stuff shortly. As to the installer, well, again, yes and no.The need for the installer is very small. All you have to do today is unzip the .zip somewhere and set the path, and possibly the vcvars thing. That's it. I should..now that you are "pressuring" me for ease of use, teach cm3 about the various Þvenvdir%, %vcinstalldir% variables, etc. That way as long as the user installed the tools, even if they elected not to change path/lib/include, cm3 can find the compiler and linker. I know everyone wants to preserve the flexibility of quake/cm3.cfg but...besides, cm3 can be pretty darn smart here. IF cm3.cfg defines SYSTEM_CC, leave things along (NT386 cm3.cfg does not today do that). However if there is now SYSTEM_CC and there is no cl.exe/link.exe in %PATH%, it could go on a quick hunt for them. On the lib and include part, well, the dependency on %include% might only be for errno..maybe not worth it. It has been whatever it is forever, like int* _errno(void); #define errno (*_errno)()). Modula-3 engages in rampant header cloning and CErrnoC was only introduced where that was deemed more fragile than usual, and for Windows it probably wasn't actually fragile -- the fragility, if I may be so rudely judgemental, is because so many other systems are so Johnny-come-lately to arriving at the obviously correct threading model that Windows has had all along (since Win95/NT, can't say anything good about Win3.1; I think OS/2 might have had it correct all along too.). Funny, everyone has FINALLY arrived at the correct threading model, only to have start changing again, since the number of cores and machines is set to be much greater than anticipated, the obvious threading model has been deemed too difficult for anyone to use (Win32, Modula-3, Java, .NET, Python, all the same supposedly too difficult model btw), and so some new one is needed, and well, truthfully, the difficulty is not just for dummies on relatively small multiproc boxes not having simple race conditions, the problem is scaling way way up. Sorry that was my usual tangential. Even the unzip and set path would benefit slightly from a GUI installer.I believe there is some easy thing you can do where you write your GUI installer and you put it in the .zip and then you prepend the GUI self extracting .zip/.exe stub, and you can tell it to run a particular command after the extraction. I guess you might use a fancier thing though?That is, are you going to, say, write a gui installer from scratch in Modula-3? It might be a fun exercise. :)And then you could build the self extracting prefix from source etc.This is the hard way I guess, but more interesing.If you can build an msi or use InstallShield or Wix or something, go ahead.If you use widely availabe free as in beer tools, that can be rolled into whatever automation there is to produce distributions, keeping it up to date shouldn't be a problem. I do have to bring up some annoying questions: 1) Does anyone besides me have the capacity to produce distributions? i.e.: Any Win32 machines left at Elego? 2) I know I ramble. So one of my many tangents/questions went unanswered.Say, in the absence of a gui installer, should .zips/.tar.bz files look like cm3-min-<version>/bin/cm3orbin/cm3orcm3/bin/cm3 The last one is more directly usable. The first one is very common. This reminds me though Randy, the distribution I made available was "min", so yeah, like if you are doing any gui programming, using any "standard" library besides m3core or libm3, you'd need to build more from source. I can readily provide you a "std" distribution though.It's "always" been a "problem" to make my files available -- no web site hosting for me -- but recently that wasn't a problem, someone picked the files out of ~jkrell on birch and put them on the web site. Good. One more thing -- Olaf, please confirm repeat yourself, 'cause I admit I wasn't sure it was the right thing, but if you insist, ok. "std" should be "all"? I should not continue to imagine there is a new larger group called "all"? And wherever things are missing from std but do build ok, add them?m3cc should be in std?(I admit some big indecisiveness around m3cc. It is far and away the longest thing to build and so the one you really want to skip the most and leave out of whatever package group. I also wonder if on Windows, there should be some check where if sh is not in the $PATH, just silently skip m3cc? You only need MingWin to build what I have on my system for NT386GNU, except you also need msys just for m3cc. msys is actually quite small, like only 20 files..) Oh, sorry, yet another question.The non-gcc backend is excluded from the build on systems that use the gcc backend.For minimization, that is appropriate.For maximizing cross build capability, it is wrong.The packages build fine. I suspect they work.I am not a user of cross build capability yet, so I don't really care. (What I do for NT386/NT386GNU /might/ be considered sort of a cross build, but it is different/easier).They are small packages.  - Jay


Date: Wed, 16 Jan 2008 08:25:49 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.com; jayk123 at hotmail.comSubject: Re: [M3devel] my status on win32
Jay:
 
There is no line in the m3makefile dealing with datefn, so I added the whole statement just before the line 
include("version.quake")
 
Now, I get an error saying that the variable datefn is not defined.
 
If you change to put quotes around datefn in the include statement, you get an error saying it can't open the file.
 
You made a reference in one of your earlier posts about trying 5.1.6 or 4.1, but note that I started with your cm3-min-WIN32-NT386-d5.5.0.zip , not the 5.1.6 or 4.1 versions.  Thus, using d5.5.0 as a base, I am trying to run your "upgrade.cmd" to rebuild the sources I checked out from CVS.  Since this doesn't work "out-of-the-box", again I say this fits into the "Broken #1" category.
 
If I can get this to work, I don't mind putting together a Windows installer program that will install the whole thing.  That way, folks won't have to install cygwin just to get CVS so that they can install a working cm3.  Folks will need to install the free Microsoft Visual C as a prerequisite, but then that's not too bad.  Having an installer program would make cm3 more friendly to Windows folks.  The current 16-step method is too laborious and error-prone for the person just getting started and they will not give cm3 a second try.  
 
As the cm3 system evolves and new releases are made, we will need to keep the installer program up-to-date and I don't mind doing that.  After someone gets familiar with cm3 they might want to delve into some of the scripts for rebuilding the system, but until then, I think the installer program is the way to go.
 
Regards,
Randy>>> Jay <jayk123 at hotmail.com> 1/16/2008 1:01 AM >>>I think I see the problem. in cm3\src\m3makefile, make this change > if not defined("NOW")     include(datefn) > end datefn is only created if certain other variables aren't defined, and do-pkg defines them.I don't know why I don't see this. Later.Could be that the Python scripts have a typo and don't define them.I've been using them more than cmd. If the host is NT, "NOW" is either gotten from an updated cm3.exe, or left to "not available".  So the initial cm3 built from older cm3 can't report when it was built, but cm3 built from current cm3 can. I need to change that:  1) to be a function for niceness  2) to make it that way for Posix, for any host but I haven't been using Posix as much lately so not testing it.  - Jay


Date: Tue, 15 Jan 2008 23:55:39 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.com; jayk123 at hotmail.comSubject: RE: my status on win32
Jay:
 
In the file version.quake, I made the following edit:
 
changed line:  local datefn = "../" & TARGET & ".datenow"
to:  local datefn = ".." & SL & TARGET & ".datenow"
 
This change has the effect of putting a backward slash in the pathname instead of a forward slash, so we getC:\CM3_CVS_SourceTree\m3-sys\cm3\src\..\NT386.datenow
 
So from that perspective, the change seems good.
 
Unfortunately, I still get the error that this file can't be opened.
 
A quick check of C:\CM3_CVS_SourceTree\m3-sys\cm3\NT386 shows that the file .datenow does not exist.
 
Regards,
Randy>>> Jay <jayk123 at hotmail.com> 1/15/2008 11:03 PM >>>That's not too bad really.  1) I'd recommend putting cm3 at the start of the path instead of the end.  2) Just comment out the offending code to make progress. It's not critical.  >> C:\CM3_CVS_SourceTree\m3-sys\cm3\src\../NT386.datenow The code is: C:\CM3_CVS_SourceTree\m3-sys\cm3\src\version.quake or such, included by cm3\src\m3makefile. Does the file exist?Does version.quake use "/" or "SL"? If it uses "/", try replacing with SL (and ampersand for string concat). Could be that a newer binary distributions works better with forward slashes.   > 4.   > 5. 
  > 6.  Launch cygwin from desktop icon.  
 
Cygwin is only needed for cvs, I think you realize.
 
Could you send me offline the result of just running "set" after running vcvarsall?
I'd like to adapt pylib.py to it maybe.
 
(I can start from 5.1.6 and will try 4.1 at some point.)
  - Jay


Date: Tue, 15 Jan 2008 22:52:59 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.com; jayk123 at hotmail.comSubject: my status on win32
Jay et al:
 
I've listed below the steps I've undertaken to try and build the current sources on Windows XP.  
 
I've also attached a text file showing the output I got when trying to build everything using Jay's upgrade.cmd script.
 
Unfortunately, I'm getting a build error (see below).
 
Please advise on how to resolve.
 
STEPS I TOOK:
=========
1.  Download Microsoft Visual Studio 2008 Express Edition All-in-One .ISO file and burn to DVD.    http://www.microsoft.com/express/download/
 
2.  Install Microsoft Visual C++ 2008 Express Edition from DVD.
 
3.  Use "Microsoft Update" service to check for updates / service packs.
 
4.  Download cygwin setup program from     http://cygwin.com/
 
5.  Ran cygwin setup.exe program to install cygwin for all users.      Under "Devel" category, make certain to select "cvs" for installation.
 
6.  Launch cygwin from desktop icon.
 
7.  In cygwin command shell window, execute following two commands, making sure to give email as password when prompted for login:    cvs -d :pserver:anonymous at modula3.elegosoft.com:/usr/cvs login    cvs -d :pserver:anonymous at modula3.elegosoft.com:/usr/cvs checkout cm3
 
8.  Moved resulting C:\cygwin\home\rcoleburn\cm3 to C:\CM3_CVS_SourceTree    Note that you should replace "rcoleburn" above with your Windows login username.
 
9.  Download cm3-min-WIN32-NT386-d5.5.0.zip and cm3-min-WIN32-NT386-d5.5.0-symbols.zip from     http://modula3.elegosoft.com/cm3/download.html
 
10. Unzipped cm3-min-WIN32-NT386-d5.5.0.zip and stored resulting cm3 folder at C:\cm3
 
11. Unzipped cm3-min-WIN32-NT386-d5.5.0-symbols.zip and stored resulting symbols folder at C:\cm3\symbols
 
12. Launch Windows Command Prompt shell.  The following steps represent commands executed within this shell.
 
13. path %path%;c:\cm3\bin
 
14. "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"
 
15. cd C:\CM3_CVS_SourceTree\scripts\win
 
16. upgrade.cmd
 
ERROR I'M GETTING:
=============
 
=== package C:\CM3_CVS_SourceTree\m3-sys\cm3 ===+++ "cm3 -build  -DROOT=C:\\CM3_CVS_SourceTree -DCM3_VERSION_TEXT=d5.5.1 -DCM3_VERSION_NUMBER=050501 -DCM3_LAST_CHANGED=2007-12-30 && cm3 -ship -DROOT=C:\\CM3_CVS_SourceTree -DCM3_VERSION_TEXT=d5.5.1 -DCM3_VERSION_NUMBER=050501 -DCM3_LAST_CHANGED=2007-12-30" +++--- building in NT386 --- 
 
ignoring ..\src\m3overrides
 
"C:\CM3_CVS_SourceTree\m3-sys\cm3\src\version.quake", line 136: quake runtime error: unable to open "C:\CM3_CVS_SourceTree\m3-sys\cm3\src\../NT386.datenow" forreading
 
--procedure--  -line-  -file---include            --  <builtin>version_impl      136  C:\CM3_CVS_SourceTree\m3-sys\cm3\src\version.quakeinclude_dir        20  C:\CM3_CVS_SourceTree\m3-sys\cm3\src\m3makefile                    8  C:\CM3_CVS_SourceTree\m3-sys\cm3\NT386\m3make.args
 
Fatal Error: package build failedERROR: "cm3 -build  -DROOT=C:\\CM3_CVS_SourceTree -DCM3_VERSION_TEXT=d5.5.1 -DCM3_VERSION_NUMBER=050501 -DCM3_LAST_CHANGED=2007-12-30 && cm3 -ship -DROOT=C:\\CM3_CVS_SourceTree -DCM3_VERSION_TEXT=d5.5.1 -DCM3_VERSION_NUMBER=050501 -DCM3_LAST_CHANGED=2007-12-30"ERROR: cd C:\CM3_CVS_SourceTree\m3-sys\cm3ERROR: set INSTALLROOT=c:\cm3
 
Regards,
Randy
 
 

Share life as it happens with the new Windows Live. Start sharing! 

Put your friends on the big screen with Windows Vista® + Windows LiveT. Start now! 

Get the power of Windows + Web with the new Windows Live. Get it now! 

Make distant family not so distant with Windows Vista® + Windows LiveT. Start now! 
_________________________________________________________________
Put your friends on the big screen with Windows Vista® + Windows Live™.
http://www.microsoft.com/windows/shop/specialoffers.mspx?ocid=TXT_TAGLM_CPC_MediaCtr_bigscreen_012008
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20080116/1335cfe8/attachment-0002.html>