[M3devel] my status on win32
Randy Coleburn
rcoleburn at scires.com
Wed Jan 16 21:55:51 CET 2008
Jay:
Thanks for your reply. I think most of your goals seem reasonable and
I concur that there are nasty problems introduced by spaces and short
names. It's a messy situation that neither you nor I created. I was
just relaying some of my prior experience in getting things to work.
Indeed, for me, I try to avoid spaces always. That's one reason I root
my cm3 at "C:\cm3" instead of in "C:\Program Files\cm3" or "C:\Documents
and Settings\userhome\cm3". I've noted that with the various Visual
Studio and Visual C tools going back for some time now that after you
install it, your environment vars for TEMP and TMP get changed to use
short names without spaces, so I guess even Microsoft has trouble with
spaces.
I also recall in the past with cm3 having trouble if my TEMP and TMP
vars included spaces, so having Microsoft switch them to use short names
without spaces seemed to solve that cm3 problem at least.
As for the VC installation, I too recall that on some prior versions it
asked you whether you wanted to set the environment vars permanently or
to use vcvars. This time, when I installed Visual Studio 2008, it did
not ask this question, so I guess it is version dependant. I'm not at
the machine right now where I did the install, but later tonight I'll
check to see if the VS2008 install set any permanent environment vars.
Again, for me, I would like to be able to build everything as the first
step in my series of tests. As soon as you can give me a fix to make
things build, I can move on to make more progress.
Thanks for your help.
Regards,
Randy
>>> Jay <jayk123 at hotmail.com> 1/16/2008 2:13 PM >>>
Hi Randy. I mostly agree and understand.
Go ahead with your installer work. :)
#2 is no longer needed. The cm3.cfg I use, that is checked in, that is
in the newer distributions, just runs plain "cl" and links to plain
"kernel32.dll", and doesn't pass any /I switches to cl => therefore it
depends on %PATH%, %LIB%, and %INCLUDE%.
As I was saying/rambling, we/I can probably teach cm3 a bit so as to
not need vcvars.
As I recall, but I need to check, when you install VC, there are some
variables it sets unconditionally, and some it asks the user if they
should be set. The ones it asks about are PATH, LIB, INCLUDE. The ones
it sets unconditionally are the "root" of the install. PATH / LIB /
INCLUDE are more invasive, more impactful, have more potential to break
things. They are also easily derived from the "root".
Similarly, the CM3 installer could ask the user if it is ok to modify
their %PATH%, and do it in the registry and set the WM_INI_CHANGED or
such message so that Explorer picks it up. Need to check MSDN on this.
Forward slashes are fine for options.
I take a dim view of these space, quoting, escaping issues.
I have very mixed experience here. In SOME situations, your command
lines pass through various layers of code, with various attempts to
handle or ignore the various issues. The result is in my opinion an
unpredictable mess.
The problem is that people have confused "an array of strings" with
"one flat string". At this point the confusion is widespread and deeply
ingrained. On Unix I believe people just ignore the issue by not using
spaces (or quotes?) on the command line, though escaping issues still
remain for other characters.
You just have to read the documenation on CreateProcess to realize how
messed up this is.
c:\program files\foo
I believe is interpreted multiple ways until something works.
It can be running c:\program with the parameter files\foo, or it can be
running c:\program files\foo.
Reliable interprocess communication should not be via command lines but
via something more strongly typed such as RPC or shared memory or
reading/writing a Pickle.
I use the command line a lot, and I frequently copy paths from the
command line and paste them into the file open dialog, which does not
accept forward slashes nor nested double slashes. It should but oh well.
Therefore I have goal before much longer to ensure that every path
generated and probably reported by cm3 is in the "normal" form. If the
user types in a forward slash, perhaps that should be reported back
though. And forward slashes should be ok to enter and then get used
internally.
The need to configure short file names are gone, along with the need to
quote them, by virtue of the dependency on environment variables. Paths
with spaces can still come in other ways, but not via SYSTEM_CC in a
"normal" cm3, for my newfangled revisionist version of "normal". :)
Short names really stink, in so many ways.
First, they aren't necessary even shorter, they are just of a limited
character set.
They can be up to twice as long as "long".
Try this:
mkdir \1.1.1.
dir /x 1*
Also, wildcard matching "always" (in FindFirstFile/FindNextFile)
operates against both long and short, and since short are unpredcitable,
wildcard matching therefore is. An example, probably, is:
echo > foo.bar
echo > food.barf
dir *.bar
Will probably, but not necessarily, match both files, though you would
only expect one match.
As well, I don't believe short file names have always been backed up
and restored -- for anyone that uses any backup softwre (not me!). So
you determine your short name, you backup and restore, and it changes.
Or more clearly, recreating files, might get a different name.
It's a mess.
I would discourage you from persisting short names ever.
Names with spaces I discourage but not as strong.
For example, in some contexts, there is just one name and there is no
ambiguity.
Actually in %PATH% there is no ambiguity because the sepArator is the
semicolon.
I think Win9x different though, sigh.
I agree from my "min" distribution, you need to build the rest.
I have to admit, I thought the "culture" on this list was to want to
and actually build the system from source.
I will put a "std" distribution up on birch this week hopefully and
hopefully someone will put it on the web page.
Maintaining the installer should not be a problem.
As long as the licensing and cost and all is right.
I have not fixed the build yet, sorry.
- Jay
Date: Wed, 16 Jan 2008 11:24:34 -0500
From: rcoleburn at scires.com
To: m3devel at elegosoft.com; jayk123 at hotmail.com
Subject: Re: [M3devel] my status on win32
Jay:
Not sure I've clued in to all of your dialog. Sorry.
I have in the past used a free tool to build an installer package and I
was thinking of using it again. Since the tool is free, we could put
all the work I do into the CVS repository so that others could also
maintain it.
What I am looking to achieve is to have a way where a "normal" Windows
user could download a program, run it, and have all of the cm3 packages
installed without having to use the MIN distribution to build it up or
any other tools to get the sources (like cvs). You could achieve the
same thing by having a big ZIP file also I suppose, but having the
installer would also let us easily put something in the start menu to
launch a command line with all the environment variables set properly
via automatic call to vcvars et al. When I get "Reactor" working, it
too could be put into the start menu. That just makes it fit in better
with the Windows user culture I think.
The reason I wanted to rebuild everything is two-fold:
1) ensure it can be done (i.e., test for "Broken #1"), and
2) get all of the rest of the packages built so I could then use cm3 to
build "reactor" and my own programs and test them to see if there are
any "Broken #2" issues. The MIN distribution is really insufficient to
do much of anything except to build the rest of the packages you need.
My understanding is that I needed to use CVS to get the current sources,
because if I went back to the source archive, it was old and known to be
Broken #1 on Windows.
I'm glad that you believe the current "Broken #1" situation is easy to
fix. Let me know what to do and I'll give it another try.
My experience in the past has been that there are 3 conditions to
getting cm3 to work properly on windows after it has been installed:
1) make sure your environment variables are set up by vcvars to point
to the compiler/linker/libraries (of course if you used a non-Microsoft
C compiler/linker you would need to use something other than vcvars);
2) make sure cm3.cfg has the right paths in place to find all the
libraries and tools using DOS style short names (no embedded spaces);
and
3) make sure C:\cm3\bin is in your path.
Of these, #2 & #3 are essentially one-time events, and #1 can be set up
by a invoking a batch/command file via a shortcut, so no big deal for
me. You could even make #1 a one-time event by modifying the
system-level (vice user-level) environment variables. The old
cm3install program seemed to do a decent job of #2 for me.
Also, I've noted some of your past dialog about accepting either '/' or
'\' as a path separator. I'm sure you are aware that on Windows '/' is
used often for switches (arguments). Plus Windows also allows spaces to
be embedded in the command line, so whatever solution you propose needs
to take into account that a valid command line might look like this:
C:\My Program Folder\My Program.exe /myArg1 /myArg2
In some situations, this command line would have to be quoted and
rendered as:
"C:\My Program Folder\My Program.exe" /myArg1 /myArg2
and in places where the backslash is an escape character, I've seen
"C:\\My Program Folder\\My Program.exe" /myArg1 /myArg2
Alas, I seem to have rambled a bit also. Sorry.
The immediate need for me now is to solve the Broken #1 condition, so
please let me know how to fix it and proceed.
Regards,
Randy
>>> Jay <jayk123 at hotmail.com> 1/16/2008 10:28 AM >>>
Randy, yes and no and clarifications.
I mention starting with 5.1.6 as a "worst" or "worse" case.
The earlier you start, the harder things are -- for me, to ensure it
will work.
If you merely start with my 5.5.0, well, what are you rebuilding from
source for?
Heck, you can use any binary distribution, 4.1, 5.1.6, and just use it.
They should work.
The n steps you/I give are really actually geared for people who WANT
to build from source, it is not what people HAVE to do.
As well there have been can be source distributions, to cut out CVS.
I'll look into the datefn stuff shortly.
As to the installer, well, again, yes and no.
The need for the installer is very small. All you have to do today is
unzip the .zip somewhere and set the path, and possibly the vcvars
thing. That's it. I should..now that you are "pressuring" me for ease of
use, teach cm3 about the various Þvenvdir%, %vcinstalldir% variables,
etc. That way as long as the user installed the tools, even if they
elected not to change path/lib/include, cm3 can find the compiler and
linker. I know everyone wants to preserve the flexibility of
quake/cm3.cfg but...besides, cm3 can be pretty darn smart here. IF
cm3.cfg defines SYSTEM_CC, leave things along (NT386 cm3.cfg does not
today do that). However if there is now SYSTEM_CC and there is no
cl.exe/link.exe in %PATH%, it could go on a quick hunt for them. On the
lib and include part, well, the dependency on %include% might only be
for errno..maybe not worth it. It has been whatever it is forever, like
int* _errno(void); #define errno (*_errno)()). Modula-3 engages in
rampant header cloning and CErrnoC was only introduced where that was
deemed more fragile than usual, and for Windows it probably wasn't
actually fragile -- the fragility, if I may be so rudely judgemental, is
because so many other systems are so Johnny-come-lately to arriving at
the obviously correct threading model that Windows has had all along
(since Win95/NT, can't say anything good about Win3.1; I think OS/2
might have had it correct all along too.). Funny, everyone has FINALLY
arrived at the correct threading model, only to have start changing
again, since the number of cores and machines is set to be much greater
than anticipated, the obvious threading model has been deemed too
difficult for anyone to use (Win32, Modula-3, Java, .NET, Python, all
the same supposedly too difficult model btw), and so some new one is
needed, and well, truthfully, the difficulty is not just for dummies on
relatively small multiproc boxes not having simple race conditions, the
problem is scaling way way up.
Sorry that was my usual tangential.
Even the unzip and set path would benefit slightly from a GUI
installer.
I believe there is some easy thing you can do where you write your GUI
installer and you put it in the .zip and then you prepend the GUI self
extracting .zip/.exe stub, and you can tell it to run a particular
command after the extraction.
I guess you might use a fancier thing though?
That is, are you going to, say, write a gui installer from scratch in
Modula-3? It might be a fun exercise. :)
And then you could build the self extracting prefix from source etc.
This is the hard way I guess, but more interesing.
If you can build an msi or use InstallShield or Wix or something, go
ahead.
If you use widely availabe free as in beer tools, that can be rolled
into whatever automation there is to produce distributions, keeping it
up to date shouldn't be a problem.
I do have to bring up some annoying questions:
1) Does anyone besides me have the capacity to produce distributions?
i.e.: Any Win32 machines left at Elego?
2) I know I ramble. So one of my many tangents/questions went
unanswered.
Say, in the absence of a gui installer, should .zips/.tar.bz files look
like
cm3-min-<version>/bin/cm3
or
bin/cm3
or
cm3/bin/cm3
The last one is more directly usable. The first one is very common.
This reminds me though Randy, the distribution I made available was
"min", so yeah, like if you are doing any gui programming, using any
"standard" library besides m3core or libm3, you'd need to build more
from source.
I can readily provide you a "std" distribution though.
It's "always" been a "problem" to make my files available -- no web
site hosting for me -- but recently that wasn't a problem, someone
picked the files out of ~jkrell on birch and put them on the web site.
Good.
One more thing -- Olaf, please confirm repeat yourself, 'cause I admit
I wasn't sure it was the right thing, but if you insist, ok. "std"
should be "all"? I should not continue to imagine there is a new larger
group called "all"? And wherever things are missing from std but do
build ok, add them?
m3cc should be in std?
(I admit some big indecisiveness around m3cc. It is far and away the
longest thing to build and so the one you really want to skip the most
and leave out of whatever package group. I also wonder if on Windows,
there should be some check where if sh is not in the $PATH, just
silently skip m3cc? You only need MingWin to build what I have on my
system for NT386GNU, except you also need msys just for m3cc. msys is
actually quite small, like only 20 files..)
Oh, sorry, yet another question.
The non-gcc backend is excluded from the build on systems that use the
gcc backend.
For minimization, that is appropriate.
For maximizing cross build capability, it is wrong.
The packages build fine. I suspect they work.
I am not a user of cross build capability yet, so I don't really care.
(What I do for NT386/NT386GNU /might/ be considered sort of a cross
build, but it is different/easier).
They are small packages.
- Jay
Date: Wed, 16 Jan 2008 08:25:49 -0500
From: rcoleburn at scires.com
To: m3devel at elegosoft.com; jayk123 at hotmail.com
Subject: Re: [M3devel] my status on win32
Jay:
There is no line in the m3makefile dealing with datefn, so I added the
whole statement just before the line
include("version.quake")
Now, I get an error saying that the variable datefn is not defined.
If you change to put quotes around datefn in the include statement, you
get an error saying it can't open the file.
You made a reference in one of your earlier posts about trying 5.1.6 or
4.1, but note that I started with your cm3-min-WIN32-NT386-d5.5.0.zip ,
not the 5.1.6 or 4.1 versions. Thus, using d5.5.0 as a base, I am
trying to run your "upgrade.cmd" to rebuild the sources I checked out
from CVS. Since this doesn't work "out-of-the-box", again I say this
fits into the "Broken #1" category.
If I can get this to work, I don't mind putting together a Windows
installer program that will install the whole thing. That way, folks
won't have to install cygwin just to get CVS so that they can install a
working cm3. Folks will need to install the free Microsoft Visual C as
a prerequisite, but then that's not too bad. Having an installer
program would make cm3 more friendly to Windows folks. The current
16-step method is too laborious and error-prone for the person just
getting started and they will not give cm3 a second try.
As the cm3 system evolves and new releases are made, we will need to
keep the installer program up-to-date and I don't mind doing that.
After someone gets familiar with cm3 they might want to delve into some
of the scripts for rebuilding the system, but until then, I think the
installer program is the way to go.
Regards,
Randy
>>> Jay <jayk123 at hotmail.com> 1/16/2008 1:01 AM >>>
I think I see the problem.
in cm3\src\m3makefile, make this change
> if not defined("NOW")
include(datefn)
> end
datefn is only created if certain other variables aren't defined, and
do-pkg defines them.
I don't know why I don't see this. Later.
Could be that the Python scripts have a typo and don't define them.
I've been using them more than cmd.
If the host is NT, "NOW" is either gotten from an updated cm3.exe, or
left to "not available".
So the initial cm3 built from older cm3 can't report when it was
built, but cm3 built from current cm3 can.
I need to change that:
1) to be a function for niceness
2) to make it that way for Posix, for any host
but I haven't been using Posix as much lately so not testing it.
- Jay
Date: Tue, 15 Jan 2008 23:55:39 -0500
From: rcoleburn at scires.com
To: m3devel at elegosoft.com; jayk123 at hotmail.com
Subject: RE: my status on win32
Jay:
In the file version.quake, I made the following edit:
changed line: local datefn = "../" & TARGET & ".datenow"
to: local datefn = ".." & SL & TARGET & ".datenow"
This change has the effect of putting a backward slash in the pathname
instead of a forward slash, so we get
C:\CM3_CVS_SourceTree\m3-sys\cm3\src\..\NT386.datenow
So from that perspective, the change seems good.
Unfortunately, I still get the error that this file can't be opened.
A quick check of C:\CM3_CVS_SourceTree\m3-sys\cm3\NT386 shows that the
file .datenow does not exist.
Regards,
Randy
>>> Jay <jayk123 at hotmail.com> 1/15/2008 11:03 PM >>>
That's not too bad really.
1) I'd recommend putting cm3 at the start of the path instead of the
end.
2) Just comment out the offending code to make progress. It's not
critical.
>> C:\CM3_CVS_SourceTree\m3-sys\cm3\src\../NT386.datenow
The code is:
C:\CM3_CVS_SourceTree\m3-sys\cm3\src\version.quake or such, included
by cm3\src\m3makefile.
Does the file exist?
Does version.quake use "/" or "SL"? If it uses "/", try replacing with
SL (and ampersand for string concat).
Could be that a newer binary distributions works better with forward
slashes.
> 4.
> 5.
> 6. Launch cygwin from desktop icon.
Cygwin is only needed for cvs, I think you realize.
Could you send me offline the result of just running "set" after
running vcvarsall?
I'd like to adapt pylib.py to it maybe.
(I can start from 5.1.6 and will try 4.1 at some point.)
- Jay
Date: Tue, 15 Jan 2008 22:52:59 -0500
From: rcoleburn at scires.com
To: m3devel at elegosoft.com; jayk123 at hotmail.com
Subject: my status on win32
Jay et al:
I've listed below the steps I've undertaken to try and build the
current sources on Windows XP.
I've also attached a text file showing the output I got when trying to
build everything using Jay's upgrade.cmd script.
Unfortunately, I'm getting a build error (see below).
Please advise on how to resolve.
STEPS I TOOK:
=========
1. Download Microsoft Visual Studio 2008 Express Edition All-in-One
.ISO file and burn to DVD.
http://www.microsoft.com/express/download/
2. Install Microsoft Visual C++ 2008 Express Edition from DVD.
3. Use "Microsoft Update" service to check for updates / service
packs.
4. Download cygwin setup program from
http://cygwin.com/
5. Ran cygwin setup.exe program to install cygwin for all users.
Under "Devel" category, make certain to select "cvs" for
installation.
6. Launch cygwin from desktop icon.
7. In cygwin command shell window, execute following two commands,
making sure to give email as password when prompted for login:
cvs -d :pserver:anonymous at modula3.elegosoft.com:/usr/cvs login
cvs -d :pserver:anonymous at modula3.elegosoft.com:/usr/cvs checkout
cm3
8. Moved resulting C:\cygwin\home\rcoleburn\cm3 to
C:\CM3_CVS_SourceTree
Note that you should replace "rcoleburn" above with your Windows
login username.
9. Download cm3-min-WIN32-NT386-d5.5.0.zip and
cm3-min-WIN32-NT386-d5.5.0-symbols.zip from
http://modula3.elegosoft.com/cm3/download.html
10. Unzipped cm3-min-WIN32-NT386-d5.5.0.zip and stored resulting cm3
folder at C:\cm3
11. Unzipped cm3-min-WIN32-NT386-d5.5.0-symbols.zip and stored
resulting symbols folder at C:\cm3\symbols
12. Launch Windows Command Prompt shell. The following steps represent
commands executed within this shell.
13. path %path%;c:\cm3\bin
14. "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"
15. cd C:\CM3_CVS_SourceTree\scripts\win
16. upgrade.cmd
ERROR I'M GETTING:
=============
=== package C:\CM3_CVS_SourceTree\m3-sys\cm3 ===
+++ "cm3 -build -DROOT=C:\\CM3_CVS_SourceTree
-DCM3_VERSION_TEXT=d5.5.1 -DCM3_V
ERSION_NUMBER=050501 -DCM3_LAST_CHANGED=2007-12-30 && cm3 -ship
-DROOT=C:\\CM3_C
VS_SourceTree -DCM3_VERSION_TEXT=d5.5.1 -DCM3_VERSION_NUMBER=050501
-DCM3_LAST_C
HANGED=2007-12-30" +++
--- building in NT386 ---
ignoring ..\src\m3overrides
"C:\CM3_CVS_SourceTree\m3-sys\cm3\src\version.quake", line 136: quake
runtime er
ror: unable to open
"C:\CM3_CVS_SourceTree\m3-sys\cm3\src\../NT386.datenow" for
reading
--procedure-- -line- -file---
include -- <builtin>
version_impl 136
C:\CM3_CVS_SourceTree\m3-sys\cm3\src\version.quake
include_dir 20 C:\CM3_CVS_SourceTree\m3-sys\cm3\src\m3makefile
8
C:\CM3_CVS_SourceTree\m3-sys\cm3\NT386\m3make.args
Fatal Error: package build failed
ERROR: "cm3 -build -DROOT=C:\\CM3_CVS_SourceTree
-DCM3_VERSION_TEXT=d5.5.1 -DCM
3_VERSION_NUMBER=050501 -DCM3_LAST_CHANGED=2007-12-30 && cm3 -ship
-DROOT=C:\\CM
3_CVS_SourceTree -DCM3_VERSION_TEXT=d5.5.1 -DCM3_VERSION_NUMBER=050501
-DCM3_LAS
T_CHANGED=2007-12-30"
ERROR: cd C:\CM3_CVS_SourceTree\m3-sys\cm3
ERROR: set INSTALLROOT=c:\cm3
Regards,
Randy
Share life as it happens with the new Windows Live. Start sharing! (
http://www.windowslive.com/share.html?ocid=TXT_TAGHM_Wave2_sharelife_012008
)
Put your friends on the big screen with Windows Vista* + Windows LiveT.
Start now! (
http://www.microsoft.com/windows/shop/specialoffers.mspx?ocid=TXT_TAGLM_CPC_MediaCtr_bigscreen_012008
)
Get the power of Windows + Web with the new Windows Live. Get it now! (
http://www.windowslive.com/?ocid=TXT_TAGHM_Wave2_powerofwindows_012008 )
Make distant family not so distant with Windows Vista* + Windows Live*.
Start now! (
http://www.microsoft.com/windows/digitallife/keepintouch.mspx?ocid=TXT_TAGLM_CPC_VideoChat_distantfamily_012008
)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20080116/a4853a11/attachment-0002.html>
More information about the M3devel
mailing list