[M3devel] Extending the tests, was: Re: M3Path.FixPath rewrite?

[Jay]

I can't argue adding tests per change is a good idea.
I was on a team where at least every bug fix had to come with an automatic regression test, if not every initial change, though of course, you could do the second. My problem historically has been lack of a framework -- how to write tests, how to run them, how to verify them, how to debug them. m3-sys\m3tests appears to be a plenty adequate framework for locally authoring, running, verifying, debugging, plus the tinderbox for ongoing status (not that, as far as I know, the tinderbox is debuggable in-situ, but still, this is plenty).
I submitted the change but disabled. It's now a very small diff to enable
 
What is the story on exposing internals to testing though?
I see several options, all with pluses and minus:
 
Expose more stuff in the .i3 file.
Possibly with names like "Internal" or "Test" in them.
Not great but easy and works.
 
Invoke test code from module initialization having checked Params for M3testfoo or such.
Not great but easy and often works.
Adds unnecessary code to the runtime.
 
Getting better I think:
  Add FooInternal.i3 or FooTest.i3.
  Export it from Foo.m3. I guess even to the pkg store?
  Call it from the text executable (also want to call FooTest.m3).
  This still exposes internals to callers, but at least with a clear distinction of what is internal or not.
  I guess another name here is FooFriend.i3.
 
For that matter, split off the testable code into FooTest.m3. Same thing?
Somehow "preprocess" the code to extract parts you want to call from test code.
I'm not keen on designing a preprocessor.
 
Aha, combine the last few?
  Split up Foo.m3 into FooTest.m3 and Foo.m3. Both exports only parts of Foo.i3, which is unchanged.
  Add FooTest.i3 that is empty?
  "Preprocessing" becomes a) copy an .m3 file b) replace an .i3 file (FooTest.i3)? There, I designed preprocessing, as file copying.
 
There's also the matter of chosing the test cases that matter and not just a bunch of random strings that are obviously handled correctly but I guess just err toward throwing in more strings.
 
 - Jay



> Date: Tue, 26 Feb 2008 11:36:07 +0100> From: wagner at elegosoft.com> To: m3devel at elegosoft.com> Subject: [M3devel] Extending the tests, was: Re: M3Path.FixPath rewrite?> > Quoting Jay <jayk123 at hotmail.com>:> > > M3Path.FixPath had some flaws, mainly: it uses a fixed size array > > to store the locations of path separators in the presence of too > > many separators, it'd ignore stuff every occurence of . or .. > > causes it to restart its work (after removing it)> > I have written a new version with these aspects fixed.> > Anyone care to try it out?> > diff and m3path.m3 attached.> > There's a small test harness built in, disabled.> > You can feed strings through the old and new and compare.> > They don't always match. I think my results are "more correct".> > Sorry, it will take some time till I can test these.> > Why don't you add the test to the package test framework> in m3-sys/cm3/tests/src/TestM3Path or similar?> Provide a list of input paths and expected output in files;> this will make things easier to understand for others.> If you add the tests first and wait one day until checking in> your change we can see the impacts in the package test results> in our tinderbox.> > Generally I think we should start adding tests for everything we> change; there's so much breakage that can be avoided this way.> > Olaf> -- > Olaf Wagner -- elego Software Solutions GmbH> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194> 
_________________________________________________________________
Helping your favorite cause is as easy as instant messaging. You IM, we give.
http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20080226/521d730b/attachment-0002.html>