[darcs-users] fwd: volunteers with always-on boxes sought for automated testing

[David Roundy]

On Wed, Mar 05, 2008 at 09:03:23AM +0900, Stephen J. Turnbull wrote:
> David Roundy writes:
> > There's also the factor that additional buildbots without additional folks
> > willing to help debug are likely to be more of a burden than a help.  It's
> > far more common to have portability bugs in our test scripts than in darcs
> > itself, and I'm not going to waste my time fixing bugs in test scripts when
> > I could be fixing bugs in darcs.  Or at least, I shouldn't be doing that.
> 
> The test scripts are part of darcs.  Thinking that they're not, just
> because they aren't executed by users in ordinary use, is a big
> mistake IMHO.

They're not.  Before we had test scripts, darcs worked and was useful.  The
test scripts are a useful debugging tool.  When they don't function as
such, they aren't useful, and I don't care about that.  There are plenty of
bugs in darcs itself that I'd rather fix than deal with some strange shell
on solaris that doesn't work like bash.

> If you're unwilling to support the test scripts on some platform,
> Possibily you should go all the way and declare that Darcs is not
> supported at all on that platform, even if it happens to work.

No, that's ridiculous.  Darcs is supported on any platform on which users
compile it and help us fix errors by reporting them.  Removing the human
element from darcs' development process is not a useful plan.  Nor is
reducing the human element to droundy a useful plan.  Platforms on which
noone cares to try out darcs are not platforms on which I care to port
darcs.

Buildbot is not a practical development platform for debugging unportable
shell scripts and (even worse) perl code.  If there is noone who owns a
particular platform and is willing to help debug darcs, then, yes, it's
reasonable that that platform receives substandard support by darcs.

> Haskell must have a unit test framework, and maybe even something like
> Python's doctests.  It might quickly repay the effort put into
> translating anything that depends on Unix shells into such frameworks.

It's not unit testing that we are talking about, but rather actual testing,
which is far more useful, and far more challenging.  We also have unit
testing, but there's no point running that on multiple platforms, because
it'll always give the same answer.

Why would translating our tests into a different language help us? We've
got tests in perl, and they're more portable and more comprehensive and far
less useful than the tests in shell.  And more challenging to maintain as
well.

David (who is not a superhero, and will not become one by the addition of
       even hundreds of buildbots)