Content-Type: |
text/plain; charset=ISO-8859-15; format=flowed |
Date: |
Tue, 27 Sep 2011 22:48:28 +0200 |
Reply-To: |
|
Subject: |
|
MIME-Version: |
1.0 |
Message-ID: |
|
Content-Transfer-Encoding: |
7bit |
Sender: |
|
From: |
|
Parts/Attachments: |
|
|
One of the recent bug reports reminded me that I wanted to ask for
volunteers to help making our regression test suite better.
As you may know for 2e we implemented a fairly extensive test suite of
files that can be run automatically and that would identify sudden
changes in the interfaces that may happen due to dependencies between
functions that may not be apparent on the surface.
Over time this suite has helped to keep the number of serious blunders
down to a minimum (though nevertheless we managed to make them).
It also helped very much in the transition from 2.09 to 2e as we
"certified" some results initially on 2.09 and could then see if 2e
would still give the same result.
The basic idea is to write test files in a certain format and store away
the log file (after some manipulation, i.e., removing stuff that would
change on rerun, or is dependent on the distribution).
One important aspect is that the test should show "results" in the log,
but it should not show a lot of implementation details if that can be
helped, e.g. a \tracingall would not be a good output as any change in
the inner implementation would result in a diff.
Once such a pair is inspected and the output is declared to be correct
it is added to the suite to be automatically run from thereafter and the
result compared with the frozen test log.
For expl3 we also started to produce such tests, but test coverage is
far from perfect.
So what we are looking for are individuals that would interested to help
here. It needs some devious mindset (writing good tests is certainly an
art :-) you need to get in to a state of mind where you think about,
what are the boundaries of the functionality, where the developer might
have messed up, as well as seriously thinking of how to test the main
functionality that you do not want to see changing.
For those interested to help, you would need a development version of
expl3, either form svn or from the github, where you can find testfiles
for example, e.g.:
https://github.com/latex3/svn-mirror/tree/master/l3kernel/testfiles
As I said help here would be really useful to stabilize the code
further. For example l3fp is probably being reimplemented fairly soon.
If so having a comprehensive test set would ensure that the
reimplementation is not suddenly changing the interfaces or contains
mistakes.
So if you are interested talk to us so that we can help you to get
comfortable with it
thanks in advance
frank
|
|
|