[Dune] distributed dune testing environment

[Andreas Dedner]

Hi Christian,
thanks for having a look.
Here is some more explanation:

default behavior is that all core modules (of a given version)
are tested, meaning:
first try to build a distribution (without options)
    if successful, take the distribution and run
      a) build
      b) make headercheck
      c) make check
    if not:
      a) build the svn
      b) make headercheck
      c) make check
      d) make dist
      e) configure the dists (that worked)
      f) make check
      g) make headercheck
In this sense the testing behavior is well defined and corresponds to
the entries in the table - where you can click on any entry and look
at the output from dunecontrol separately for each module.
So I think it is very easy to figure out what went wrong.

The options file for dunecontrol on the other hand (and the unix system,
compiler...) are not specified by the test.

I hope that makes some things clearer
Andreas


Christian Engwer wrote:
> Hi Andreas,
> 
> I had a look at the website. If I get it correctly the important point
> is that there are no prescribed test-sets, but every database entry
> defines its own list of modules, options, etc. In that sense it is
> truely more decentralized than the autobuild code. And I think it
> serves a different purpose. I like both approaches. My impression is
> that your approach is very good for high-level tests (run your
> application and check that the computations are valid etc.) while it
> is hard to do low-level tests (you just see failure but have no idea
> where the failure occured). This is where autobuild is strong, it
> allows to inspect the different tests on a fine-grained level.
> 
> I think for the time being we should try to have both systems
> running. On the long run we could discuss how we can integrate the two
> approaches.
> 
> Cheers Christian
> 
> 
> On Fri, Nov 20, 2009 at 12:22:04PM +0100, Andreas Dedner wrote:
>> Hi,
>>
>> since testing is also a topic of the upcoming dune meeting, I wanted to
>> present a testing environment, we have been working on for the last two
>> weeks. I think it follows a slightly different approach from the
>> original testing system, so that it can be seen as an addition or as a
>> replacement, we can decide that at the meeting. We in Freiburg will at
>> least use this approach for our internal testing but I think it has
>> potential as a general testing system for dune.
>>
>> The idea is to have a decentralized system where everybody can
>> contribute in maintaining the stability of dune with little work for
>> everybody. A link to the page with the test results is
>>      http://dune.mathematik.uni-freiburg.de/testing
>> At the end of that page there are some remarks on how it is supposed to
>> work. Basically you register as a tester by sending us an email and get
>> an account. Then you run a testing script on your system which performs
>> the tests and transmits the results to us. The hope is that in this way
>> we can at least cover the configuration needed by the dune users.
>>
>> It would be good if a few people ( please not all at once :-) )
>> would have a look at the system which means they have to send us an
>> email for an account.
>>
>> Best
>> Andreas
>>
>> _______________________________________________
>> Dune mailing list
>> Dune at dune-project.org
>> http://lists.dune-project.org/mailman/listinfo/dune
>>