[Dune-devel] [GSoC] Performance testing: more detailed schedule

[Christian Engwer]

Hi Miha,

> I've been thinking of using intermediate files and a wildcard target
> (pattern rule), so that you would add something like
> 
> perftest_myexample: myexample perftest_myexample.log
> 
> and there would be a general rule
> 
> %_perftest.log:
>   <run the test here>
> 
> If will try such a rule today, unless you have a better suggestion.

As I have written a while ago, I suggest to look at the code in
dune-common/am/checklog this is an automake snippet, which can be
included in the Makefiles and provides a new target "make
checklog". The list of tests etc. is available in different makefile
variables, so the new phony target can directly loop through these
lists and run the necessary compile operations and execute the tests.

> > It would be great, to have a short readme in doc, which explains, how
> > the tests interact with the buildsystem and what the user is supposed
> > to do.
> >

> Yes, of course. But first the interaction should work :)

If you can desccribe the current state for cmake, I can comment on how
this can be achieved with automake

> > Then I have some additional comments...
> > - all test should be performed in the build directory, this is then
> >   also the place where results should be stored.
> > - it would be convenient to have all results in a seperate
> >   subdirectory (of the build dir) and then the html files again in a
> >   sub dir.
> >
> I agree, will make it that way.
> 
> > - in order to be able to integrate the perftests with the autobuild
> >   services, it is necessary to seperate data aquisition and
> >   storage. Therefore it ise necessary to write some kind of log files
> >   which can then be transferred to the server (or is this already
> >   integrated?).
> >
> This is somewhat separated right now. There are separate python modules,
> but so far the main script calls them all in sequence.
> Log files are generated in any case, and are optionally deleted afterwards.
> There shouldn't be problems there.
> The flow of data is like this: (measure -> logfile), (logfile -> sqlite
> database), (database -> HTML).

OK, then it shouldn't be a problem to do this later.


Ciao
Christian