[Dune-devel] [GSoC] Performance testing: more detailed schedule

[Christoph Grüninger]

Hi Miha,
please keep in mind that we prefer to communicate over our devel
mailinglist. This way every interested developer can keep track
of your progress. If you think that a message is better to
be sent in private, you must not forget to add both Christian
and me.

Make the change to Doxygen and CMake if you like. But hurry up,
there is no time to waste. Ask early because I am only available
until Friday. In general, I think we should gain momentum and
speed up again.

Would you mind to update the time table you gave in your application?
It describes what you wanted to achieve for every week. That's a
handy way to track progress and see what we can expect. And it reveals
delays what becomes more important towards the end.

I am not sure about how to get the information about git revision and
the compiler command. But you can try to capture the output from
some git command and the first lines of make. That would include all
the necessary information.

Bye
Christoph

Miha Čančula <miha at noughmad.eu> schrieb:
> 2013/8/5 Christoph Grüninger <christoph.grueninger at iws.uni-stuttgart.de>
> 
> > Hi Miha,
> > if you have no strong feelings about Sphinx versus Doxygen I'd ask you
> > to change for the latter; but the change has low priority.
> >
> With things like that, I have a feeling that if it's not done immediately
> it may never get done.
> I should probably convert to Doxygen before writing any more docs.
> I won't take much time anyway.
> 
> >
> > > Yes, I agree. I must confess I'm still not entirely familiar with
> > > autotools,
> > > so I didn't make it optional yet, but it's definitely on my to-do list.
> >
> > I know, autotools can be hard to begin with. If you have questions,
> > don't hesitate to ask us. Currently Doxygen is optional. The class
> > documentation is only built for systems with Doxygen installed. But
> > this is not an issue after the change.
> >
> > [CMake]
> > > No, it's because I haven't included the CMake files yet.
> >
> > I just saw that you committed a CMakeLists.txt file but I did not
> > check that.
> > You don't have to include CMake support. You are right, we are
> > evaluating the pros and cons for a switch or for keeping both. If
> > you want, you can share your opinion, too.
> > You can add CMake support if you like. I'll help you either way. But
> > autotools is a must have.
> >
> I like CMake, and the C++ part of my module is pretty small and simple.
> I have a feeling it won't take much time. If I encounter any problems I'll
> write to you.
> 
> >
> > Thanks for posting your plan, though it is kind of wage.
> > I would like to more details in the table. Either per default or
> > hidden until you hover over a specific area or click a link.
> > The details would be:
> > - date and time
> > - git revision hash
> > - if possible compiler, compiler version, and compiler flags
> > - maybe compile time?
> >
> > Would it be possible to add a graph for the memory consumption,
> > too?
> > And if the table could be sorted by clicking on the table header
> > would be useful, too.
> > But I think these ideas are for a later stage and to polish the
> > output. First we need the basic functions.
> >
> Yes, a vague plan is suitable for a blog post. I'll try to put more details
> here.
>  - First, now the procedure of measure->store->show is working, the next
> thing is adding more data.
> Memory consumption is already measured, and I've already added a separate
> graph for it.
> 
>  - Date and time are also already measured, just not displayed, so I will
> add this today.
> Git revision is more tricky, as it's only useful for built-in tests.
> I was thinking of adding the option of specifying some details at runtime,
> such as problem size.
> Perhaps the same should be done for git revision as well.
> 
>  - Complile time is more complicated, because we have to combine two
> commands.
> Usually you compile and run separately, with different commands, and there
> has to be a way of connecting the two.
> I could make it a runtime arguments as well, but I doubt many people would
> be happy running like this:
> 
> cd mycoolprog
> ./perftest --program "MyCoolProg" make
> ./perftest --program "MyCoolProg" ./mycoolprog
> 
> My best idea about this is having configuration files.
> Something like .ini (or .desktop) files should do, like so
> 
> [Dune Perftest Configuration]
> ProgramName="MyCoolProg"
> CompileCommand="make"
> RunCommand="./mycoolprog"
> 
> // Extra data that will just be added to the measurement result
> Compiler="GCC"
> CompilerFlags="-O3"
> <with possibility of specifying other data here>
> 
> I'm not sure about this, any suggestions?
> 
> So, my timeline for this week is
>  - convert the docs to Doxygen, and make it optional
>  - add memory consumption to graphs (already done).
>  - automated (as in, just calling one command) running of built-in tests.
>  - statistics (only mean, stdev for outliers, and linear trends)