<div dir="ltr"><div><div><div><div>Hello, Christian!<br><br></div>I'm sorry for not replying earlier, my girlfriend came back from a conference and we had to visit her family. I read email regularly, but I didn't do anything since the weekend because I have my development setup at home. I also get few chances to reply, so I hope to address all the points now. <br>
<br></div>1.) Python's configparser changed during the 2->3 transition, but I didn't know the changes were that significant. I'll get back home tomorrow, and I will test your changes then. According to the docs, the get() syntax is available in the new version, so we can go with this one. <br>
<br></div>2.) Using the buildsystem to call the measurement routine. I agree, this is probably the best way to get all the needed compiler information. Of course, it should be easy to use, so a single macro or target (per measured program) should be used. I will try to do it in automake, but would it be ok if I first write a CMake macro, get the details worked out, and then do the same with autotools? Considering the use of two build systems, I this part would be a pretty thin wrapper, just getting the compiler info and passing it to python. <br>
<br></div><div>3.) Split HTML files. I agree, this would be a good idea. I don't know if it works for you, but I added table filtering, so you can choose to only display Run, Compile or all measurements. I would keep the total file, but I can also generate type-specific ones. <br>
<br></div><div>4.) No graphs in resIults. I don't know why that should be. Yes, the data is in a separate CSV file, but in my testing, it was always loaded correctly. There might be some errors in handling paths, I'll check when I get home. <br>
<br></div><div>5.) Automatic testing. The program now parses a configuration file (perftest.ini) and runs the tests described in it. This means you still have to manually run a command, but it's only one command per run. I suppose "make perftest" is easier to remember than "perftest.py perftest.ini", so some integration with the buildsystem is needed. However, I wouldn't run performance tests automatically every time the program is compiled, this would take too much time. <br>
</div><div><div><br></div><div>6.) My progress. I added statistics for finding outliers, you may notice in the results some rows are different colors depending on their distance from the mean. Currently all points with at least 1-sigma deviation are marked, but I don't think that should be the case. There are more rigorous definition of outliers. The documentation uses Doxygen, <br>
but I haven't converted all the actual docstrings yet. The graphs show both memory consumption and time spent, and there are separate graphs for compilation and running. <br></div></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">2013/8/14 Christian Engwer <span dir="ltr"><<a href="mailto:christian.engwer@uni-muenster.de" target="_blank">christian.engwer@uni-muenster.de</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Miha,<br>
<br>
I didn't yet get a response on my mail from the weekend. Is everythin<br>
all right?<br>
<br>
Christian<br>
<div><div class="h5"><br>
On Tue, Aug 13, 2013 at 09:22:58AM +0200, Christian Engwer wrote:<br>
> > a) ran into a problem:<br>
> ><br>
> > christi@sansibar:~/Uni/Dune/dune-perftest$ ./dune/perftest/perftest.py perftest.ini<br>
> > Traceback (most recent call last):<br>
> > File "./dune/perftest/perftest.py", line 86, in <module><br>
> > test_project(sys.argv[1])<br>
> > File "./dune/perftest/perftest.py", line 30, in test_project<br>
> > if not CONFIG_TOP_SECTION in config:<br>
> > TypeError: argument of type 'instance' is not iterable<br>
><br>
> the attached patch fixed the problems for me... not sure what it will<br>
> do to python3?!<br>
><br>
> Christian<br>
<br>
</div></div>> diff --git a/dune/perftest/perftest.py b/dune/perftest/perftest.py<br>
> index f477589..e541a3e 100755<br>
> --- a/dune/perftest/perftest.py<br>
> +++ b/dune/perftest/perftest.py<br>
> @@ -27,28 +27,30 @@ def test_project(config_file):<br>
> config = configparser.ConfigParser()<br>
> config.read(config_file)<br>
><br>
> - if not CONFIG_TOP_SECTION in config:<br>
> + if not CONFIG_TOP_SECTION in config.sections():<br>
> raise SyntaxError("The configuration file must have a [DUNE Perftest] section")<br>
><br>
> - program_name = config[CONFIG_TOP_SECTION][KEY_PROGRAM_NAME]<br>
> - output_file = config[CONFIG_TOP_SECTION][KEY_OUTPUT]<br>
> + program_name = config.get(CONFIG_TOP_SECTION,KEY_PROGRAM_NAME)<br>
> + output_file = config.get(CONFIG_TOP_SECTION,KEY_OUTPUT)<br>
><br>
> revision = None<br>
> - if KEY_VCS in config[CONFIG_TOP_SECTION]:<br>
> - revision = get_revision(dir_name, config[CONFIG_TOP_SECTION][KEY_VCS])<br>
> + if KEY_VCS in config.items(CONFIG_TOP_SECTION):<br>
> + revision = get_revision(dir_name, config.get(CONFIG_TOP_SECTION,KEY_VCS))<br>
><br>
> - for section in config:<br>
> + for section in config.sections():<br>
> + print "Parsing section " + section<br>
> if section == CONFIG_TOP_SECTION:<br>
> continue<br>
><br>
> - if not KEY_COMMAND in config[section]:<br>
> + if not config.has_option(section, KEY_COMMAND):<br>
> + print("No Entry '" + KEY_COMMAND + "' ... Skipping sections " + section)<br>
> continue;<br>
><br>
> - command_type = config[section][KEY_COMMAND_TYPE] if KEY_COMMAND_TYPE in config[section] else section<br>
> + command_type = config.get(section,KEY_COMMAND_TYPE) if config.has_option(section, KEY_COMMAND_TYPE) else section<br>
><br>
> - if KEY_PRE_COMMAND in config[section]:<br>
> - subprocess.call(config[section][KEY_PRE_COMMAND].split())<br>
> - data = measure(program_name, config[section][KEY_COMMAND].split(), command_type=command_type, revision=revision)<br>
> + if config.has_option(section, KEY_PRE_COMMAND):<br>
> + subprocess.call(config.get(section,KEY_PRE_COMMAND).split())<br>
> + data = measure(program_name, config.get(section,KEY_COMMAND).split(), command_type=command_type, revision=revision)<br>
><br>
> log_file = os.path.join(dir_name, "%s_%s.log" % (output_file, section))<br>
> db_file = os.path.join(dir_name, "%s.db" % output_file)<br>
<div class="HOEnZb"><div class="h5"><br>
> _______________________________________________<br>
> Dune-devel mailing list<br>
> <a href="mailto:Dune-devel@dune-project.org">Dune-devel@dune-project.org</a><br>
> <a href="http://lists.dune-project.org/mailman/listinfo/dune-devel" target="_blank">http://lists.dune-project.org/mailman/listinfo/dune-devel</a><br>
<br>
<br>
--<br>
Prof. Dr. Christian Engwer<br>
Institut für Numerische und Angewandte Mathematik<br>
Fachbereich Mathematik und Informatik der Universität Münster<br>
Einsteinstrasse 62<br>
48149 Münster<br>
<br>
E-Mail <a href="mailto:christian.engwer@uni-muenster.de">christian.engwer@uni-muenster.de</a><br>
Telefon <a href="tel:%2B49%20251%2083-35067" value="+492518335067">+49 251 83-35067</a><br>
FAX <a href="tel:%2B49%20251%2083-32729" value="+492518332729">+49 251 83-32729</a><br>
</div></div></blockquote></div><br></div>