[Dune-devel] JIT compiling of python code uses wrong compile_commands

[Simon Praetorius]

Ok, I might have found the problem. It was a small detail in you mail, 
Andres, that pointed me in a direction to investigate:


There should be a file " in the builddir of dune-vtk under 
python/dune/data/dune-vtk.cmake.", but I have found mine in a slightly 
different directory, in |python/vtk/data/dune-vtk.cmake|. First, I 
though, this might have been a typo, this seems to be the problem.


How is this file generated? I don't know exactly, but it might be 
somehow in the dune_python_configure_bindings() macro. This macro was 
changed some time ago and I had to update the call from the old function 
to the new one. There was not indication what the new arguments mean, in 
relation to the old macro parameters, so my solution in 
dune-vtk/python/CMakeLists.txt was


dune_python_configure_bindings(
PATH"."
PACKAGENAME vtk
CMAKE_METADATA_FLAGS DUNE_OPTS_FILE
)


I named the package "vtk". This seems to be a problem. I don't know 
where this "PACKAGENAME" argument might be used, but it generates the 
file in the wrong directory, which is then not found in the whole 
machinery.

Removing this argument made the file dune-vtk.cmake appear in the 
correct directory and also the compile_command.json (and the CMakeLists 
file that it is generated from) contain the dune-vtk directory.


Thanks for the hints and the questions where to start looking.


Best,
Simon


Am 03.04.25 um 23:12 schrieb Simon Praetorius:
>
> Hi Andreas,
>
>
> thanks for the hints already. I have followed your questions (see 
> below), but still one step in the middle seems to fail.
>
>
>> |My first question would be:|
>> |what does|| build/run-in-dune-venv.py pip list | grep dune |give 
>> you? Most importantly does it list dune-vtk?
>
> Yes, dune-vtk is listed there
>
>
>> If that is found by pip then check the `dune-py/CMakeLists.txt` file 
>> if the line |set(dune-vtk_DIR PATH) ||is there.
>> |
>
> The dune-py/CMakeLists.txt does **not** contain the vtk dir or 
> anything else from dune-vtk (e.g. no HAVE_DUNE_VTK variable)
>
>
>> |
>> The path comes from the ||metadata |file which should be in the 
>> builddir of dune-vtk under |python/dune/data/dune-vtk.cmake|.
>
> The dune-vtk.cmake file exists and contains the following lines (and a 
> few more)
>
>
> DEPBUILDDIRS=/opt/sources/dune/dune-vtk/build/gcc14-debug;/opt/sources/dune/dune-common/build/gcc14-debug;/opt/sources/dune/dune-geometry/build/gcc14-debug;/opt/sources/dune/dune-localfunctions/build/gcc14-debug;/opt/sources/dune/dune-uggrid/build/gcc14-debug;/opt/sources/dune/dune-grid/build/gcc14-debug;/opt/sources/dune/dune-istl/build/gcc14-debug;/opt/sources/dune/dune-typetree/build/gcc14-debug;/opt/sources/dune/dune-functions/build/gcc14-debug;/opt/sources/dune/dune-alugrid/build/gcc14-debug;/opt/sources/dune/dune-foamgrid/build/gcc14-debug
> DEPS=dune-vtk dune-common dune-geometry dune-localfunctions 
> dune-uggrid dune-grid dune-istl dune-typetree dune-functions 
> dune-alugrid dune-foamgrid
>
>
> This indicates that the metadata is correct, but somehow ignored.
>
>
>> Only after all that looks correct I would check the 
>> |compile_commands.json| file.
>
> This file also does not contain the required include directories of 
> dune-vtk.
>
>
>> That is generated in python/dune/generator/cmakebuilder.py:547
>> by compiling a empty cc and extracting the information from the files 
>> that cmake generated.
>
> Since the CMakeLists.txt file in the dune-py dir is incomplete, the 
> generated compile_commands cannot be correct either. So, now the 
> question is, how is this CMakeLists file generated. How can I inspect 
> or activate the logging tools? And where to find the log output? Maybe 
> there is an information that helps.
>
>
>
>> I guess this step depends on internal structures of cmake generated 
>> files (i.e. link.txt' and could possibly fail with some newer CMake 
>> version.
>>
>> Best
>> Andreas
>> ------------------------------------------------------------------------
>> *From:* Dune-devel <dune-devel-bounces at lists.dune-project.org> on 
>> behalf of Simon Praetorius <simon.praetorius at tu-dresden.de>
>> *Sent:* 03 April 2025 20:04
>> *To:* dune-devel at lists.dune-project.org 
>> <dune-devel at lists.dune-project.org>
>> *Subject:* [Dune-devel] JIT compiling of python code uses wrong 
>> compile_commands
>>
>> Hi everyone,
>>
>> I have posted a question already in the Dune support channel, but 
>> maybe here in the mailing list are more python experts:
>>
>> I am currently trying to run a simple Python example in the dune-vtk 
>> module, i.e. |dune/python/test/testvtk.py|. I have built everything 
>> without a user specified venv, using the default that is created in 
>> the build directory. To run the code, I use the script 
>> |build/run-in-dune-venv.py| in my build directory |build/|. The 
>> problem I have is: The JIT compilation step uses an incomplete list 
>> of include directories. In particular, the include dirs from dune-vtk 
>> are not there. This can be seen in the |compile_commands.json| file, 
>> which is located in the dune-py directory in the dune-common build 
>> directory |build/dune-venv/.cache/dune-py|.
>>
>> The error I get is "fatal error: dune/python/vtk/writer.hh: file or 
>> directory not found".
>>
>> Can anyone tell me how this |compile_commands.json| file is 
>> generated? I am trying to figure out why it does not include all the 
>> include directories of the modules involved.
>>
>> Best, Simon
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dune-project.org/pipermail/dune-devel/attachments/20250403/67c2d487/attachment.htm>