[Dune-devel] Evolution of the CMake environment

[Simon Praetorius]

Hi everyone,

First of all, I want to thank for the development of the current cmake 
system. It is grown in a time where many of the recent cmake features 
were not readily available and we ended up with a system that works and 
that all dune modules are using. That's great. And I don't want to 
destroy this working system by imposing some 
cleanup/modernization/potentially improvement changes. Like the C++ (and 
now also python) code of the dune ecosystem develops and gets improved, 
also the buildsystem must continuously see progress. Otherwise it might 
stop working in some time when old cmake versions are not supported 
anymore or external packages cannot be found by our old FindModules.

After the big merge of the python overhaul, the cmake proposals might 
need an adaption or extension. Also, these buildsystem changes need a 
lot of testing. My own dune environment contains only a small subset of 
all modules. And there are multiple usage patterns. I already got 
valuable feedback, e.g. recently by Markus, after testing some branches. 
Not everyone wants to dive into the cmake development and that's 
perfectly fine. That's why I try to give these updates about some 
planned steps to get you on the track.

Best wishes,
Simon

Am 08.02.22 um 23:14 schrieb Christoph Grüninger:
> Hi Simon,
> thanks for your ceaseless work on the build system! You touched and 
> improved much code that has remained unchanched since Markus and 
> myself initially wrote it. But now we know more with regard to CMake 
> and our needs, and we don't need compatibility to Autotools anymore.
>
> Thank you for presenting your plans. I support them and if you need a 
> sparrings partner, some testing, or a review, please reach out!
> Besides that, do you ask for confirmation or something to discuss at 
> the upcoming developer meeting?
>
> Bye
> Christoph
>
> Am 06.02.22 um 13:49 schrieb Simon Praetorius:
>> Hi everyone,
>>
>> with this mail I want to summarize some proposals and suggested next 
>> steps for the CMake overhaul, as preparation and discussion base for 
>> the next developer meeting
>>
>> 1. What have we achieved (since the last meeting):
>>
>> - The big `DuneMacos.cmake` file is split up into several files 
>> containing only a few interface function plus related implementation 
>> details (This simplifies the cmake development significantly and 
>> revealed some historic outdated macros that could be removed)
>> - Some deprecated and outdated macros and functions are removed to 
>> use more modern 3.13 version implementations
>> - Several `Find<Package>` modules are rewritten to provide a common 
>> style, documentation, and imported targets to link against. Those 
>> often have the name `<Package>::[<Package>|<Subpackage>]`.
>> - The output during cmake configuration is reduced, e.g., by moving 
>> some operations to doc targets, or print messages into a logfile 
>> instead of stdout
>> - The important cmake function `dune_add_library` is rewritten and 
>> cleaned up. It is the main workhorse for creating dune module 
>> libraries and lead to a simplification of several main CMakeLists files.
>>
>> 2. What is proposed and planned:
>>
>> a) I propose a cleaner mechanism for setting the c++ standard for the 
>> compiler, by adding a target feature requirement. This allows to 
>> specifically state which standard a target (library or executable) 
>> needs (minimum requirement) and also allows to communicate this 
>> requirement to all other targets linking against it. A global c++ 
>> standard can always be set by using `CMAKE_CXX_STANDARD` variable, 
>> acting on all targets in the current project. Putting this into an 
>> options file, allows to enforce this for all modules. But also 
>> changing the standard in just dune-common will forward the new 
>> requirement to all downstream modules. There are some discussions 
>> going on whether this is the most stable/portable/secure way to 
>> enforce this. See 
>> https://gitlab.dune-project.org/core/dune-common/-/merge_requests/862.
>>
>> Some consequences:
>>    * property is communicated by targets: you have to link against a 
>> dune target to derive the dune c++ standard requirement.
>>    * Standard can be influenced by setting `CMAKE_CXX_STANDARD` 
>> variable, or by setting your own (higher) standard requirement on a 
>> downstream target
>>
>> b) I propose to make a dune module target a special 
>> supported/featured target. All dune modules should have a targets the 
>> contains the module sources (if any), and all properties (include 
>> dirs, compile flags, external dependencies) of the module. By having 
>> this and giving this target a special name, e.g., 
>> `DUNE_MODULE_TARGET`, we can, for example, fill in some properties 
>> automatically, like include directories, and can link against this 
>> target in module tests. This would simplify some code and would allow 
>> to move complicated properties to a common function, instead of 
>> setting this all by hand in each downstream module. See 
>> https://gitlab.dune-project.org/core/dune-common/-/merge_requests/944.
>>
>> Some consequences:
>>    * Every dune module, that is supposed to be used by downstream 
>> modules, needs to call `dune_project()` and 
>> `finalize_dune_project()`. Thus, the creation of such a module target 
>> could be added to these macros.
>>    * If we register include directories in an (exported) module 
>> target that is imported in downstream modules, these includes are 
>> treated as "system-includes". This is the default for all external 
>> libraries. This means, for example, that warnings emitted by included 
>> system headers are not shown when compiling downstream code. (This 
>> can be reactivated, though)
>>
>> c) I propose to define a namespace for dune module targets, like 
>> `Dune::`. Namespaces are very common in cmake, see external package 
>> imported targets above, and thus should be provided by Dune as well. 
>> A namespace target is typically just an alias to an internal library 
>> target, i.e., `Dune::common -> dunecommon`, or it is an imported 
>> target when used in a downstream module. There, the former "internal 
>> library target" is not visible any more. Advantages for such a 
>> namespace structure: it follows the pattern of most other libraries, 
>> it groups the targets into a namespace, instead of setting a prefix 
>> to the name itself, it prevents from linking against an unknown dune 
>> module since a target must be available.
>>
>> Some consequences and discussion:
>>    * The name that comes after the :: needs to be decided on. It is 
>> made flexible in the `dune_add_library` function and in MR!944 by 
>> providing an option "EXPORT_NAME". There is a default (if nothing 
>> else is specified) that is the project name without the "dune-" 
>> prefix. Other discretization modules could set their own export name, 
>> e.g. for dumux or amdis. But, if you are using the `dune_project` 
>> macros, the namespace will always be `Dune::`
>>    * We have actually already > 2 spellings of dune modules that we 
>> are using in cmake, e.g., "dune-common", "dunecommon", "DUNE_COMMON", 
>> and now additionally "Dune::common". This gets complicated and it is 
>> not easy to guess which spelling to use in which location. Thus, I 
>> propose to remove the `dunecommon` in favor of `dune-common`. Note, 
>> this is unrelated to the filename of the created library. That one 
>> can be set by the option `OUTPUT_NAME` in dune_add_library` and 
>> `dune_project`.
>>
>> d) I propose to move the cmake commands currently written in the 
>> `Dune<Module>Macros.cmake` file, into the 
>> `dune-<module>-config.cmake` file. This can be done by automatically 
>> including the "old" Macros file. By having all the macros now 
>> directly in the config file, a dune package can be used in a project 
>> without including all the other dune-common cmake functions and 
>> without using `dune_project` and even without writing a `dune.module` 
>> file. There is still some work to do to get this running but the 
>> steps are already clear.
>>
>> Some consequences:
>>    * The `dune-<module>-config.cmake` file will be used by any 
>> `find_package(<module>)` call. Thus no need to manually include 
>> anything. Also it is not necessary to include the old Macros file 
>> again, because it is already included in the config file
>>    * I will try to make this an transparent change, i.e., a config 
>> file is already created automatically. This file will be extended by 
>> these additional includes. Only if you are writing your own 
>> `dune-<module>-config.cmake` file you have to add a few lines there. 
>> This will be documented once the implementation has some progress. 
>> I'm thinking about a smooth transition but this needs to come up once 
>> a draft is implemented.
>>
>>
>> Why do I propose this CMake overhaul? I envision dune modules that 
>> can be used without being forced to create a dune-project with all 
>> the cmake functions, just by linking against dune-<module>:
>>
>> --- CMakeLists.txt
>>
>> cmake_minimum_required(VERSION 3.13)
>> project(foo LANGUAGES CXX)
>> find_package(dune-istl REQUIRED)
>> add_executable(bar ...)
>> target_link_libraries(bar PRIVATE Dune::istl)
>>
>> ---
>>
>>
>> Best wishes,
>> Simon
>>
>>
>> _______________________________________________
>> Dune-devel mailing list
>> Dune-devel at lists.dune-project.org
>> https://lists.dune-project.org/mailman/listinfo/dune-devel
>
-- 
Dr. Simon Praetorius
Technische Universität Dresden
Institute of Scientific Computing
phone: +49 351 463-34432
mail: simon.praetorius at tu-dresden.de
web: https://tu-dresden.de/mn/math/wir/das-institut/beschaeftigte/simon-praetorius

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5204 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://lists.dune-project.org/pipermail/dune-devel/attachments/20220210/3b01c2b2/attachment.bin>