[Dune] Dune and spack

Wed Apr 29 17:41:09 CEST 2020

Dear Olivier,

thanks for putting so much work into this.

I have looked at the Github issue (for reference, it is this one:
https://github.com/spack/spack/pull/391).
The first presented answer is exactly what made me stop working on this, I
do not think that is an appropriate
solution for handling this (exponentially growing) problem.

Concerning resource directives: I have never heard of them and I wasn't
able to quickly find documentation
on how they are meant to work! Do you have an example how such a thing
would look like? Also, is it a mature
feature of spack? I found myself struggling more than once with
half-implemented or abandoned features of
spack that were not clearly advertised as such (e.g. spack setup) so I am a
bit scared of niche features in spack.

Best,
Dominic

On Wed, Apr 29, 2020 at 4:31 PM Gauthier Folzan <gauthier.folzan at gmail.com>
wrote:

> Hi,
> I asked advices on the spack github and I had an interesting response from
> Massimiliano Culpo :
>
> Maybe a better solution, if it makes sense for Dune, is to treat Dune
> modules as resources. This means having a single "Dune" package with
> multiple resource directives. These resources might be activated by a
> corresponding variant and Dune should be able to install them on request.
>
>
> It was obvious for me to have one spack package by dune module. I didn't
> think about it but having one unique package makes sense. It ensures
> options coherence and version coherence between the modules within a
> package installation. Spack is already multiplying packages installations
> with different variant, so it's not a problem to have multiple and
> independent dune installation with different activated modules if needed.
>
> What do you think?
>
> Regards,
>
> Gauthier
>
> On Tue, Apr 28, 2020 at 1:19 PM Oliver Sander <oliver.sander at tu-dresden.de>
> wrote:
>
>> Hi,
>>
>> as this topic seems to interest quite a number of people, can we not
>> seize the momentum and actually work on improving Dune towards
>> the Spack usecase?  I am not a build system expert, but I have yet
>> to meet somebody who is happy with the current state of module
>> dependency tracking.  What would a reasonable first step be?
>> How difficult would it be, e.g., to have Dune modules provide
>> at least rudimentary config-file packages?
>>
>> Best,
>> Oliver
>>
>> On 28.04.20 12:17, Christian Engwer wrote:
>> > Hi,
>> >
>> > this all sound rather nasty. Can we work around this by specifying the
>> > correct cmake variables from spack and passing them to dependent
>> > modules?
>> >
>> > Christian
>> >
>> > On Tue, Apr 28, 2020 at 11:59:53AM +0200, Gauthier Folzan wrote:
>> >> Hi Dominic,
>> >> indeed, I realized that I needed to define the dune variants everywhere
>> >> (what I did more or less).
>> >> I think defining a common group of variants everywhere is not a so big
>> deal
>> >> but the big problem is to propagate these variants across the dependent
>> >> modules.
>> >> The approach found in some spack packages is to write all possible
>> >> combinations but this is cumbersome and I think this is what you mean
>> by
>> >> not elegant. Am I right ?
>> >> In a first approach, I wanted to have at least a first package to be
>> able
>> >> to install dune (at least for me), so what I did is to set every
>> variant to
>> >> true to have a consistent set of dune modules but I agree this is not a
>> >> solution.
>> >> I found some discussions about variant propagation in spack forums but
>> >> nothing seems to have been merged in the main branch.
>> >> To be sure, don't you have the same problem of flags coherence across
>> >> modules using only cmake but compiling one by one the dune modules ?
>> >> Regards,
>> >> Gauthier
>> >>
>> >> On Wed, Apr 22, 2020 at 11:05 AM Dominic Kempf <
>> dominic.r.kempf at gmail.com>
>> >> wrote:
>> >>
>> >>> Hey Gauthier,
>> >>>
>> >>> sorry for jumping in late.
>> >>>
>> >>> Christian has it right here, I was looking into Spack as a way of
>> setting
>> >>> up our CI and maybe moving towards testing variability in user
>> landscapes
>> >>> as well. The main reason I stopped working on it was that the
>> following
>> >>> particularity (some people call it a bug) of the Dune CMake build
>> system
>> >>> clashes with Spack:
>> >>> Each Dune module reruns cmake find scripts for dependencies of all
>> Dune
>> >>> modules it depends on instead of importing the full configuration of
>> the
>> >>> upstream modules. Due to this behaviour only modules that were built
>> in the
>> >>> same environment and with the same flags can be safely used together
>> (as
>> >>> their cmake find scripts found the same things).
>> >>> For Spack, that means that all variants that you define need to be
>> >>> redundantly defined across all modules, and dependencies between
>> those need
>> >>> to be specified manually. That is neither elegant nor feasible in
>> >>> maintenance.
>> >>> The Dune developers have voiced their willingness to abandon this
>> >>> behaviour, but a volunteer to do such a tedious, unrewarding work is
>> yet to
>> >>> be found.
>> >>>
>> >>> Best,
>> >>> Dominic
>> >>>
>> >>> On Wed, Apr 1, 2020 at 3:29 PM Gauthier Folzan <
>> gauthier.folzan at gmail.com>
>> >>> wrote:
>> >>>
>> >>>> Hi Christian,
>> >>>> here is the link : https://github.com/gauthier12/dune_spack_repo.git
>> >>>> It is still work in progress and I am still testing so I don't
>> guarantee
>> >>>> anything but it seems to work on ubuntu, centos and archlinux for
>> >>>> dune-pyhon with dune-common, dune-geometry, dune-grid, dune-istl and
>> >>>> dune-uggrid
>> >>>> Once you have spack installed, installing dune is quite simple, just
>> add
>> >>>> the repo and install dune
>> >>>> spack repo add dune_spack_repo
>> >>>> spack install dune-python
>> >>>> Cheers
>> >>>> Gauthier
>> >>>>
>> >>>> On Tue, Mar 31, 2020 at 4:46 PM Christian Engwer <
>> >>>> christian.engwer at uni-muenster.de> wrote:
>> >>>>
>> >>>>> Hi Gauthier,
>> >>>>>
>> >>>>> welcome to Dune,
>> >>>>>
>> >>>>>> Does someone use spack to compile Dune ? If it can be useful to
>> >>>>> someone,
>> >>>>>> the spack repo is on github.
>> >>>>>
>> >>>>> I think nobody is actively using spack for Dune currently, Dominik
>> had
>> >>>>> been workin on spack packages... I think his intention was to ease
>> the
>> >>>>> CI setup.
>> >>>>>
>> >>>>> On the other hand, if spack packages are available, it might become
>> >>>>> more appealing for other people ;-) Can you post a link to your
>> repo?
>> >>>>>
>> >>>>> Thanks
>> >>>>> Christian
>> >>>>>
>> >>>> _______________________________________________
>> >>>> Dune mailing list
>> >>>> Dune at lists.dune-project.org
>> >>>> https://lists.dune-project.org/mailman/listinfo/dune
>> >>>
>> >>>
>> >
>> >> _______________________________________________
>> >> Dune mailing list
>> >> Dune at lists.dune-project.org
>> >> https://lists.dune-project.org/mailman/listinfo/dune
>> >
>> >
>>
>> _______________________________________________
> Dune mailing list
> Dune at lists.dune-project.org
> https://lists.dune-project.org/mailman/listinfo/dune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20200429/7177a72e/attachment.htm>