[Dune] Assertion error when running Dune greed in parallel

[Jorrit Fahlke]

Am Fr,  5. Jul 2019, 14:06:39 +0000 schrieb Guichard, Roland:
> Thank you for the clarification. Some comments below:
> 
> On 5 Jul 2019, at 14:38, Jö Fahlke <jorrit.fahlke at wwu.de<mailto:jorrit.fahlke at wwu.de>> wrote:
> 
> Am Fr,  5. Jul 2019, 12:28:51 +0000 schrieb Guichard, Roland:
> I see thanks.
> A quick question though, does it mean that there are several MPI initializations possible within Dune and the communicator that needs to be passed to the GridPtr (from dune-fem) is responsible for handling the mesh in parallel ?
> 
> MPI_Init is only ever called once.
> 
> 
> Yes, but potentially through multiple ways. I mean by this that for instance
> the Dune::Fem::MPIManager::initialize( argc, argv ) implicitly calls the
> MPIHelper::instance( argc, argv ).

And MPIHelper::instance() ensures that it will create the internal MPIHelper
singleton just once, see
- https://gitlab.dune-project.org/core/dune-common/blob/master/dune/common/parallel/mpihelper.hh#L225
- https://en.cppreference.com/w/cpp/language/storage_duration#Static_local_variables

In addition, the constructor of the MPIHelper singleton checks whether
MPI_Init() has been called previously, and if so won't call it itself, see
https://gitlab.dune-project.org/core/dune-common/blob/master/dune/common/parallel/mpihelper.hh#L249
That should catch any initializations done before MPIHelper::instance() has
been called the first time.

> I understand that managers are wrappers around the MPIHelper one. However what I don’t quiet get so far is that, in the code, MPI ranks and size are retrieved using direct MPI calls:
> 
> MPI_Comm_rank(MPI_COMM_WORLD, &myRank);
> MPI_Comm_size(MPI_COMM_WORLD, &numProcs);
> 
> But since the managers provide that functionality I don’t see the point in doing this. Ideally and for readability, you would access anything you need through a single manager. And this also applies to the communicator.

If by "managers" you mean "grid managers": Some of those predate Dune, so
began life on plain MPI.  They could be changed to get the MPI communicator
from MPIHelper, but that's always going to be MPI_COMM_WORLD, so there's
little point.  Others are actually independend of Dune, and Dune
only provides a wrapper around them.  Since they don't know they're being used
from inside Dune, they can't assume the presence of Dune::MPIHelper.  If their
authors chose to hardcode them to use MPI_COMM_WORLD, then that is was Dune
has to roll with.

The purpose of Dune::MPIHelper was not the provide a general C++ abstraction
over plain MPI.  Instead, it was meant to provide a very basic abstraction
over the two cases of compiling with and without MPI, so the user can avoid
the #ifdef's.

> But that is merely my opinion.
> 
> 
> Though it is up to the grid manager implementation which MPI communicators are
> supported.  Some managers may implicitly assume MPI_COMM_WORLD, others may not
> support MPI at all (which is implicitly equivalent to MPI_COMM_SELF).
> 
> Hm, do I need to understand managers in a larger scope that MPI only then ?
> 
> 
> For yet others it may be possible to specify the communicator when
> constructing the grid.  So if you obtained that communicator by splitting
> MPI_COMM_WORLD into two subsets of nodes, you will end up with two independent
> grids, each operating within one of the subsets.
> 
> OK got that one.
> 
> 
> Regards,
> Jö.
> 
> --
> Jorrit (Jö) Fahlke, Institute for Computational und Applied Mathematics,
> University of Münster, Orleans-Ring 10, D-48149 Münster
> Tel: +49 251 83 35146 Fax: +49 251 83 32729
> 
> This message is protected by DoubleROT13 encryption
> Attempting to decode it violates the DMCA/WIPO acts
> 

-- 
Jorrit (Jö) Fahlke, Institute for Computational und Applied Mathematics,
University of Münster, Orleans-Ring 10, D-48149 Münster
Tel: +49 251 83 35146 Fax: +49 251 83 32729

Interpunktion, Orthographie und Grammatik der Email ist frei erfunden.
Eine Übereinstimmung mit aktuellen oder ehemaligen Regeln wäre rein
zufällig und ist nicht beabsichtigt.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20190705/d86733ae/attachment.sig>