[Dune] Assertion error when running Dune greed in parallel

[Guichard, Roland]

Cool, thanks Jorrit for the nice explanation :)

Dr. Roland Guichard
Research Software Engineer
UCL-RITS
Internal Extension: 86947
External Number: 02031086947







On 5 Jul 2019, at 16:41, Jorrit Fahlke <jorrit at jorrit.de<mailto:jorrit at jorrit.de>> wrote:

Am Fr,  5. Jul 2019, 14:06:39 +0000 schrieb Guichard, Roland:
Thank you for the clarification. Some comments below:

On 5 Jul 2019, at 14:38, Jö Fahlke <jorrit.fahlke at wwu.de<mailto:jorrit.fahlke at wwu.de><mailto:jorrit.fahlke at wwu.de>> wrote:

Am Fr,  5. Jul 2019, 12:28:51 +0000 schrieb Guichard, Roland:
I see thanks.
A quick question though, does it mean that there are several MPI initializations possible within Dune and the communicator that needs to be passed to the GridPtr (from dune-fem) is responsible for handling the mesh in parallel ?

MPI_Init is only ever called once.


Yes, but potentially through multiple ways. I mean by this that for instance
the Dune::Fem::MPIManager::initialize( argc, argv ) implicitly calls the
MPIHelper::instance( argc, argv ).

And MPIHelper::instance() ensures that it will create the internal MPIHelper
singleton just once, see
- https://gitlab.dune-project.org/core/dune-common/blob/master/dune/common/parallel/mpihelper.hh#L225
- https://en.cppreference.com/w/cpp/language/storage_duration#Static_local_variables

In addition, the constructor of the MPIHelper singleton checks whether
MPI_Init() has been called previously, and if so won't call it itself, see
https://gitlab.dune-project.org/core/dune-common/blob/master/dune/common/parallel/mpihelper.hh#L249
That should catch any initializations done before MPIHelper::instance() has
been called the first time.

I understand that managers are wrappers around the MPIHelper one. However what I don’t quiet get so far is that, in the code, MPI ranks and size are retrieved using direct MPI calls:

MPI_Comm_rank(MPI_COMM_WORLD, &myRank);
MPI_Comm_size(MPI_COMM_WORLD, &numProcs);

But since the managers provide that functionality I don’t see the point in doing this. Ideally and for readability, you would access anything you need through a single manager. And this also applies to the communicator.

If by "managers" you mean "grid managers": Some of those predate Dune, so
began life on plain MPI.  They could be changed to get the MPI communicator
from MPIHelper, but that's always going to be MPI_COMM_WORLD, so there's
little point.  Others are actually independend of Dune, and Dune
only provides a wrapper around them.  Since they don't know they're being used
from inside Dune, they can't assume the presence of Dune::MPIHelper.  If their
authors chose to hardcode them to use MPI_COMM_WORLD, then that is was Dune
has to roll with.

The purpose of Dune::MPIHelper was not the provide a general C++ abstraction
over plain MPI.  Instead, it was meant to provide a very basic abstraction
over the two cases of compiling with and without MPI, so the user can avoid
the #ifdef's.

But that is merely my opinion.


Though it is up to the grid manager implementation which MPI communicators are
supported.  Some managers may implicitly assume MPI_COMM_WORLD, others may not
support MPI at all (which is implicitly equivalent to MPI_COMM_SELF).

Hm, do I need to understand managers in a larger scope that MPI only then ?


For yet others it may be possible to specify the communicator when
constructing the grid.  So if you obtained that communicator by splitting
MPI_COMM_WORLD into two subsets of nodes, you will end up with two independent
grids, each operating within one of the subsets.

OK got that one.


Regards,
Jö.

--
Jorrit (Jö) Fahlke, Institute for Computational und Applied Mathematics,
University of Münster, Orleans-Ring 10, D-48149 Münster
Tel: +49 251 83 35146 Fax: +49 251 83 32729

This message is protected by DoubleROT13 encryption
Attempting to decode it violates the DMCA/WIPO acts


--
Jorrit (Jö) Fahlke, Institute for Computational und Applied Mathematics,
University of Münster, Orleans-Ring 10, D-48149 Münster
Tel: +49 251 83 35146 Fax: +49 251 83 32729

Interpunktion, Orthographie und Grammatik der Email ist frei erfunden.
Eine Übereinstimmung mit aktuellen oder ehemaligen Regeln wäre rein
zufällig und ist nicht beabsichtigt.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20190705/8ad05f39/attachment.htm>