[Dune] Assertion error when running Dune greed in parallel

[Markus Blatt]

On Fri, Jul 05, 2019 at 02:37:51PM +0000, Guichard, Roland wrote:
> @Markus,
> 
> I passed a communicator I got from the Dune::Fem::MPIManager but the error persists:
> 
> dune-peits: /home/vagrant/dune-grid-2.6.0/dune/grid/io/file/dgfparser/gridptr.hh:291: const std::vector<double>& Dune::GridPtr<GridType>::parameters(const Entity&) const [with Entity = Dune::Entity<0, 3, const Dune::ALU3dGrid<3, 3, (Dune::ALU3dGridElementType)4, Dune::ALUGridMPIComm>, Dune::ALU3dGridEntity>; GridType = Dune::ALUGrid<3, 3, (Dune::ALUGridElementType)0, (Dune::ALUGridRefinementType)0>]: Assertion `(unsigned int)gridView.indexSet().index( entity ) < elParam_.size()' failed.
> 
> When I print out the indexSet.size( 0 ) from which elParam_.size() is initialised, I get:
> 
> indexSet.size( 0 ) before the elParam_ initialisation: 0
> After the DFGfactory call, after initialize, elParam_ size is: 0

If this after ther loadBalance call, then this is weired. The loadBalance call
should have communicated them.
>
> For the MPI rank 1. Any thoughts maybe ?
> That’d be really appreciated.
>



Unfortunately your example does not seem include the line that triggers this
call. (A backtrace would be nice). So I am still guessing wild:

Which grid view are you using? The loadbalanced grid should be level 1. I would
recommend to work with the leafGridView.

If you are then please check that that gridPtrs->comm()->size() is greater than
1 initially.


Markus

> 
> 
> On 5 Jul 2019, at 15:06, Guichard, Roland <r.guichard at ucl.ac.uk<mailto:r.guichard at ucl.ac.uk>> wrote:
> 
> Thank you for the clarification. Some comments below:
> 
> On 5 Jul 2019, at 14:38, Jö Fahlke <jorrit.fahlke at wwu.de<mailto:jorrit.fahlke at wwu.de>> wrote:
> 
> Am Fr,  5. Jul 2019, 12:28:51 +0000 schrieb Guichard, Roland:
> I see thanks.
> A quick question though, does it mean that there are several MPI initializations possible within Dune and the communicator that needs to be passed to the GridPtr (from dune-fem) is responsible for handling the mesh in parallel ?
> 
> MPI_Init is only ever called once.
> 
> 
> Yes, but potentially through multiple ways. I mean by this that for instance the Dune::Fem::MPIManager::initialize( argc, argv ) implicitly calls the MPIHelper::instance( argc, argv ). I understand that managers are wrappers around the MPIHelper one. However what I don’t quiet get so far is that, in the code, MPI ranks and size are retrieved using direct MPI calls:
> 
> MPI_Comm_rank(MPI_COMM_WORLD, &myRank);
> MPI_Comm_size(MPI_COMM_WORLD, &numProcs);
> 
> But since the managers provide that functionality I don’t see the point in doing this. Ideally and for readability, you would access anything you need through a single manager. And this also applies to the communicator.
> But that is merely my opinion.
> 
> 
> Though it is up to the grid manager implementation which MPI communicators are
> supported.  Some managers may implicitly assume MPI_COMM_WORLD, others may not
> support MPI at all (which is implicitly equivalent to MPI_COMM_SELF).
> 
> Hm, do I need to understand managers in a larger scope that MPI only then ?
> 
> 
> For yet others it may be possible to specify the communicator when
> constructing the grid.  So if you obtained that communicator by splitting
> MPI_COMM_WORLD into two subsets of nodes, you will end up with two independent
> grids, each operating within one of the subsets.
> 
> OK got that one.
> 
> 
> Regards,
> Jö.
> 
> --
> Jorrit (Jö) Fahlke, Institute for Computational und Applied Mathematics,
> University of Münster, Orleans-Ring 10, D-48149 Münster
> Tel: +49 251 83 35146 Fax: +49 251 83 32729
> 
> This message is protected by DoubleROT13 encryption
> Attempting to decode it violates the DMCA/WIPO acts
> 
> 

-- 
Using PT-Scotch with DUNE https://t.co/hZH8lxf5ZF

Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de
Pedettistr. 38, 85072 Eichstätt, Germany
Tel.: +49 (0) 160 97590858
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20190708/4ed92e73/attachment.sig>