[Dune] Assertion error when running Dune greed in parallel

Fri Jul 5 14:53:00 CEST 2019

Hi again,

It looks from

https://gitlab.dune-project.org/dune-fem/dune-fem/blob/master/dune/fem/misc/mpimanager.hh#L122

That you can get the communicator using something like Dune::Fem::MPIManager::comm() (not tested yet though)

Best wishes,

Dr. Roland Guichard
Research Software Engineer
UCL-RITS
Internal Extension: 86947
External Number: 02031086947

On 5 Jul 2019, at 13:28, Guichard, Roland <r.guichard at ucl.ac.uk<mailto:r.guichard at ucl.ac.uk>> wrote:

Hi,

I see thanks.
A quick question though, does it mean that there are several MPI initializations possible within Dune and the communicator that needs to be passed to the GridPtr (from dune-fem) is responsible for handling the mesh in parallel ?

Best wishes,

Dr. Roland Guichard
Research Software Engineer
UCL-RITS
Internal Extension: 86947
External Number: 02031086947

On 5 Jul 2019, at 12:51, Markus Blatt <markus at dr-blatt.de<mailto:markus at dr-blatt.de>> wrote:

Hi,

please always reply to the list (using e.g. reply to all). I took the liberty
to bounce your mail to list.

You definitely need to pass the communicator to the constructor of the GridPtr.
Not sure whether this is the complete fix.

Without dune-fem one would do

Dune::MPIHelper& helper = Dune::MPIHelper::instance(argc, argv);
// Dune::GridPtr< HGridType > gridPtr( meshName.str() );
Dune::GridPtr<GridType> gridPtr(meshName.str(),helper.getCommunicator());
gridPtr.loadBalance();

How to get the communicator using dune-fem somebody else needs to tell you.

Markus

On Fri, Jul 05, 2019 at 11:28:36AM +0000, Guichard, Roland wrote:
Dear Markus,

Thank you for your prompt response.

Sure, you can find attached a stripped out version of the main. I only have inherited that code, not developed it myself.

You’ll see that the same path to the dgf file is passed to all MPI ranks but I don’t understand why initialising the DGFGridFactory is rank dependent.

Besides, any comments/suggestions on the code are very welcome. Part of my job is also to refactor it for a better use. The main issue is that there is such a gap between Dune2.3 and Dune2.6 which makes it uneasy to track (owing that I have never used Dune before :)

Thank you again for your time

Kind regards,

Dr. Roland Guichard
Research Software Engineer
UCL-RITS
Internal Extension: 86947
External Number: 02031086947

On 5 Jul 2019, at 12:00, Markus Blatt <markus at dr-blatt.de<mailto:markus at dr-blatt.de><mailto:markus at dr-blatt.de>> wrote:

Hi,

would you mind to share your setup code for the grid?

Markus

On Fri, Jul 05, 2019 at 09:24:33AM +0000, Guichard, Roland wrote:
Dear Dune people,

I am currently working on a project to update a home made Dune module (peits) to the newest Dune modules. Peits was previously running fine with Dune modules (common, fem, grid, alugrid, geometry) v2.3.

Now I am trying to update to the same Dune modules v2.6. I have managed to compile (under GNU compiler) and run serially Peits with Dune2.6 but the issue appears when I run in parallel using MPI. This occurs when the gridPtr is called,

And trying to create the grid from a dgf file, especially the assertion:

https://gitlab.dune-project.org/core/dune-grid/blob/master/dune/grid/io/file/dgfparser/gridptr.hh#L281<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.dune-project.org%2Fcore%2Fdune-grid%2Fblob%2Fmaster%2Fdune%2Fgrid%2Fio%2Ffile%2Fdgfparser%2Fgridptr.hh%23L281&data=02%7C01%7C%7C6894be42ceb84941f59108d701445643%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C636979265323126881&sdata=Ldn%2BCq1cmUz6EyZAid2a8%2BGSCz0OiUb1SP3zGoeVbJ8%3D&reserved=0>

Basically, what I have narrowed down is:

*   The MPI rank 0 runs fine. It fetches all parameters from files, calls the DGFGridFactory, initializes it etc.
*   The MPI rank 1 comes to the same point but I have checked that the initialisation of the DGFGridFactory is not setting the necessary parameters the same way, I.e. the elParam_ size is set to 0 (whereas ~57000 in MPI rank 0) causing the above assertion to fail.

I wanted to know if there is a dedicated procedure to create a grid in parallel which I am missing at this point. Could you advise on this issue ?

That would be really helpful.

Kind regards,

Dr. Roland Guichard
Research Software Engineer
UCL-RITS
Internal Extension: 86947
External Number: 02031086947

_______________________________________________
Dune mailing list
Dune at lists.dune-project.org<mailto:Dune at lists.dune-project.org><mailto:Dune at lists.dune-project.org>
https://lists.dune-project.org/mailman/listinfo/dune<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.dune-project.org%2Fmailman%2Flistinfo%2Fdune&data=02%7C01%7C%7C6894be42ceb84941f59108d701445643%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C636979265323126881&sdata=rKd8iHjZId9Y5wYt%2FRAnsHZEEcj7TkNCWWsV0Y0cJbU%3D&reserved=0>

--
Using PT-Scotch with DUNE https://t.co/hZH8lxf5ZF<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ft.co%2FhZH8lxf5ZF&data=02%7C01%7C%7C6894be42ceb84941f59108d701445643%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C636979265323136877&sdata=RSMu625vrKSfTYkjh4f3zp78dTMQFz0%2BOmWnZIoHlrs%3D&reserved=0>

Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dr-blatt.de%2F&data=02%7C01%7C%7C6894be42ceb84941f59108d701445643%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C636979265323136877&sdata=l7JjPJkzv%2B0cOH63F1lrkjbOB4xwV7mkyMsQeGi%2Fj%2Bk%3D&reserved=0>
Pedettistr. 38, 85072 Eichstätt, Germany
Tel.: +49 (0) 160 97590858

--
Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dr-blatt.de%2F&data=02%7C01%7C%7C6894be42ceb84941f59108d701445643%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C636979265323146869&sdata=olkW1mL2yOxxGRdAaF1lFdm%2BPGm6%2Br%2BidMK2wutYJSE%3D&reserved=0>
Pedettistr. 38, 85072 Eichstätt, Germany,  USt-Id: DE279960836
Tel.: +49 (0) 160 97590858

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20190705/2e3ddb04/attachment.htm>