[Dune] Scalability of parallel grid construction

[Markus Blatt]

Hi,

On Mon, Dec 05, 2016 at 04:02:16PM +0100, Christian Engwer wrote:
> The interface allows you to communicate vertex data. Thus you can
> communicate the rank via the grid commincate method on all
> vertices. In the gather you store all ranks and afterwards you have
> the set of all neighboring ranks stored for each vertex.

I  do not think that this is an answer to Aleksejs question. I sounded
to as if he was asking how to setup the communication interfaces
within the grid implementation in a scalable manner.

> On Mon, Dec 05, 2016 at 03:35:05PM +0100, Aleksejs Fomins wrote:
> > Namely, the question is how to determine the neighboring processes for each shared entity on a 3D grid, namely, the neighboring processes containing the shared vertices, edges and faces. Currently, our algorithm requires to communicate the process boundary vertices from all to all processes, such that each process can determine its neighbors by comparing received vertex global indices with its own.
> >

If you cannot assume anything on how the grid got partitioned
(e.g. black box loadbalancing) then this might be the only
alternative.


> > Clearly, all-to-all communication does not scale with growing
> > process number. However, I could not figure out a good algorithm
> > at the time.

That might depend on the communication topology. Bear in mind that you might
not need to do this that often (without adaptivity: only
once). Therefore the amount of time might still be small compared to
the rest of the computation time.
As an improvment you could try to send this information in a ring. The
parallel index sets of dune-common do it like this.

Markus
-- 
Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de
Hans-Bunte-Str. 8-10, 69123 Heidelberg, Germany
Tel.: +49 (0) 160 97590858