[Dune] parallel DUNE grid for implementing a spherical shell (UGGrid)

[Eike Mueller]

What I could try is construct the grid as I do at the moment, refine it until I have one element per processor in a sequential 
run, and write it to disk as a dgf file. Then insert the projection for each boundary segment by hand (i.e. with a script). If I 
then read this in as the macro grid on p processors, each should get one element and the spherical projection should be 
preserved as I refine it further.

Eike

Dedner, Andreas wrote:
> I might be wrong but I think Yasp only distributes the macro grid like ALU does?
> So if you start with only one element then there is no way to loadbalance it.
> As far as I know only UG differs from this behaviour allowing to loadbalance leaf elements
> "arbitrarily" over the processors....
> ________________________________________
> From: dune-bounces+a.s.dedner=warwick.ac.uk at dune-project.org [dune-bounces+a.s.dedner=warwick.ac.uk at dune-project.org] on behalf of Eike Mueller [E.Mueller at bath.ac.uk]
> Sent: 13 November 2012 11:04
> To: Oliver Sander
> Cc: dune at dune-project.org
> Subject: Re: [Dune] parallel DUNE grid for implementing a spherical shell (UGGrid)
> 
> To make it even simpler, I replaced the UGGrid by an 1x1x1 YaspGrid, i.e. I start with a grid with one element. If I
> globalRefine() this once, and then loadBalance(), one processor always ends up with the entire grid. I must be missing something
> very basic here.
> Does the load balancing in DUNE assume that if a father cell is owned by a processor, then all it's children on the finer levels
> are owned by the same processor? But then calling loadBalance() after grid refinement would not make sense. If I start with a
> 2x2x2 grid and do not refine, then it works, i.e. if I run on 8 cores, then each of them ends up getting one element.
> 
> Thanks,
> 
> Eike
> 
> Oliver Sander wrote:
>> Am 12.11.2012 16:01, schrieb Eike Mueller:
>>> Hi Oliver,
>>>
>>> I tried several other strategies, but without any luck. Whatever I do,
>>> the algorithm seems to refuse to split up the macro cells, i.e. the 6
>>> elements I insert with the grid factory.
>>>
>>> I also tried to simplify the problem as much as possible. I now create
>>> one unit cube with the grid factory, do not insert any boundary
>>> segments and refine the grid by calling globalRefine(refcount), so
>>> that I end up with a cartesian unit cube split into 8^refcount
>>> elements. I then balance the grid with loadBalance() (i.e. no
>>> arguments). I would have thought that that should work. Still, if I
>>> refine 1,2 or 3 times (i.e. I should end up with 8,64,512 elements),
>>> for an 8 core run only one process stores the entire grid.
>> This should really work, can you post your test program?
>>
>> But be careful: If you load-balance small grids then all processors
>> get all elements, but most of them only as ghosts.  Did you
>> check that?
>>
>> cheers,
>> Oliver
>>
>>> Could this be a problem with the grid factory?
>>>
>>> Thanks,
>>>
>>> Eike
>>>
>>> Oliver Sander wrote:
>>>> Am 06.11.2012 08:44, schrieb Markus Blatt:
>>>>> Hey Eike,
>>>>>
>>>>> On Mon, Nov 05, 2012 at 07:17:48PM +0000, Eike Mueller wrote:
>>>>>>> And loadbalancing would only be needed for the macrogrid, e.g.,
>>>>>>> not dynamic.
>>>>>>>
>>>>>> That's right, in the code I would refine the grid until there is
>>>>>> only cell per processor (this is the macrogrid). Then I would call
>>>>>> loadBalance, followed by further grid refinement. So for example
>>>>>> with 24 processors, I would subdivide each of the six cells in the
>>>>>> original grid into four cells, then loadbalance that grid and refine
>>>>>> further.
>>>>> Actually, this approach could be the root of the problem. The
>>>>> loadbalancing is a heuristic algorithm and normally one always gets
>>>>> some load imbalance here. But if you just have as many cells as
>>>>> processor, then naturally some will end up with no cells at all.
>>>> This is true, but it is not the whole truth.
>>>>
>>>> The default load balancing strategy of UG is Recursive Coordinate
>>>> Bisection.
>>>> This means roughly that the grid bounding box is partitionend into
>>>> axis-aligned
>>>> cells, and these cells are assigned to processors.  I reckon (I
>>>> didn't check)
>>>> that since your grid is a hollow sphere, some cells simply remain empty.
>>>>
>>>> UG offers several other strategies, but all this really is hardly
>>>> tested.
>>>> Have a look at the lbs method in ug/parallel/dddif/lb.c:533 for some
>>>> alternatives.
>>>>
>>>> good luck,
>>>> Oliver
>>>>
>>>>> How about doing some more refinement before load balancing?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Markus
>>>>>
>>>>
>>>> _______________________________________________
>>>> Dune mailing list
>>>> Dune at dune-project.org
>>>> http://lists.dune-project.org/mailman/listinfo/dune
>>>
> 
> 
> --
> Dr Eike Mueller
> Research Officer
> 
> Department of Mathematical Sciences
> University of Bath
> Bath BA2 7AY, United Kingdom
> 
> +44 1225 38 5633
> e.mueller at bath.ac.uk
> http://people.bath.ac.uk/em459/
> 
> _______________________________________________
> Dune mailing list
> Dune at dune-project.org
> http://lists.dune-project.org/mailman/listinfo/dune
> 
> 


-- 
Dr Eike Mueller
Research Officer

Department of Mathematical Sciences
University of Bath
Bath BA2 7AY, United Kingdom

+44 1225 38 5633
e.mueller at bath.ac.uk
http://people.bath.ac.uk/em459/