[Dune] Dynamic load balancing with ALUGrid
Bernd Flemisch
bernd at iws.uni-stuttgart.de
Thu Feb 21 19:12:44 CET 2013
Actually, the Ids are the same again after the load
balancing step is over. But apparently not during scatter,
where I would need it. Hmmm...
On Thu, 21 Feb 2013 17:55:51 +0100
Bernd Flemisch <bernd at iws.uni-stuttgart.de> wrote:
> Hi,
>
> only a few months have passed and I found the time to
>look at this again. I tried to follow Andreas' suggestion
>and work with Ids rather than indices, since they should
>be persistent. I was hoping that "persistent" means that,
>if an element is moved from one process to another, it
>keeps its global Id. Looking at
> http://www.dune-project.org/doc/doxygen/dune-grid-html/class_dune_1_1_id_set.html#_details
> does not necessarily say that (I think).
>
> I set up a test for my efforts, see the attached file.
>The coarse grid is 5x1x1. The two left elements are
>interior on process 1, the three right ones interior on
>process 2. The two left elements get refined. After
>adaption, loadBalance with my DataHandle is called. The
>three right elements are all moved to process 1, so that
>after the balancing step, process 1 holds all elements,
>while process 0 has none.
>
> During load balance, I print the cell centers, global
>Ids, and gathered/scattered values on each process. This
>gives the following:
> Process 0:
> 0: 0.5 0.5 0.5, ([2,3,8,14],8,0): gather 0
> 0: 0.7 0.5 0.5, ([3,4,9,15],8,0): gather 1
> 0: 0.9 0.5 0.5, ([4,5,10,16],8,0): gather 2
> Process 1:
> 1: 0.5 0.5 0.5, ([2,3,8,14],71,0): scatter 0
> 1: 0.7 0.5 0.5, ([2,3,8,14],70,0): scatter 1
> 1: 0.9 0.5 0.5, ([2,3,8,14],69,0): scatter 2
> It seems to be correct that information from three
>elements is sent from process 0 and received for the
>corresponding elements by process 1. However, the global
>Ids are changed. Is this correct? Then how tf can I find
>out which elements correspond to each other? Or am I
>still doing something wrong?
>
> Thank you for having another look. Kind regards
> Bernd
>
> On Sat, 27 Oct 2012 07:43:48 +0000
> "Dedner, Andreas" <A.S.Dedner at warwick.ac.uk> wrote:
>> Is it not even so that one can not use an undexset at
>>all? Load balancing changes the grid so the indexsets are
>>not valid. You need to use id containers.
>> Andreas
>>
>> Martin Nolte <nolte at mathematik.uni-freiburg.de> wrote:
>> Hi Bernd,
>>
>> I have an educated guess for you (don't blame me, if I'm
>>wrong):
>>
>> In normal communication, you only communicate data on a
>>certain grid view. Hence
>> you can use the index set to address data stored on that
>>grid view.
>>
>> When doing load balancing, you rather communicate data
>>on the entire hierarchy.
>> Therefore, your entity might not be contained an index
>>set (leading to this
>> cryptic assertion). So, if you want to communicate data
>>only on a certain grid
>> view, you have to add a call to indexSet.contains to the
>>gather / scatter / size
>> methods. Usually, fixedSize will be false in such
>>situations (I think ALUGrid
>> ignores this anyway, so it should not matter, here).
>>
>> Just a side note: In my opionion, a normal communicate
>>method exchanging data on
>> the entire hierarchy would be nice, too. But alas, this
>>method would naturally
>> be "grid().communicate( dataHandle, interface, direction
>>)".
>>
>> Best,
>>
>> Martin
>>
>> On 10/26/2012 01:32 PM, Bernd Flemisch wrote:
>>> Hey Andreas,
>>>
>>> thank you for your fast answer. This indeed solves my
>>>second issue. However, the
>>> first one remains. I attach an updated test file.
>>>
>>> Kind regards
>>> Bernd
>>>
>>> On 10/26/2012 01:08 PM, Dedner, Andreas wrote:
>>>> Hi Bernd
>>>>
>>>> You should be able to cast the derived class onto the
>>>>interface class
>>>> and that triggers the call to the correct method.
>>>>
>>>> Andreas
>>>>
>>>> ________________________________________
>>>> From:
>>>>dune-bounces+a.s.dedner=warwick.ac.uk at dune-project.org
>>>> [dune-bounces+a.s.dedner=warwick.ac.uk at dune-project.org]
>>>>on behalf of Bernd
>>>> Flemisch [bernd at iws.uni-stuttgart.de]
>>>> Sent: 26 October 2012 11:53
>>>> To: Dune
>>>> Subject: [Dune] Dynamic load balancing with ALUGrid
>>>>
>>>> Dear Dune,
>>>>
>>>> I try to use dynamic load balancing with ALUGrid. The
>>>>grid itself gets
>>>> balanced nicely, but I don't succeed in getting data
>>>>balanced.
>>>>
>>>> From dune/grid/alugrid/3d/grid.hh lines 701ff, I guess
>>>>that I can pack
>>>> a "usual" data handle which is used for communication
>>>>into a
>>>> ALUGridLoadBalanceDataHandle and call loadBalance with
>>>>that handle. My
>>>> initial attempt to do this fails with
>>>> test-dynamiclb:
>>>> ../../../dune/grid/alugrid/common/defaultindexsets.hh:218:
>>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::IndexType
>>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::index(const
>>>>typename
>>>> GridImp::Codim<cd>::Entity&) const [with int cd = 0;
>>>>GridImp =
>>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>;
>>>>IteratorImp =
>>>> Dune::EntityIterator<0, const
>>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>,
>>>> Dune::ALU3dGridLeafIterator<0,
>>>>(Dune::PartitionIteratorType)4u, const
>>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u> > >;
>>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::IndexType =
>>>>unsigned int;
>>>> typename GridImp::Codim<cd>::Entity = Dune::Entity<0, 3,
>>>>const
>>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>,
>>>> Dune::ALU3dGridEntity>]: Assertion `indexContainer( cd
>>>>)[ en ].index()
>>>> >= 0' failed.
>>>>
>>>> I attach my effort, where I use a data handle copied
>>>>from the
>>>> grid-howto. It would be great if you could take a look.
>>>>
>>>> I have a second issue: I guess that I should be able to
>>>>call loadBalance
>>>> directly with a usual data handle, so that grid.hh lines
>>>>701ff takes
>>>> care of it and packs it for me into a
>>>>ALUGridLoadBalanceDataHandle
>>>> before sending it to the real loadBalance method.
>>>>However, since my data
>>>> handle is only inherited from CommDataHandleIF, the
>>>>packing routine is
>>>> not triggered but the real loadBalance method is called
>>>>directly leading
>>>> to a compilation error. How can I trigger the execution
>>>>of the packing
>>>> routine for such a derived data handle?
>>>>
>>>> Thank you for having a look. Kind regards
>>>> Bernd
>>>>
>>>> --
>>>> _____________________________________________________________________
>>>>
>>>> Bernd Flemisch phone: +49 711 685 69162
>>>> IWS, Universität Stuttgart fax: +49 711 685 60430
>>>> Pfaffenwaldring 61 email: bernd at iws.uni-stuttgart.de
>>>> D-70569 Stuttgart url:
>>>>www.hydrosys.uni-stuttgart.de<http://www.hydrosys.uni-stuttgart.de>
>>>> _____________________________________________________________________
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Dune mailing list
>>> Dune at dune-project.org
>>> http://lists.dune-project.org/mailman/listinfo/dune
>>
>> --
>> Dr. Martin Nolte <nolte at mathematik.uni-freiburg.de>
>>
>> Universität Freiburg
>> phone: +49-761-203-5630
>> Abteilung für angewandte Mathematik
>> fax: +49-761-203-5632
>> Hermann-Herder-Straße 10
>> 79104 Freiburg, Germany
>>
>> _______________________________________________
>> Dune mailing list
>> Dune at dune-project.org
>> http://lists.dune-project.org/mailman/listinfo/dune
>>
>
> ___________________________________________________________
>
> Bernd Flemisch phone: +49 711 685
>69162
> IWS, Universitaet Stuttgart fax: +49 711 685
>67020
> Pfaffenwaldring 61 email:
>bernd at iws.uni-stuttgart.de
> D-70569 Stuttgart url:
>www.hydrosys.uni-stuttgart.de
> ___________________________________________________________
___________________________________________________________
Bernd Flemisch phone: +49 711 685
69162
IWS, Universitaet Stuttgart fax: +49 711 685
67020
Pfaffenwaldring 61 email:
bernd at iws.uni-stuttgart.de
D-70569 Stuttgart url:
www.hydrosys.uni-stuttgart.de
___________________________________________________________
More information about the Dune
mailing list