[Dune] Dynamic load balancing with ALUGrid
Bernd Flemisch
bernd at iws.uni-stuttgart.de
Thu Feb 21 17:55:51 CET 2013
Hi,
only a few months have passed and I found the time to look
at this again. I tried to follow Andreas' suggestion and
work with Ids rather than indices, since they should be
persistent. I was hoping that "persistent" means that, if
an element is moved from one process to another, it keeps
its global Id. Looking at
http://www.dune-project.org/doc/doxygen/dune-grid-html/class_dune_1_1_id_set.html#_details
does not necessarily say that (I think).
I set up a test for my efforts, see the attached file. The
coarse grid is 5x1x1. The two left elements are interior
on process 1, the three right ones interior on process 2.
The two left elements get refined. After adaption,
loadBalance with my DataHandle is called. The three right
elements are all moved to process 1, so that after the
balancing step, process 1 holds all elements, while
process 0 has none.
During load balance, I print the cell centers, global Ids,
and gathered/scattered values on each process. This gives
the following:
Process 0:
0: 0.5 0.5 0.5, ([2,3,8,14],8,0): gather 0
0: 0.7 0.5 0.5, ([3,4,9,15],8,0): gather 1
0: 0.9 0.5 0.5, ([4,5,10,16],8,0): gather 2
Process 1:
1: 0.5 0.5 0.5, ([2,3,8,14],71,0): scatter 0
1: 0.7 0.5 0.5, ([2,3,8,14],70,0): scatter 1
1: 0.9 0.5 0.5, ([2,3,8,14],69,0): scatter 2
It seems to be correct that information from three
elements is sent from process 0 and received for the
corresponding elements by process 1. However, the global
Ids are changed. Is this correct? Then how tf can I find
out which elements correspond to each other? Or am I still
doing something wrong?
Thank you for having another look. Kind regards
Bernd
On Sat, 27 Oct 2012 07:43:48 +0000
"Dedner, Andreas" <A.S.Dedner at warwick.ac.uk> wrote:
> Is it not even so that one can not use an undexset at
>all? Load balancing changes the grid so the indexsets are
>not valid. You need to use id containers.
> Andreas
>
> Martin Nolte <nolte at mathematik.uni-freiburg.de> wrote:
> Hi Bernd,
>
> I have an educated guess for you (don't blame me, if I'm
>wrong):
>
> In normal communication, you only communicate data on a
>certain grid view. Hence
> you can use the index set to address data stored on that
>grid view.
>
> When doing load balancing, you rather communicate data
>on the entire hierarchy.
> Therefore, your entity might not be contained an index
>set (leading to this
> cryptic assertion). So, if you want to communicate data
>only on a certain grid
> view, you have to add a call to indexSet.contains to the
>gather / scatter / size
> methods. Usually, fixedSize will be false in such
>situations (I think ALUGrid
> ignores this anyway, so it should not matter, here).
>
> Just a side note: In my opionion, a normal communicate
>method exchanging data on
> the entire hierarchy would be nice, too. But alas, this
>method would naturally
> be "grid().communicate( dataHandle, interface, direction
>)".
>
> Best,
>
> Martin
>
> On 10/26/2012 01:32 PM, Bernd Flemisch wrote:
>> Hey Andreas,
>>
>> thank you for your fast answer. This indeed solves my
>>second issue. However, the
>> first one remains. I attach an updated test file.
>>
>> Kind regards
>> Bernd
>>
>> On 10/26/2012 01:08 PM, Dedner, Andreas wrote:
>>> Hi Bernd
>>>
>>> You should be able to cast the derived class onto the
>>>interface class
>>> and that triggers the call to the correct method.
>>>
>>> Andreas
>>>
>>> ________________________________________
>>> From:
>>>dune-bounces+a.s.dedner=warwick.ac.uk at dune-project.org
>>> [dune-bounces+a.s.dedner=warwick.ac.uk at dune-project.org]
>>>on behalf of Bernd
>>> Flemisch [bernd at iws.uni-stuttgart.de]
>>> Sent: 26 October 2012 11:53
>>> To: Dune
>>> Subject: [Dune] Dynamic load balancing with ALUGrid
>>>
>>> Dear Dune,
>>>
>>> I try to use dynamic load balancing with ALUGrid. The
>>>grid itself gets
>>> balanced nicely, but I don't succeed in getting data
>>>balanced.
>>>
>>> From dune/grid/alugrid/3d/grid.hh lines 701ff, I guess
>>>that I can pack
>>> a "usual" data handle which is used for communication
>>>into a
>>> ALUGridLoadBalanceDataHandle and call loadBalance with
>>>that handle. My
>>> initial attempt to do this fails with
>>> test-dynamiclb:
>>> ../../../dune/grid/alugrid/common/defaultindexsets.hh:218:
>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::IndexType
>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::index(const
>>>typename
>>> GridImp::Codim<cd>::Entity&) const [with int cd = 0;
>>>GridImp =
>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>;
>>>IteratorImp =
>>> Dune::EntityIterator<0, const
>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>,
>>> Dune::ALU3dGridLeafIterator<0,
>>>(Dune::PartitionIteratorType)4u, const
>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u> > >;
>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::IndexType =
>>>unsigned int;
>>> typename GridImp::Codim<cd>::Entity = Dune::Entity<0, 3,
>>>const
>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>,
>>> Dune::ALU3dGridEntity>]: Assertion `indexContainer( cd
>>>)[ en ].index()
>>> >= 0' failed.
>>>
>>> I attach my effort, where I use a data handle copied
>>>from the
>>> grid-howto. It would be great if you could take a look.
>>>
>>> I have a second issue: I guess that I should be able to
>>>call loadBalance
>>> directly with a usual data handle, so that grid.hh lines
>>>701ff takes
>>> care of it and packs it for me into a
>>>ALUGridLoadBalanceDataHandle
>>> before sending it to the real loadBalance method.
>>>However, since my data
>>> handle is only inherited from CommDataHandleIF, the
>>>packing routine is
>>> not triggered but the real loadBalance method is called
>>>directly leading
>>> to a compilation error. How can I trigger the execution
>>>of the packing
>>> routine for such a derived data handle?
>>>
>>> Thank you for having a look. Kind regards
>>> Bernd
>>>
>>> --
>>> _____________________________________________________________________
>>>
>>> Bernd Flemisch phone: +49 711 685 69162
>>> IWS, Universität Stuttgart fax: +49 711 685 60430
>>> Pfaffenwaldring 61 email: bernd at iws.uni-stuttgart.de
>>> D-70569 Stuttgart url:
>>>www.hydrosys.uni-stuttgart.de<http://www.hydrosys.uni-stuttgart.de>
>>> _____________________________________________________________________
>>
>>
>>
>>
>> _______________________________________________
>> Dune mailing list
>> Dune at dune-project.org
>> http://lists.dune-project.org/mailman/listinfo/dune
>
> --
> Dr. Martin Nolte <nolte at mathematik.uni-freiburg.de>
>
> Universität Freiburg
> phone: +49-761-203-5630
> Abteilung für angewandte Mathematik
> fax: +49-761-203-5632
> Hermann-Herder-Straße 10
> 79104 Freiburg, Germany
>
> _______________________________________________
> Dune mailing list
> Dune at dune-project.org
> http://lists.dune-project.org/mailman/listinfo/dune
>
___________________________________________________________
Bernd Flemisch phone: +49 711 685
69162
IWS, Universitaet Stuttgart fax: +49 711 685
67020
Pfaffenwaldring 61 email:
bernd at iws.uni-stuttgart.de
D-70569 Stuttgart url:
www.hydrosys.uni-stuttgart.de
___________________________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-dynamiclb.cc
Type: text/x-c++src
Size: 6186 bytes
Desc: not available
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20130221/ee6da63a/attachment.cc>
More information about the Dune
mailing list