[Dune] Dynamic load balancing with ALUGrid

Sat Oct 27 09:37:22 CEST 2012

Hey Martin,

thank you for your educated guess, which proves to be 
correct!

Following your suggestions gets the program quite a bit 
further. It now fails in the duneUnpackAll routine of 
ALUGrid's duneinterface/gitter_dune_pll_mgb.cc (full stack 
below and test program attached). Apparently, packing went 
well, and it already unpacked several (I don't think all) 
of the hexahedrons together with the correct data. Now it 
is surprised because something is to be unpacked which 
apparently is not associated with a valid entity type.

It would be great if you could invest another look. Thank 
you. Kind regards
Bernd

**FEHLER (FATAL) Unbekannte Gitterobjekt-Codierung gelesen 
[16843010] on p = 1
   Weitermachen unm"oglich. In 
duneinterface/gitter_dune_pll_mgb.cc 371
**FEHLER (FATAL) Unbekannte Gitterobjekt-Codierung gelesen 
[33] on p = 0
   Weitermachen unm"oglich. In 
duneinterface/gitter_dune_pll_mgb.cc 371
test-dynamiclb: duneinterface/gitter_dune_pll_mgb.cc:372: 
void 
ALUGridSpace::DuneParallelGridMover::duneUnpackAll(std::vector<ALUGridSpace::ObjectStream>&, 
ALUGridSpace::GatherScatterType&): Assertion `false' 
failed.
[lantula:29991] *** Process received signal ***
[lantula:29991] Signal: Aborted (6)
[lantula:29991] Signal code:  (-727883773)
test-dynamiclb: duneinterface/gitter_dune_pll_mgb.cc:372: 
void 
ALUGridSpace::DuneParallelGridMover::duneUnpackAll(std::vector<ALUGridSpace::ObjectStream>&, 
ALUGridSpace::GatherScatterType&): Assertion `false' 
failed.
[lantula:29990] *** Process received signal ***
[lantula:29990] Signal: Aborted (6)
[lantula:29990] Associated errno: Unknown error 32767 
(32767)
[lantula:29990] Signal code:  (9631766)
[lantula:29990] [ 0] /lib64/libc.so.6(+0x34da0) 
[0x7fa56b83eda0]
[lantula:29990] [ 1] /lib64/libc.so.6(gsignal+0x35) 
[0x7fa56b83ed25]
[lantula:29990] [ 2] /lib64/libc.so.6(abort+0x148) 
[0x7fa56b8401a8]
[lantula:29990] [ 3] /lib64/libc.so.6(+0x2dc22) 
[0x7fa56b837c22]
[lantula:29990] [ 4] /lib64/libc.so.6(+0x2dcd2) 
[0x7fa56b837cd2]
[lantula:29990] [ 5] 
./test-dynamiclb(_ZN12ALUGridSpace21DuneParallelGridMover13duneUnpackAllERSt6vectorINS_12ObjectStreamESaIS2_EERNS_13GatherScatterE+0x242) 
[0x9f4c30]
[lantula:29990] [ 6] 
./test-dynamiclb(_ZN12ALUGridSpace13GitterDunePll24duneRepartitionMacroGridERNS_12LoadBalancer8DataBaseERNS_13GatherScatterE+0x751) 
[0x9dc271]
[lantula:29990] [ 7] 
./test-dynamiclb(_ZN12ALUGridSpace13GitterDunePll15duneLoadBalanceERNS_13GatherScatterERNS_6Gitter20AdaptRestrictProlongE+0x4fb) 
[0x9dd447]
[lantula:29990] [ 8] 
./test-dynamiclb(_ZN4Dune19ALU3dGridCommHelperILNS_20ALU3dGridElementTypeE7EP19ompi_communicator_tE11loadBalanceINS_28ALUGridLoadBalanceDataHandleINS_9ALU3dGridILS1_7ES3_EE14VectorExchangeINS_8IndexSetIS8_NS_15DefaultIndexSetIS8_NS_14EntityIteratorILi0EKS8_NS_21ALU3dGridLeafIteratorILi0ELNS_21PartitionIteratorTypeE4ESD_EEEEEEjEESt6vectorIiSaIiEEEiEEEEbRS8_RT_+0x27a) 
[0x8b0de6]
[lantula:29990] [ 9] 
./test-dynamiclb(_ZN4Dune9ALU3dGridILNS_20ALU3dGridElementTypeE7EP19ompi_communicator_tE11loadBalanceINS_28ALUGridLoadBalanceDataHandleIS4_14VectorExchangeINS_8IndexSetIS4_NS_15DefaultIndexSetIS4_NS_14EntityIteratorILi0EKS4_NS_21ALU3dGridLeafIteratorILi0ELNS_21PartitionIteratorTypeE4ESB_EEEEEEjEESt6vectorIiSaIiEEEiEEEEbRT_+0x23) 
[0x8a9ff1]
[lantula:29990] [10] 
./test-dynamiclb(_ZN4Dune9ALU3dGridILNS_20ALU3dGridElementTypeE7EP19ompi_communicator_tE11loadBalanceI14VectorExchangeINS_8IndexSetIS4_NS_15DefaultIndexSetIS4_NS_14EntityIteratorILi0EKS4_NS_21ALU3dGridLeafIteratorILi0ELNS_21PartitionIteratorTypeE4ESA_EEEEEEjEESt6vectorIiSaIiEEEiEEbRNS_16CommDataHandleIFIT_T0_EE+0x3a) 
[0x8a3a3c]
[lantula:29990] [11] ./test-dynamiclb(main+0x4c9) 
[0x892fbe]
[lantula:29990] [12] 
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fa56b82b455]
[lantula:29990] [13] ./test-dynamiclb() [0x8928cd]
[lantula:29990] *** End of error message ***
[lantula:29991] [ 0] /lib64/libc.so.6(+0x34da0) 
[0x7f50d3ea6da0]
[lantula:29991] [ 1] /lib64/libc.so.6(gsignal+0x35) 
[0x7f50d3ea6d25]
[lantula:29991] [ 2] /lib64/libc.so.6(abort+0x148) 
[0x7f50d3ea81a8]
[lantula:29991] [ 3] /lib64/libc.so.6(+0x2dc22) 
[0x7f50d3e9fc22]
[lantula:29991] [ 4] /lib64/libc.so.6(+0x2dcd2) 
[0x7f50d3e9fcd2]
[lantula:29991] [ 5] 
./test-dynamiclb(_ZN12ALUGridSpace21DuneParallelGridMover13duneUnpackAllERSt6vectorINS_12ObjectStreamESaIS2_EERNS_13GatherScatterE+0x242) 
[0x9f4c30]
[lantula:29991] [ 6] 
./test-dynamiclb(_ZN12ALUGridSpace13GitterDunePll24duneRepartitionMacroGridERNS_12LoadBalancer8DataBaseERNS_13GatherScatterE+0x751) 
[0x9dc271]
[lantula:29991] [ 7] 
./test-dynamiclb(_ZN12ALUGridSpace13GitterDunePll15duneLoadBalanceERNS_13GatherScatterERNS_6Gitter20AdaptRestrictProlongE+0x4fb) 
[0x9dd447]
[lantula:29991] [ 8] 
./test-dynamiclb(_ZN4Dune19ALU3dGridCommHelperILNS_20ALU3dGridElementTypeE7EP19ompi_communicator_tE11loadBalanceINS_28ALUGridLoadBalanceDataHandleINS_9ALU3dGridILS1_7ES3_EE14VectorExchangeINS_8IndexSetIS8_NS_15DefaultIndexSetIS8_NS_14EntityIteratorILi0EKS8_NS_21ALU3dGridLeafIteratorILi0ELNS_21PartitionIteratorTypeE4ESD_EEEEEEjEESt6vectorIiSaIiEEEiEEEEbRS8_RT_+0x27a) 
[0x8b0de6]
[lantula:29991] [ 9] 
./test-dynamiclb(_ZN4Dune9ALU3dGridILNS_20ALU3dGridElementTypeE7EP19ompi_communicator_tE11loadBalanceINS_28ALUGridLoadBalanceDataHandleIS4_14VectorExchangeINS_8IndexSetIS4_NS_15DefaultIndexSetIS4_NS_14EntityIteratorILi0EKS4_NS_21ALU3dGridLeafIteratorILi0ELNS_21PartitionIteratorTypeE4ESB_EEEEEEjEESt6vectorIiSaIiEEEiEEEEbRT_+0x23) 
[0x8a9ff1]
[lantula:29991] [10] 
./test-dynamiclb(_ZN4Dune9ALU3dGridILNS_20ALU3dGridElementTypeE7EP19ompi_communicator_tE11loadBalanceI14VectorExchangeINS_8IndexSetIS4_NS_15DefaultIndexSetIS4_NS_14EntityIteratorILi0EKS4_NS_21ALU3dGridLeafIteratorILi0ELNS_21PartitionIteratorTypeE4ESA_EEEEEEjEESt6vectorIiSaIiEEEiEEbRNS_16CommDataHandleIFIT_T0_EE+0x3a) 
[0x8a3a3c]
[lantula:29991] [11] ./test-dynamiclb(main+0x4c9) 
[0x892fbe]
[lantula:29991] [12] 
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f50d3e93455]
[lantula:29991] [13] ./test-dynamiclb() [0x8928cd]
[lantula:29991] *** End of error message ***

On Fri, 26 Oct 2012 21:06:13 +0200
  Martin Nolte <nolte at mathematik.uni-freiburg.de> wrote:
> Hi Bernd,
> 
> I have an educated guess for you (don't blame me, if I'm 
>wrong):
> 
> In normal communication, you only communicate data on a 
>certain grid view. Hence you can use the index set to 
>address data stored on that grid view.
> 
> When doing load balancing, you rather communicate data 
>on the entire hierarchy. Therefore, your entity might not 
>be contained an index set (leading to this cryptic 
>assertion). So, if you want to communicate data only on a 
>certain grid view, you have to add a call to 
>indexSet.contains to the gather / scatter / size methods. 
>Usually, fixedSize will be false in such situations (I 
>think ALUGrid ignores this anyway, so it should not 
>matter, here).
> 
> Just a side note: In my opionion, a normal communicate 
>method exchanging data on the entire hierarchy would be 
>nice, too. But alas, this method would naturally be 
>"grid().communicate( dataHandle, interface, direction )".
> 
> Best,
> 
> Martin
> 
> On 10/26/2012 01:32 PM, Bernd Flemisch wrote:
>> Hey Andreas,
>>
>> thank you for your fast answer. This indeed solves my 
>>second issue. However, the
>> first one remains. I attach an updated test file.
>>
>> Kind regards
>> Bernd
>>
>> On 10/26/2012 01:08 PM, Dedner, Andreas wrote:
>>> Hi Bernd
>>>
>>> You should be able to cast the derived class onto the 
>>>interface class
>>> and that triggers the call to the correct method.
>>>
>>> Andreas
>>>
>>> ________________________________________
>>> From: 
>>>dune-bounces+a.s.dedner=warwick.ac.uk at dune-project.org
>>> [dune-bounces+a.s.dedner=warwick.ac.uk at dune-project.org] 
>>>on behalf of Bernd
>>> Flemisch [bernd at iws.uni-stuttgart.de]
>>> Sent: 26 October 2012 11:53
>>> To: Dune
>>> Subject: [Dune] Dynamic load balancing with ALUGrid
>>>
>>> Dear Dune,
>>>
>>> I try to use dynamic load balancing with ALUGrid. The 
>>>grid itself gets
>>> balanced nicely, but I don't succeed in getting data 
>>>balanced.
>>>
>>> From dune/grid/alugrid/3d/grid.hh lines 701ff, I guess 
>>>that I can pack
>>> a "usual" data handle which is used for communication 
>>>into a
>>> ALUGridLoadBalanceDataHandle and call loadBalance with 
>>>that handle. My
>>> initial attempt to do this fails with
>>> test-dynamiclb:
>>> ../../../dune/grid/alugrid/common/defaultindexsets.hh:218:
>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::IndexType
>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::index(const 
>>>typename
>>> GridImp::Codim<cd>::Entity&) const [with int cd = 0; 
>>>GridImp =
>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>; 
>>>IteratorImp =
>>> Dune::EntityIterator<0, const
>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>,
>>> Dune::ALU3dGridLeafIterator<0, 
>>>(Dune::PartitionIteratorType)4u, const
>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u> > >;
>>> Dune::DefaultIndexSet<GridImp, IteratorImp>::IndexType = 
>>>unsigned int;
>>> typename GridImp::Codim<cd>::Entity = Dune::Entity<0, 3, 
>>>const
>>> Dune::ALU3dGrid<(Dune::ALU3dGridElementType)7u>,
>>> Dune::ALU3dGridEntity>]: Assertion `indexContainer( cd 
>>>)[ en ].index()
>>> >= 0' failed.
>>>
>>> I attach my effort, where I use a data handle copied 
>>>from the
>>> grid-howto. It would be great if you could take a look.
>>>
>>> I have a second issue: I guess that I should be able to 
>>>call loadBalance
>>> directly with a usual data handle, so that grid.hh lines 
>>>701ff takes
>>> care of it and packs it for me into a 
>>>ALUGridLoadBalanceDataHandle
>>> before sending it to the real loadBalance method. 
>>>However, since my data
>>> handle is only inherited from CommDataHandleIF, the 
>>>packing routine is
>>> not triggered but the real loadBalance method is called 
>>>directly leading
>>> to a compilation error. How can I trigger the execution 
>>>of the packing
>>> routine for such a derived data handle?
>>>
>>> Thank you for having a look. Kind regards
>>> Bernd
>>>
>>> --
>>> _____________________________________________________________________
>>>
>>> Bernd Flemisch phone: +49 711 685 69162
>>> IWS, Universität Stuttgart fax: +49 711 685 60430
>>> Pfaffenwaldring 61 email: bernd at iws.uni-stuttgart.de
>>> D-70569 Stuttgart url: www.hydrosys.uni-stuttgart.de
>>> _____________________________________________________________________
>>
>>
>>
>>
>> _______________________________________________
>> Dune mailing list
>> Dune at dune-project.org
>> http://lists.dune-project.org/mailman/listinfo/dune
> 
> -- 
> Dr. Martin Nolte <nolte at mathematik.uni-freiburg.de>
> 
> Universität Freiburg 
>                                  phone: +49-761-203-5630
> Abteilung für angewandte Mathematik 
>                   fax:   +49-761-203-5632
> Hermann-Herder-Straße 10
> 79104 Freiburg, Germany
> 
> _______________________________________________
> Dune mailing list
> Dune at dune-project.org
> http://lists.dune-project.org/mailman/listinfo/dune

___________________________________________________________

Bernd Flemisch                     phone: +49 711 685 
69162
IWS, Universitaet Stuttgart          fax: +49 711 685 
67020
Pfaffenwaldring 61        email: 
bernd at iws.uni-stuttgart.de
D-70569 Stuttgart        url: 
www.hydrosys.uni-stuttgart.de
___________________________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-dynamiclb.cc
Type: text/x-c++src
Size: 3963 bytes
Desc: not available
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20121027/1e907e23/attachment.cc>