[Dune] load balancing with dune-alugrid

[Alexander Kissinger]

Dear Dune,

I am using Dumux and I want to run parallel simulations on our cluster 
using dune-alugrid as a grid manager. When I use the internal load 
balancing methods of dune-alugrid I notice when looking at the ranks in 
my vtu ouput that partitioning only takes place in one spatial 
direction. Back when ALUGrid was an external module I used ParMetis for 
load balancing and was happy with it.
I am a bit confused about what options I actually have with 
dune-alugrid. Looking at the alugrid.cfg-file I see that besides the the 
internal methods there still exist METIS_PartGraphKway and 
METIS_PartGraphRecursive.

I tried the following:
1) installing METIS-5.1.0, according to the cmake log (see attached) 
dune-alugrid finds Metis. However when running a simulation and setting 
METIS_PartGraphKway or METIS_PartGraphRecursive in the cfg file I get 
the error:

**ERROR The use of METIS_PartGraphRecursive is not supported, when the 
METIS library is missing!  in: 
/nfs/home_simtech/iwskissi/DUMUX_DUNE_TRUNK/dune-alugrid/dune/alugrid/impl/parallel/alumetis.hh 
line: 137
**INFO Due to license reasons the library METIS is
        not part of the ALUGrid library distribution.
        To use this feature get a copy of the METIS library
        (see http://www-users.cs.umn.edu/~karypis/metis/metis/ )
        and re-configure the ALUGrid library with the
        --with-metis=PATH_TO_METIS option,
        or choose another Graph partitioning method.
        Exiting program, bye!

2) I also tried installing ParMETIS 3.2 with its version of Metis which 
leads to the same error (I have not attached the cmake log).

- Are METIS_PartGraphKway and METIS_PartGraphRecursive still working 
with dune-alugrid?
- Is ParMETIS only working in combination the package ZOLTAN now?

I would appreciate any advice on what might be a promising way to go. I 
also tried using ZOLTAN with ParMETIS, but this I think this mail is 
already long enough :-)

The system I use is:
gcc-4-9-3
openmpi 1.8.6
trunk version of all dune modules (including dune-alugrid)
trunk version of dumux
METIS 5.1.0
ParMETIS 3.2

Thank you!

Best regards
Alex








-- 
Alexander Kissinger
Institut für Wasser- und Umweltsystemmodellierung
Lehrstuhl für Hydromechanik und Hydrosystemmodellierung
Pfaffenwaldring 61
D-70569 Stuttgart

Telefon: +49 (0) 711 685-64729
E-Mail:  alexander.kissinger at iws.uni-stuttgart.de

-------------- next part --------------
EXTDIR=~/DUMUX_DUNE_TRUNK/external

GXX_WARNING_OPTS=" \
    -Wall \
    -Wmissing-include-dirs \
    -Wcast-align \
    -Wno-sign-compare \
    -pedantic \
    -Wno-unused-parameter"

GXX_OPTS=" \
    -fno-strict-aliasing \
    -fstrict-overflow \
    -ffast-math \
    -fno-finite-math-only \
    -O3 \
    -march=native \
    -DNDEBUG=1"

BUILDDIR=build-gcc-4-9-3_metis-5-1-0
 
CONFIGURE_FLAGS="--enable-parallel"

USE_CMAKE=yes

CMAKE_FLAGS=" \
    -DCMAKE_CXX_COMPILER=/sw/simtech/apps/gcc/4.9/3/bin/g++ \
    -DCMAKE_C_COMPILER=/sw/simtech/apps/gcc/4.9/3/bin/gcc \
    -DCMAKE_CXX_FLAGS=\"$GXX_WARNING_OPTS $GXX_OPTS\" \
    -DDUNE_GRID_EXPERIMENTAL_GRID_EXTENSIONS=TRUE \
    -DBoost_NO_BOOST_CMAKE=ON \
    -DMETIS_LIBRARY=$EXTDIR/metis-5.1.0/build/Linux-x86_64/libmetis/libmetis.a \
    -DMETIS_ROOT=$EXTDIR/metis-5.1.0 \
"

# Grids
#   -DALBERTA_ROOT=$EXTDIR/alberta-3.x-20120926-3742 \
#   -DALUGRID_ROOT=$EXTDIR/ALUGrid-1.52 \
#   -DUG_DIR=$EXTDIR/ug-3.11.0 \

# UMFPack should be installed for openSuse and package should be available for ubuntu
#   -DUMFPACK_LIBRARIES=/usr/lib64/ \
#   -DUMFPACK_INCLUDE_DIRS=/usr/include/suitesparse/ \

# SuperLU
# SuperLU should be installed for openSuse and package should be available for ubuntu
#   -DSUPERLU_ROOT=$EXTDIR/SuperLU_4.3 \

# Experimental Grid Extensions
#  --enable-experimental-grid-extensions \ 
#  -DCMAKE_DISABLE_FIND_PACKAGE_MPI=FALSE \
#    -DCMAKE_CXX_COMPILER=/sw/simtech/apps/gcc/4.9/0p/bin/g++ \
#    -DCMAKE_C_COMPILER=/sw/simtech/apps/gcc/4.9/0p/bin/gcc \
#    -DALUGRID_ROOT=$EXTDIR/ALUGrid-1.52 \
#    -DUG_DIR=$EXTDIR/ug-3.11.0 \

#    -DMETIS_ROOT=$EXTDIR/ParMetis-3.1 \
#    -DPARMETIS_ROOT=$EXTDIR/ParMetis-3.1 \
#    -DZOLTAN_ROOT=$EXTDIR/Zoltan_v3.82/build-autotools/ \
#    -DMETIS_LIBRARY=$EXTDIR/metis-5.1.0/libmetis \

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cmake_config_metis-5-1-0.log
Type: text/x-log
Size: 158173 bytes
Desc: not available
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20150715/e5ec099a/attachment.bin>