[dune-pdelab] Petsc backend

[Steffen Müthing]

Hi Markus,

I'm afraid this won't work at all with MPI. It was meant as a proof-of-concept from
the outset, and I did not have the time to get it working in parallel. There are some
support bits in there, but the integration is a little delicate because Petsc uses global
vectors and matrices as opposed to the local ones in ISTL.

The main reason I didn't want to waste a whole lot of effort is that I have the strong impression
that proper (i.e. production-quality) support for Petsc will require a major rethinking of the
current minimalistic LA container backends (I think I posted a mail to the list about it some months
ago). My motivation for getting Petsc going was its support for nested containers in order to
implement block preconditioners for coupled systems - that works (see the Nested... backends),
but is a real kludge because currently the backend only gets the completely flattened index from
the GFS and then has to use lookup tables to find the correct subvector / submatrix for a given DOF.

So, long story short: I'm sorry, but there is absolutely no way the backend is going to work with MPI.
That said, I haven't used the backend for some time now, and the errors look more like the problem
I already had with Petsc when writing the backend: The Petsc developers seem to make a sport of
constantly changing some function signatures. What is the version of Petsc you are using? You need
at least something like 3.1.9x, 3.2 would be better.

Steffen


Am 14.12.2011 um 22:11 schrieb Markus Blatt:

> Hi,
> 
> I am having a hard time trying to test the Petsc backend. After fixing
> the Petsc test my installation is now found by configure.
> 
> Now I get compile errors like below and am wondering what Version of
> Petsc and MPI this is tested with. At least with OpenMPI the current
> version seems to have problems.
> 
> Markus
> 
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab/backend/petscnestedvectorbackend.hh:
> In destructor ‘Dune::PDELab::PetscNestedVectorContainer::~PetscNestedVectorContainer()’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab/backend/petscnestedvectorbackend.hh:111:9:
> error: cannot convert ‘_p_Vec**’ to ‘Vec {aka _p_Vec*}’ for argument
> ‘1’ to ‘PetscErrorCode VecDestroy(Vec)’
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab/backend/petscnestedvectorbackend.hh:
> In member function ‘Dune::PDELab::PetscVectorContainer Dune::PDELab::PetscNestedVectorContainer::subVector(Dune::PDELab::PetscNestedVectorContainer::size_type)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab/backend/petscnestedvectorbackend.hh:278:9:
> error: ‘VecNestGetSubVec’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab/backend/petscnestedvectorbackend.hh:
> In member function ‘void
> Dune::PDELab::PetscNestedVectorContainer::checkin() const’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab/backend/petscnestedvectorbackend.hh:294:13:
> error: ‘VecNestGetSubVecs’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab/backend/petscnestedvectorbackend.hh:
> In member function ‘void
> Dune::PDELab::PetscNestedVectorContainer::checkout() const’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab/backend/petscnestedvectorbackend.hh:315:13:
> error: ‘VecNestGetSubVecs’ was not declared in this scope
> In file included from petscsinglephaseflowccfv.cc:42:0:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In constructor
> ‘Dune::PDELab::PetscMatrixContainer::PetscMatrixContainer(Dune::PDELab::petsc_types::size_type,
> Dune::PDELab::petsc_types::size_type, const std::vector<int>&)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:230:9:
> error: invalid conversion from ‘Dune::PDELab::petsc_types::size_type
> {aka long unsigned int}’ to ‘MPI_Comm {aka ompi_communicator_t*}’
> [-fpermissive]
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:230:9:
> error: invalid conversion from ‘const int*’ to ‘PetscInt {aka int}’
> [-fpermissive]
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:230:9:
> error: invalid conversion from ‘_p_Mat**’ to ‘PetscInt {aka int}’
> [-fpermissive]
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:230:9:
> error: too few arguments to function ‘PetscErrorCode
> MatCreateSeqAIJ(MPI_Comm, PetscInt, PetscInt, PetscInt, const
> PetscInt*, _p_Mat**)’
> /usr/lib/petscdir/3.1/include/petscmat.h:226:42: note: declared here
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:232:9:
> error: ‘MAT_NO_OFF_PROC_ZERO_ROWS’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In copy constructor
> ‘Dune::PDELab::PetscMatrixContainer::PetscMatrixContainer(const Dune::PDELab::PetscMatrixContainer&)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:242:9:
> error: ‘MAT_NO_OFF_PROC_ZERO_ROWS’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In destructor ‘virtual Dune::PDELab::PetscMatrixContainer::~PetscMatrixContainer()’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:266:11:
> error: cannot convert ‘_p_Mat**’ to ‘Mat {aka _p_Mat*}’ for argument
> ‘1’ to ‘PetscErrorCode MatDestroy(Mat)’
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In member function ‘virtual void Dune::PDELab::PetscMatrixContainer::flush(MatAssemblyType)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:354:17:
> error: too many arguments to function ‘PetscErrorCode MatZeroRows(Mat,
> PetscInt, const PetscInt*, PetscScalar)’
> /usr/lib/petscdir/3.1/include/petscmat.h:557:42: note: declared here
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In constructor
> ‘Dune::PDELab::PetscNestedMatrixContainer::PetscNestedMatrixContainer(Dune::PDELab::petsc_types::size_type,
> Dune::PDELab::petsc_types::size_type, const MatrixArray&)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:523:9:
> error: ‘MatCreateNest’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:525:9:
> error: ‘MAT_NO_OFF_PROC_ZERO_ROWS’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In constructor
> ‘Dune::PDELab::PetscNestedMatrixContainer::PetscNestedMatrixContainer(const GO&)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:540:9:
> error: ‘MAT_NO_OFF_PROC_ZERO_ROWS’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In member function ‘Dune::PDELab::PetscMatrixContainer
> Dune::PDELab::PetscNestedMatrixContainer::subMatrix(Dune::PDELab::petsc_types::size_type, Dune::PDELab::petsc_types::size_type)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:550:9:
> error: ‘MatNestGetSubMat’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In member function ‘virtual void Dune::PDELab::PetscNestedMatrixContainer::flush(MatAssemblyType)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:567:13:
> error: ‘MatNestGetSubMats’ was not declared in this scope
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:600:27:
> error: too many arguments to function ‘PetscErrorCode MatZeroRows(Mat,
> PetscInt, const PetscInt*, PetscScalar)’
> /usr/lib/petscdir/3.1/include/petscmat.h:557:42: note: declared here
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:
> In member function ‘Dune::PDELab::PetscMatrixAccessorBase&
> Dune::PDELab::PetscNestedMatrixAccessor<LFSV,
> LFSU>::accessor(Dune::PDELab::petsc_types::size_type&, Dune::PDELab::petsc_types::size_type&)’:
> /home/mblatt/src/dune/current/dune-pdelab/dune/pdelab//backend/petscmatrixbackend.hh:753:13:
> error: there are no arguments to ‘MatNestGetSubMat’ that depend on a
> template parameter, so a declaration of ‘MatNestGetSubMat’ must be
> available [-fpermissive]
> 
> 
> -- 
> Do you need more support with DUNE or HPC in general? 
> 
> Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de
> Rappoltsweilerstr. 5, 68229 Mannheim, Germany
> Tel.: +49 (0) 160 97590858  Fax: +49 (0)322 1108991658 
> 
> _______________________________________________
> dune-pdelab mailing list
> dune-pdelab at dune-project.org
> http://lists.dune-project.org/mailman/listinfo/dune-pdelab

Steffen Müthing
Universität Stuttgart
Institut für Parallele und Verteilte Systeme
Universitätsstr. 38
70569 Stuttgart
Tel: +49 711 685 88429
Fax: +49 711 685 88340
Email: steffen.muething at ipvs.uni-stuttgart.de