[Dune-devel] SOCIS 2014: overlapping shwartz operator

[Agnese, Marco]

Hi DUNEers,
I have some trouble implementing what Markus asked me. What I have done so far can be found at the following link

https://github.com/magnese/dune-istl/blob/threads/dune/istl/test/threadsmpitest.cc

This example uses both threads and mpi to assemble the stiffness matrix and the rhs term.
Markus asked me to:
>1. modify your assembly routine such that each thread sets up local
>  matrix and a local vector with some overlap.
>
>   What I mean by this is the following. Partition the unknows between
>   the threads (the owner region). Then augment the partition by
>   adding all unknowns to the set that are connected by a non-zero
>  entry of the global matrix to an unknown already in the initial
>   partition. The added unknowns form the overlap/copy part of the thread.
> 2. Create the corresponding parallel index sets.
> 3. Perform a parallel matrix vector product using
>  OverlappingSchwarzOperator.

First of all I would like to know if I understood correctly the overlaps. For example for the RHS, each process has exactly one element overlapped which is the node shared by the two processes which shares the node (analogously for the stiffness matrix there is also one overlapped element). Is it correct?

Secondly, I have created the the ParallelIndexSet imposing the overlapped element as ghosts in both processes. The communication is performed coupling a forward with an add policy and a backward with a copy policy. The values on the b vector are set correctly after this but I don't know if should be done in another way which avoids a double communication. 

Moreover, I haven't created the IndexSet for the stiffness matrix yet since I am not sure how to do it. I suppose that I have to map the couple of index i,j to an unique index k (something like this k=i*size_matrix+j) and use this to perform an update of the entries as I have done for b. Is it correct or there is a better way?

Finally for what concern the OverlappingSchwartzoperator I am not sure how to use it. I suppose that in Dune::OverlappingSchwarzOperator< M, X, Y, C > the matrix type M,X,Y are already a parallel matrix and parallel vectors ( not the sequential structures which I have used). Am I right?Can I just create a parallel matrix/vector which is composed by the local matrices/vectors which I have assembled using the ParallelIndexSet created?

Thank you very much,
regards,
Marco.