[Dune] A question on CollectiveCommunication

[Markus Blatt]

Hi,

On Mon, Jan 19, 2015 at 12:36:23PM +0100, Steffen Müthing wrote:
> 
> Am 19.01.2015 um 11:51 schrieb Aleksejs Fomins <aleksejs.fomins at lspr.ch>:
> 
> > Signierter PGP Teil
> > Dear Dune,
> > 
> > I am trying to understand the paradigm of the communication used in Dune
> > 
> > The question concerns the following
> > Dune::CommDataHandleIF< DataHandleImp, DataTypeImp >
> > 
> > and
> > 
> > int Dune::CollectiveCommunication< C >::gather( T * in, T * out, int
> > len, int root) const
> > 
> > In both cases we define a template parameter which defines the data
> > type to be communicated. As I understood from discussions with Marcus
> > and Peter, internally the communication is performed using MPI_BYTE
> > communication, i.e. sending size_of(T) data from a given address to a
> > given address.
> > 
> > It looks to me that this should not work for T = some stl object, because:
> > 1) If T has variable size, and we are passing a pointer to the array
> > of T's in gather, then gather has no information on the length of each
> > individual T
> > 2) If I would like to communicate T = std::vector<int>, would not the
> > internal implementation of vector contain a pointer to an array for
> > the current processor?

Then please pass &(vec[0]) to the method.
> > 
> > That said, the question is as follows:
> > 
> > Is dune communication interface intended to handle standard stl types?
> > If no, which DataTypes are expected/allowed to be communicated?
> 
> as you have correctly figured out, all internal communication happens by means
> of a char buffer, so the only kind of data you can safely
> communicate are POD types.

At least for ColllectiveCommunication This is not entirely
correct. You can send any type that the size can be deduced at compile
time. This includes pods. The default is to query the size of the
type, convert it bytes, and send. One can specialize this behaviour
using templates. There are a number of specializations that use MPI to
store and retrieve the values. See
e.g. dune/common/parallel/mpitraits.hh. This mechanism is also used by
the VariableSizeCommunicator.

Unfortunately, the grid classes are much dumber and always use the
default above.

> YaspGrid was bitten by this some time ago because it tried to send objects of a
> class with virtual functions. MPI then happily copied the vtable pointer to the target
> machine. That machine then crashed as soon as it tried to dereference the pointer
> for a function call.
> 
> TL;DR: only communicate POD data.

Only compute data where you can query the size based only on the
information of the type.

Markus

-- 
Do you need more support with DUNE or HPC in general? 

Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de
Hans-Bunte-Str. 8-10, 69123 Heidelberg, Germany
Tel.: +49 (0) 160 97590858
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <https://lists.dune-project.org/pipermail/dune/attachments/20150119/32b6b8dc/attachment.sig>