[Dune] FW: H5Fed into Dune as a grid manager, comments, ideas ???
Andreas Dedner
dedner at mathematik.uni-freiburg.de
Sun Nov 15 20:47:30 CET 2009
Hi,
perhaps a short remark on DGF with ALU/Alberta:
DGF was never designed for large grids and ALU/Alberta have an internal
format
which can be used for reading/writing grids.
DGF provides a Gridparameter block where one can add a parameter
dumpfile name
which tells ALU/Alberta to dump the grid before creation in their
internal format.
This file can then be directly passed to the GridPtr instead of a DGF file
(if the keyword DGF in the file is missing, the DGFParser just passes
the file on the
grid manager - and hopes that it can handle it...
This should solve some of the time problems, just read the dgf file
once, dump the native
format file and then use that for all follwoing runs.
The problem with reading in seriell is not solved in this way. Here we
can suggest th following
procedure in ALU:
read a (smaller) grid using dgf. Refine and load balance the grid using
the dune bindings,
write the (parallel) grid using the writeGrid(name,time) method on
ALUGrid (see
grid/alugrid/3d/grid.hh for some details).
This of course only works, if the grid you wan to use for a presentation
is a refined
grid from a coarser macro grid.
Alternativly go on a computer with a large amount of memory, read the
dgf file, call
loadbalance and use the writeGrid method to write the grid in parallel.
I hope that this can help - and I think the idea of having a further
parallel, locally adapted
grid in DUNE would be a great idea, so view my comments as a work around
until your
project is finished.
Andreas
Oswald Benedikt wrote:
> Hi Oliver, indeed, you're right. H5Fed initially was designed as a standard for storing finite element meshes
> and associated data only, including hierarchically refined grids with hanging nodes.
>
> When we then proceeded with its development we realized that optimum use of H5Fed in our
> codes requires tight integration into Dune; for example we need checkpoint-restart capability, especially for
> larger machines such as the Cray where run time is often limited to 48 hours, larger jobs then must be
> resubmitted to go through.
>
> Also, we do need the capability of reading large meshes and associated dats (volume id, boundary id)
> NOW, especially for larger macro grids (large meaning n * 10^5 tets). While we intensely use
> the DGF format, this does not scale as other users have come to realize. Eventually, this prevents
> us from running the problems that we must run: waiting for hours sometimes until the mesh
> and its parameters are read, is just not an option.
>
> Further, we think that reading large meshes should really be done in parallel, i.e. not reading
> the whole mesh on the master node (memory bottleneck, especially on future supercomputers
> which may have hundreds of thousands of cores but quite little memory per core; e.g. our
> Cray XT5 in Manno has 22 kCores but only 1.3 GByte per core...)
>
> Eventually, H5Fed will not stop short of only having tets as elements. Whereas we now focus
> on tetrahedra, we foresee its extension to hexahedra and maybe also hybrid meshes,
> combining tets and hexahedra.
>
> To make a long story short, we need a grid manager and a data storage format that can handle
> practical meshes (while the macro grid may have some 10^5 tets, the leaf grid may have hundreds
> of millions of tets eventually, this number may sound huge but it is foreseeable).
>
> Let me make one point very clear: we would like to contribute the Dune community
> to this actively in order to make it a success. In no way this is designed to introduce
> competition, but really it is intended to solve our comp. needs, together with Dune
> since we think Dune to be one of the greatest project, if not the greatest , in this field
>
>
> Have a nice weekend! Benedikt
>
>
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------------------------------------------
> Benedikt Oswald, Dr. sc. techn., dipl. El. Ing. ETH, www.psi.ch, Computational Accelerator Scientist
> Paul Scherrer Institute (PSI), CH-5232 Villigen, Suisse, benedikt.oswald at psi.ch, +41(0)56 310 32 12
> "Passion is required for any great work, and for the Revolution passion and audacity are required in big doses.",
> Ernesto 'Che' Guevara, Letter to his parents.
> http://maxwell.psi.ch/amaswiki/index.php/User:BenediktOswald
> ------------------------------------------------------------------------------------------------------------------
>
>
>
>
> -----Original Message-----
> From: Oliver Sander [mailto:sander at mi.fu-berlin.de]
> Sent: Fri 13.11.2009 16:05
> To: Oswald Benedikt
> Cc: dune at dune-project.org
> Subject: Re: [Dune] FW: H5Fed into Dune as a grid manager, comments, ideas ???
>
> Hi Benedikt!
> So far I thought that what you guys did was developing a general
> and powerful file format for fe grids and data. Now all of a sudden
> it is a grid. My impression reading your email below is that you are
> reinventing the wheel. What can your grid do that ALUGrid or UGGrid
> cannot do already? Why spend the time to write yet another grid
> manager (yagm)? If the problem is that there is currently no way to
> get a hierarchical grid from your file into a ALU/Alberta/UGGrid then
> we should discuss the possibility of extending the GridFactory interface.
>
> Yours,
> Oliver
>
> Oswald Benedikt schrieb:
>
>>
>> Hi Christian, yes, local refinement with hanging nodes is is already
>> available in the current H5Fed (which has no connection to Dune yet)
>> including tags and adjacencies; i.e. we can attach and read / write
>> data to any top. entity on any level; data may be scalar, vector and
>> matrices of type double, complex, integer.
>>
>> Maybe we should use derive from the grid factory class
>> and add additional member functions ?
>>
>> Benedikt
>>
>>
>>
>>
>> -----Original Message-----
>> From: Christian Engwer [mailto:christi at uni-hd.de]
>> Sent: Fri 13.11.2009 14:03
>> To: Oswald Benedikt
>> Cc: Markus Blatt; dune at dune-project.org
>> Subject: Re: [Dune] H5Fed into Dune as a grid manager, comments, ideas ???
>>
>> On Fri, Nov 13, 2009 at 01:56:26PM +0100, Oswald Benedikt wrote:
>>
>>
>>> indeed yes, the idea is to fully connect/integrate H5Fed into Dune so that we can also use
>>> H5Fed as a grid manager. So far H5Fed works in serial mode, without connection to Dune,
>>> and has been tested up to about 2 million tets, but tests are ongoing and of course this
>>> number of tets makes more sense in parallel aynway.
>>>
>>> One of the questions is then if we should adhere to the grid factory approach as do the
>>> other grid managers ?
>>>
>>>
>> Well, the grid factory is only for coarse grid creation. I suggest to
>> support this interface, but you will need more. Do you intend to
>> implement a full grid manager with local refinement etc? In this case
>> you will also want to add read/write methods for your "internal" data
>> format.
>>
>> Christian
>>
>>
>>
>>> Greetings, Benedikt
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: dune-bounces+benedikt.oswald=psi.ch at dune-project.org on behalf of Markus Blatt
>>> Sent: Fri 13.11.2009 13:51
>>> To: dune at dune-project.org
>>> Subject: Re: [Dune] H5Fed into Dune as a grid manager, comments, ideas ???
>>>
>>> Hi,
>>>
>>> On Thu, Nov 12, 2009 at 05:17:06PM +0100, Oswald Benedikt wrote:
>>>
>>>
>>>> NOW, we want to integrate H5Fed into Dune as a (parallel) grid manager,
>>>> in a way like other grid managers are connected to Dune:
>>>>
>>>>
>>>>
>>> Up to now I thought that you are just concerned about reading and
>>> writing data. Do I interpret your comment correctly in sense that you
>>> will provide a full Dune grid based on H5fed? Or am I just not
>>> comprehending your sentence?
>>>
>>> Markus
>>>
>>> _______________________________________________
>>> Dune mailing list
>>> Dune at dune-project.org
>>> http://lists.dune-project.org/mailman/listinfo/dune
>>>
>>>
>>> _______________________________________________
>>> Dune mailing list
>>> Dune at dune-project.org
>>> http://lists.dune-project.org/mailman/listinfo/dune
>>>
>>>
>>>
>>
>> _______________________________________________
>> Dune mailing list
>> Dune at dune-project.org
>> http://lists.dune-project.org/mailman/listinfo/dune
>>
>>
>
>
>
More information about the Dune
mailing list