[visit-users] FLASH hdf5 reader

Alexander Wagner alexander.y.wagner at googlemail.com
Wed Mar 10 07:57:09 EST 2010


Hi Hank,

Thanks for the quick reply. That's good news. I'll give 2.0 a go. I hope 
someday I can become more familiar with the plugin contribute to 
improvements too.

Best wishes,
Alex


Hank Childs wrote:
> Hi Alex,
>
> Yes, 2.0 will have much better performance.  There was a problem in
> the data structure for managing subset relations.  For AMR data sets
> that had lots of subsets, performance was tanking.  When I did tuning
> a few months ago, FLASH data was the driver.  (Note: the FLASH reader
> in particular still has a problem with 1D curves, which I intend to
> address for the release.)
>
> If you want to try out 2.0 early, you can access the source code at:
> http://visitusers.org/index.php?title=Anonymous_svn
>
> BTW, SetTimeSliderState still has a slowness that I'd like to tune
> additional, but I believe it isn't nearly as bad as it was before.
>
> -h
>
> On Wed, Mar 10, 2010 at 4:21 AM, Alexander Wagner
> <alexander.y.wagner at googlemail.com> wrote:
>   
>> Hi Hank & developers,
>>
>> I'm picking up an issue I left dangling a while ago and couldn't pursue due
>> to other commitments.
>>
>> I'm now using 1.12.2 but I'm still struggling with FLASH hdf5 data. I am
>> working with 512^3 cell AMR datasets.  I feel that
>> the i/o is rather sluggish. I'm not exactly sure what is going on, but,
>> for example, opening a large database (say of more than 100 files, each
>> ~1-5GB) can take a long time ( ~tens of minutes) before I can add a
>> plot. Also, SetTimeSliderState(<>) takes a long time to execute.
>> Looking at the progress bars on the compute engine, all parallel
>> compute processes, I find, go really fast. But there are long pauses in
>> between.
>>
>> Another thing i still keep encountering is that the compute engine dies
>> after changing the timeslider state n times, where n is smaller the
>> more complex the plot. I monitored the memory use of the compute engine
>> nodes and they kept creeping up, reaching 100%, whereupon the compute
>> engine dies. This seems to me like there are memory leaks. Not all data
>> of the previous time step seem to be cleared.
>>
>> The questions i have are: - Might I expect better performance in VisIt 2?
>> - Would the performance improve if I exported my data to a different format
>> e.g. vtk - perhaps after re-sampling to a uniform 512^3 data cube?
>> - Should I try and run visit under valgrind to find the memory leak if there
>> is one? I've never used it before, and i'm not familiar with the reader
>> either.
>> - Any other options?
>>
>> I have made some FLASH hdf5 available, if that helps
>> http://rsaa.anu.edu.au/~ayw/ftp/FLASH_hdf5_files.tar.gz (contains 7 files
>> 13G total)
>> http://rsaa.anu.edu.au/~ayw/ftp/jet3d_hdf5_plt_cnt_0204 (1 file 1.7G)
>>
>> Many thanks in advance,
>> Alex
>>
>>
>>     
>>> Hi Alex,
>>>
>>>       
>>>> / Thanks a lot for your detailed responses! They help me a lot.
>>>>         
>>> /
>>> Great.
>>>
>>>       
>>>> / I will definitely be using the software for a long time to come (others
>>>> here
>>>>         
>>> />/ will too). I have been doing simple visualizations of FLASH hdf5 data.
>>> You
>>> />/ mentioned that you had improved the reader.
>>> />/
>>> />/ I have, so far, only applied this patch:
>>> />/
>>> />/
>>> https://email.ornl.gov/mailman/htdig/visit-users/2009-November/005621.html
>>> />/
>>> />/ If you have any more patches, could you let me know where I can get
>>> them
>>> />/ from? Or is 1.12.2 around the corner?
>>> /
>>> It is new code.  I just checked in the changes yesterday.  VisIt was
>>> struggling mightily on AMR data sets with large numbers of patches.
>>> The changes are on the trunk.  (I still have a few more FLASH changes
>>> to make, btw.)  There are noticeable delays with tens of thousands of
>>> patches.  With hundreds of thousands, it became unusable.  So if you
>>> are at that scale, then we should discuss options.
>>>
>>> Best regards,
>>> Hank
>>>       
>> --
>> List subscription information:
>> https://email.ornl.gov/mailman/listinfo/visit-users
>> Searchable list archives: https://email.ornl.gov/pipermail/visit-users
>> VisIt Users Wiki: http://visitusers.org/
>> Frequently Asked Questions for VisIt: http://visit.llnl.gov/FAQ.html
>>
>>
>>     
> --
> List subscription information: https://email.ornl.gov/mailman/listinfo/visit-users
> Searchable list archives: https://email.ornl.gov/pipermail/visit-users
> VisIt Users Wiki: http://visitusers.org/
> Frequently Asked Questions for VisIt: http://visit.llnl.gov/FAQ.html
>
>   


More information about the visit-users mailing list