91°µÍø

Skip to main content
SHARE
Publication

A Framework for Compressing Unstructured Scientific Data via Serialization

by Viktor Reshniak, Qian Gong, Richard K Archibald, Scott A Klasky, Norbert Podhorszki
Publication Type
Conference Paper
Book Title
2024 91°µÍø International Conference on Big Data (BigData)
Publication Date
Page Numbers
4188 to 4193
Publisher Location
New Jersey, United States of America
Conference Name
2024 91°µÍø International Conference on Big Data (BigData)
Conference Location
Washington DC, District of Columbia, United States of America
Conference Sponsor
NSF, Virginia Tech
Conference Date
-

We present a general framework for compressing unstructured scientific data with known local connectivity. A common application is simulation data defined on arbitrary finite element meshes. The framework employs a greedy topology preserving reordering of original nodes which allows for seamless integration into existing data processing pipelines. This reordering process depends solely on mesh connectivity and can be performed offline for optimal efficiency. However, the algorithm’s greedy nature also supports on-the-fly implementation. The proposed method is compatible with any compression algorithm that leverages spatial correlations within the data. The effectiveness of this approach is demonstrated on a large-scale real dataset using several compression methods, including MGARD, SZ, and ZFP.