Residual Modeling for High-Fidelity Learned Compression of Scientific Data
Quick Answer
The proposed residual-centric approach introduces two coders, LBRC and NGLR, enhancing compression ratios by 30-60% over GAE and outperforming SZ in high-fidelity regimes.
Quick Take
LBRC adapts to target NRMSE using deterministic methods, while NGLR incorporates a neural predictor to further reduce residual code entropy. These advancements are crucial for efficient lossy compression of scientific data.
Key Points
- LBRC improves compression ratios by 30-60% over Guaranteed Autoencoder (GAE).
- NGLR adds an additional 10-40% compression improvement over LBRC.
- Both methods target block-level NRMSE from 10^-6 to 10^-4.
- Residual representations tailored to learned-compressor residuals enhance performance.
- Results are validated across datasets like E3SM, JHTDB, and ERA5.
Paper Resources
Source Excerpt
arXiv:2606. 05389v1 Announce Type: new Abstract: Lossy compression is essential for massive spatiotemporal data from scientific simulations. Learned compressors can achieve high compression ratios at moderate accuracy targets, but their aggregate reconstruction losses do not guarantee accuracy for each block. Existing Guaranteed Autoencoder (GAE) methods add a per-block residual correction by retaining SVD/PCA-style coefficients until the target is met.
This works at moderate tolerances, but in the high-fidelity regime with block-level NRMSE from 10^-6 to 10^-4, the number of retained coefficients grows quickly and the correction stream dominates the total rate. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →RAIL Guard: Closing the Evaluation-to-Remediation Gap in Responsible AI for Agents
RAIL Guard introduces a closed-loop AI pipeline for large language models (LLMs) that evaluates outputs across eight dimensions and iteratively remediates failures, achieving 96.9% convergence compared to 49.1% for traditional block-and-retry methods. The system reduces unsafe agent executions by 33% without impacting task completion and is available as open-source SDKs.