SENTRY: Statistical Reliability Analysis of Vision Transformers Under Soft Errors
Quick Answer
The SENTRY framework offers a statistical approach to analyze the reliability of Vision Transformers (ViTs) like ViT-Tiny and ViT-Small against soft errors, achieving a 10,700 times reduction in experimental costs while ensuring failure rates are within 1% at 99% confidence.
Quick Take
The SENTRY framework offers a statistical approach to analyze the reliability of Vision Transformers (ViTs) like ViT-Tiny and ViT-Small against soft errors, achieving a 10,700 times reduction in experimental costs while ensuring failure rates are within 1% at 99% confidence. It identifies critical vulnerabilities in normalization layers and IEEE-754 exponent bits, crucial for designing robust ViT architectures in safety-critical applications.
Key Points
- Statistical fault injection ensures reliability of ViTs with only a few thousand samples.
- Failure rates are bounded within a 1% margin at 99% confidence across various models.
- Identified vulnerabilities are localized to normalization layers and critical exponent bits.
- Cost reduction of up to 10,700 times compared to exhaustive fault injection methods.
- Only 3% of FP32 bit-flips lead to failure, with most causing significant accuracy loss.
Article Content
From source RSS / original summaryarXiv:2606. 07620v1 Announce Type: new Abstract: With the growth of Vision Transformers in safety-critical domains like autonomous systems and medical imaging, ensuring their reliability against soft errors is paramount. While ViTs offer state-of-the-art accuracy, their massive parameter counts render exhaustive fault injection campaigns infeasible. To bridge this gap, a statistical fault injection framework is presented, leveraging finite-population sampling theory to provide formal reliability guarantees.
It is demonstrated that failure rates are bounded within a 1% margin at 99\% confidence using only a few thousand samples, regardless of model scale. This methodology achieves up to a 10,700 times reduction in experimental cost compared to exhaustive approaches, while preserving the ability to localize vulnerabilities across architectural components. Through extensive evaluation of different architectures like ViT-Tiny and ViT-Small, a highly non-uniform reliability landscape is uncovered.
It is shown that while only 3% of FP32 bit-flips result in failure, the vast majority of these events lead to catastrophic accuracy collapse. Specific vulnerabilities are localized to normalization layers and critical exponent bits within the IEEE-754 format, providing a mathematical foundation and actionable insights for the design of hardened, edge-deployed ViT architectures.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.
