Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits
Quick Answer
This paper introduces a loss-aware feature-map pruning framework for convolutional neural networks using multi-armed bandits, specifically UCB1 and Thompson Sampling.
Quick Take
The approach effectively reduces computation while maintaining accuracy across various datasets, including MNIST and CIFAR-10, outperforming traditional pruning methods. Results indicate that UCB1 achieves the highest mean rank and significantly enhances model efficiency without sacrificing performance.
Key Points
- Utilizes multi-armed bandits for structured feature-map pruning in CNNs.
- Evaluates UCB1 and Thompson Sampling against various datasets like MNIST and CIFAR-10.
- UCB1 and Thompson Sampling maintain accuracy while reducing convolutional computation.
- Both methods significantly outperform greedy and magnitude-based pruning techniques.
- Friedman and Nemenyi tests confirm UCB1's superior performance ranking.
DeepSignal Analysis
What happened
The paper presents a framework for pruning feature maps in convolutional neural networks using multi-armed bandits, specifically UCB1 and Thompson Sampling. This method aims to reduce computational costs while maintaining accuracy across various datasets, including MNIST and CIFAR-10. UCB1 reportedly achieves the highest mean rank in terms of efficiency.
Key evidence
- The study evaluates the performance of UCB1 and Thompson Sampling against direct evaluations on LeNet/MNIST, demonstrating their effectiveness in preserving accuracy while reducing computation.
- Results indicate that UCB1 and Thompson Sampling significantly outperform traditional greedy and magnitude-based pruning methods, while remaining statistically comparable to unpruned models.
- The framework ranks candidate feature maps based on learned scores after a fixed budget of evaluations, allowing for the permanent removal of the least effective maps.
Why it matters
This research addresses the challenge of redundancy in convolutional neural networks, which can lead to increased storage and inference costs. By employing a loss-aware approach to feature-map pruning, the framework not only enhances computational efficiency but also maintains model performance. This could have implications for deploying neural networks in resource-constrained environments, where efficiency is critical.
Paper Resources
Source Excerpt
Convolutional neural networks often contain redundant feature maps that increase storage and inference cost. This paper presents a loss-aware feature-map pruning framework using multi-armed bandits. Feature-map pruning is structured because it removes complete convolutional output channels and their producing filters rather than isolated scalar weights. Each candidate feature map is treated as an arm. At each play time, one map is temporarily masked and evaluated on a sampled mini-batch; the map
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →AINTMA: Agentic AI Architecture for Autonomous Test Management with Generative Intelligence, Secure Cloud Communication and Adaptive Quality Analytics
AINTMA, an autonomous test management architecture utilizing six specialized AI agents, achieves 88.4% test prioritization accuracy and reduces defect escape rates from 8.3% to 2.1%. The system demonstrates a 340% ROI within nine months, showcasing the potential of agentic AI in enhancing software quality management in cloud environments.