OSCS-SupCon: Orthogonal Sigmoid-based Common and Style Supervised Contrastive Learning for Robust Feature Disentanglement
Quick Answer
OSCS-SupCon introduces a novel supervised contrastive learning framework that enhances feature disentanglement by employing orthogonal constraints and a sigmoid-based loss.
Quick Take
OSCS-SupCon introduces a novel supervised contrastive learning framework that enhances feature disentanglement by employing orthogonal constraints and a sigmoid-based loss. It achieves a 3.4% accuracy improvement on the CUB200-2011 dataset with ResNet-18 compared to CS-SupCon, showcasing superior robustness and generalization across six benchmark datasets.
Key Points
- OSCS-SupCon addresses negative-sample dilution and feature entanglement in existing methods.
- Introduces a sigmoid-based contrastive loss with learnable temperature and bias parameters.
- Enforces orthogonality between common and style feature subspaces to improve disentanglement.
- Outperforms state-of-the-art methods across multiple backbone architectures in extensive experiments.
- Ablation studies validate the effectiveness of each component in the proposed framework.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 11233v1 Announce Type: new Abstract: Supervised Contrastive Learning (SupCon) has achieved strong performance by explicitly modeling pairwise relationships among samples. However, existing SupCon-based methods suffer from two key limitations: negative-sample dilution induced by the standard InfoNCE loss, and feature-space entanglement caused by the lack of explicit constraints separating category-relevant (common) and category-irrelevant (style) features.
These limitations reduce feature discriminability and generalization ability. To address these issues, we propose OSCS-SupCon (Orthogonal Sigmoid-based Common and Style Supervised Contrastive Learning), a unified framework that combines a sigmoid-based pairwise contrastive objective with explicit orthogonality constraints.
Specifically, we introduce a sigmoid-based contrastive loss with two learnable parameters, temperature and bias, which adaptively modulate pairwise decision boundaries and alleviate negative-sample dilution. Furthermore, we enforce orthogonality between common and style feature subspaces via a linear projection with ReLU nonlinearity, thereby reducing feature overlap and improving disentanglement of style-irrelevant representations.
Extensive experiments on six benchmark datasets demonstrate that OSCS-SupCon consistently outperforms state-of-the-art supervised contrastive learning methods across multiple backbone architectures. In particular, on the fine-grained CUB200-2011 dataset with a ResNet-18 backbone, the proposed method achieves a 3. 4% improvement in classification accuracy over CS-SupCon, highlighting its robustness and generalization capability. Ablation studies further confirm the effectiveness of each component.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.