Anchor Divergence for Semantic Geometry in Contrastive Learning
Quick Answer
This paper introduces 'Anchor Divergences' to define context-specific semantic geometries in contrastive learning, enhancing the modeling of semantic similarity in vector representations.
Quick Take
By leveraging the interplay between contrastive learning and information geometry, the method effectively captures diverse geometries based on semantic context, improving retrieval tasks.
Key Points
- Anchor Divergences establish a link between probability distributions and Bregman geometries.
- The method allows for context-specific semantic similarity in fixed representations.
- Experiments show improved effectiveness and efficiency in retrieval tasks.
- Semantic similarity is shown to be context-dependent, challenging fixed cosine similarity metrics.
Paper Resources
📖 Reader Mode
~2 min readAbstract:This paper concerns how semantic context determines geometry in learned vector representations. Similarity is typically measured using cosine similarity, which provides a single fixed geometry. Semantic similarity, however, is inherently context dependent: two images may be similar because they depict the same object, share a visual style, or are relevant to the same clinical finding. We show that contrastive representations naturally encompass a family of geometries that can be specialized to particular semantic structure. The key idea is to use an interplay between contrastive learning, exponential families, and information geometry to establish a correspondence between probability distributions over "anchors" and Bregman geometries on the representation space. We use this correspondence to define "Anchor Divergences", a method for specifying context-specific semantic geometries on fixed representations. Under this correspondence, modeling the anchor distribution models the geometry itself. Experiments on retrieval show that anchor divergences provide an effective and efficient way to specify context-specific semantic similarity.
| Comments: | Code is available at this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.06919 [cs.AI] |
| (or arXiv:2610.06919v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.06919 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Akash Kannan [view email]
[v1]
Fri, 2 Oct 2026 21:23:21 UTC (7,928 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.