Is Domain Adaptation Always Helpful? A Frozen-Backbone Study of Cross-Domain Sentiment Transfer
Quick Answer
This study evaluates the effectiveness of domain adaptation in sentiment analysis using frozen pre-trained language models like Qwen3 and FinBERT.
Quick Take
Results show negligible gains on SST-2 movie reviews but significant performance recovery on financial news with small backbones, highlighting that adaptation efficacy depends on existing target-domain knowledge in the backbone.
Key Points
- Evaluated frozen backbones: Qwen3-Embedding (0.6B, 4B, 8B), RoBERTa-base, and FinBERT.
- Domain adaptation yielded negligible gains on SST-2, regardless of model scale.
- Significant performance recovery observed in financial news with small general-purpose models.
- Adversarial alignment (DANN) degraded performance for domain-specialized models like FinBERT.
- Supervised contrastive loss preserved domain-specific structures better than DANN.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Sentiment analysis with frozen pre-trained language model (PLM) backbones has become a common paradigm, yet the practical benefit of explicit domain adaptation remains unclear, particularly when backbones encode varying degrees of target-domain knowledge. We present a preliminary case study evaluating a controlled family of frozen embedding backbones (Qwen3-Embedding 0.6B, 4B, 8B), alongside RoBERTa-base and FinBERT. We train a lightweight MLP adapter on consumer reviews using Domain-Adversarial Neural Networks (DANN), Maximum Mean Discrepancy (MMD), and Supervised Contrastive Learning (SCL), and evaluate transfer to movie reviews (SST-2) and a heavily restricted subset of financial news (Financial PhraseBank). Within this constrained sample, we observe two distinct transfer patterns. On SST-2, domain adaptation provides negligible gain regardless of scale. On the financial subset, explicit domain adaptation appears to recover substantial performance for small general-purpose backbones. Notably, we find that adversarial alignment (DANN) is associated with degraded performance for domain-specialized backbones like FinBERT, consistent with erosion of pre-existing domain-specific structure, whereas supervised contrastive loss appears to preserve it. These preliminary findings suggest that the efficacy of explicit domain adaptation is highly contingent on whether the frozen backbone already possesses target-domain coverage.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.05937 [cs.CL] |
| (or arXiv:2607.05937v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05937 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Phat Tran [view email]
[v1]
Tue, 7 Jul 2026 07:40:00 UTC (69 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.