TriAgent: Divergence-Aware Multi-Agent Committees for Cost-Efficient Financial Sentiment Analysis
Quick Answer
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5.
Quick Take
It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.
Key Points
- TriAgent uses a three-tiered approach: word-level, sentence-level, and cross-sentence reasoning.
- Achieves F1 score of ~0.87, outperforming a 3-persona voting system at F1=0.66.
- Shared Consensus Dictionary answers 95% of Chinese queries with F1=0.99 from English cache.
- SDI serves as a hallucination detector with AUC=0.90.
- Saves $9.3M/year at a 10M-user scale compared to GPT-4o-mini.
DeepSignal Analysis
What happened
TriAgent is a multi-agent system designed for financial sentiment analysis, integrating VADER, FinBERT, and Qwen2.5. It reportedly achieves an F1 score of approximately 0.87 while saving $9.3 million annually at a scale of 10 million users compared to GPT-4o-mini. Additionally, it detects LLM hallucinations with an AUC of 0.90.
Key evidence
- TriAgent combines three models: VADER for word-level analysis, FinBERT for sentence-level, and Qwen2.5 for cross-sentence reasoning.
- The system achieves an F1 score of ~0.87 and saves $9.3 million per year at a 10 million user scale compared to GPT-4o-mini.
- TriAgent's Semantic Divergence Index (SDI) serves as a post-hoc LLM hallucination detector, achieving an AUC of 0.90.
Why it matters
The introduction of TriAgent addresses the cost inefficiencies in processing financial sentiment analysis queries, where many are easily classifiable. By utilizing a multi-agent approach, it optimizes resource allocation and reduces operational costs significantly. The ability to detect hallucinations in LLM outputs is also crucial for maintaining the reliability of financial analyses, which can have substantial implications for decision-making in finance.
Paper Resources
Source Excerpt
Production -based financial sentiment analysis faces a structural cost trap: most queries are trivially classifiable, yet expensive cloud reasoners process them all, and the bill scales linearly with user count. We present TriAgent, a committee stratified by contextual granularity -- a word-level lexicon (VADER), a sentence-level domain transformer (FinBERT), and a cross-sentence reasoner (Qwen2. 5, 0. 5B-14B-4bit, with Mistral-7B and Phi-3. 5-mini cross-family checks). A three-way S
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →RF-Agent: A Practical Framework for Building Language Agents for RFIC Design
RF-Agent introduces a novel framework for RF circuit design using , creating a unique RF-domain reasoning dataset with over 11,000 samples. The study reveals that domain-specific supervised fine-tuning and semantic retrieval strategies significantly enhance RF reasoning performance, particularly for smaller models.