DWT-Fusion: A Signal-Based Framework for Training-Free LLM-Generated Text Detection
Quick Answer
DWT-Fusion introduces a training-free framework for detecting LLM-generated text using discrete wavelet analysis of token-level log-probabilities.
Quick Take
Evaluated on HC3, M4, and MAGE datasets with models like GPT-Neo-2.7B and LLaMA-3-8B, it achieved AUROC scores up to 0.9919 through calibration-weighted voting, demonstrating effective signal extraction without supervised training.
Key Points
- DWT-Fusion utilizes discrete wavelet analysis for detecting -generated text.
- Achieved AUROC scores of 0.9919 on HC3 with calibration-weighted voting.
- Evaluated on datasets HC3, M4, and MAGE using various proxy models.
- Framework combines multiple wavelet configurations without a supervised meta-classifier.
- Local and multiscale variations in token predictability are effectively utilized.
DeepSignal Analysis
What happened
DWT-Fusion presents a framework for detecting text generated by large language models (LLMs) without requiring training. It utilizes discrete wavelet analysis to assess token-level log-probabilities from various proxy models, achieving high AUROC scores in evaluations across multiple datasets.
Key evidence
- DWT-Fusion employs discrete wavelet analysis of token-level log-probabilities to detect LLM-generated text, addressing challenges in zero-shot and training-free conditions.
- The framework was evaluated on HC3, M4, and MAGE datasets using models like GPT-Neo-2.7B and LLaMA-3-8B, achieving AUROC scores up to 0.9919 with calibration-weighted voting.
- The best single wavelet configurations achieved AUROC values of 0.9872, 0.8185, and 0.7138 on HC3, M4, and MAGE, respectively.
Why it matters
The ability to detect LLM-generated text without training is significant, especially as these models become more prevalent. DWT-Fusion's approach leverages local variations in token predictability, which may enhance detection accuracy compared to traditional methods that rely on global measures. This framework could be crucial for applications requiring reliable content verification, such as academic integrity and misinformation detection.
Paper Resources
Source Excerpt
Detecting -generated text remains challenging under zero-shot and training-free conditions, especially when detectors must generalize across datasets, domains, and unseen generators. While existing training-free approaches exploit language-model statistics as detection signals, they typically characterize a text through global measures that summarize overall model behavior. Consequently, potentially informative local and multiscale variations in token-level predictability may remain underutil
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.