Self-Supervised Skill Optimization
Quick Answer
This paper shows that Self-Supervised Skill Optimization (SSO) introduces a framework for optimizing skills in large language models (LLMs) without ground-truth feedback.
Quick Take
By leveraging unlabeled task instances, SSO outperforms existing prompt optimizers in both closed-ended and open-ended tasks, achieving results comparable to GT-based optimizers. This method enhances the reusability of procedural guidance for agents.
Key Points
- SSO learns reusable skills solely from unlabeled task instances.
- It generates skill probes using a subset of execution results.
- An LLM judge evaluates behaviors without ground-truth labels.
- SSO outperforms GT-free optimizers on closed and open-ended tasks.
- The method achieves results comparable to GT-based optimizers.
DeepSignal Analysis
What happened
Self-Supervised Skill Optimization (SSO) is a new framework designed to enhance skills in large language models (LLMs) without relying on ground-truth feedback. It utilizes unlabeled task instances to optimize skills, achieving performance levels comparable to ground-truth-based optimizers. SSO demonstrates effectiveness in both closed-ended and open-ended tasks.
Key evidence
- SSO operates by running current skills on unlabeled batches and generating skill probes from the results, which are then evaluated by an LLM judge.
- The framework aggregates evidence for and against observed behaviors across instances, ranking them to create a new skill that is accepted only if it outperforms the previous one on an unlabeled validation set.
- In tests, SSO outperformed existing ground-truth-free prompt optimizers and approached or exceeded the performance of the best ground-truth-based skill optimizer.
Why it matters
The introduction of SSO is significant as it addresses the challenge of optimizing LLM skills in scenarios where ground-truth feedback is unavailable. This advancement could enhance the utility of LLMs in various applications, making them more adaptable and efficient. By improving skill optimization without needing labeled data, SSO could lower the barrier to deploying LLMs in real-world tasks.
Paper Resources
Source Excerpt
Agent skills provide frozen (LLM) agents with reusable procedural guidance, and recent work shows that such skills can be optimized with ground-truth (GT) feedback. Many applications, however, lack GT labels, task scores, rewards, or reliable task-specific evaluators. We therefore introduce Self-Supervised Skill Optimization (SSO), a comparative framework that learns a reusable skill from unlabeled task instances alone. At each step, SSO runs the current skill on an unlabele
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.