SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents
Quick Answer
This paper shows that SkillCorpus consolidates over 821,000 skills into a curated corpus of 96,401 skills, improving LLM agent performance by up to 7.5 percentage points on SkillsBench.
Quick Take
The framework evaluates skills based on utility, robustness, and safety, providing a comprehensive resource for real-world agent tasks.
Key Points
- Aggregates 821,000 skills into a curated corpus of 96,401 skills.
- Improvements observed across three benchmarks, notably +7.5 pp on SkillsBench.
- Skills categorized by a 16-class taxonomy and three quality facets.
- First end-to-end evaluation of curated community corpus for agent tasks.
- Dataset, models, and code to be released upon acceptance.
Paper Resources
Source Excerpt
Agent skills, SKILL. md files that package reusable procedural knowledge for an agent, are a popular mechanism for extending agent capabilities. Public repositories now host them in large and growing numbers, yet these artifacts are fragmented, redundant, and uneven in quality, and their value in practice is unclear. A core question remains open, namely how to consolidate this open-source SKILL. md ecosystem into a single usable corpus, and what bounds its benefit on real-world agent tasks. We
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.