Schema-Constrained Document-Level Event Argument Extraction with Lightweight LLM Fine-Tuning
Quick Answer
This study demonstrates that mid-sized open LLMs, particularly Phi-4 (14B), can effectively perform schema-constrained Event Argument Extraction (EAE) at the document level, achieving a 42.39% F1 score on the MAVEN-ARG benchmark.
Quick Take
The approach utilizes role-set injection, LoRA fine-tuning, and deterministic decoding to enhance performance and reliability in extracting structured event records.
Key Points
- Phi-4 (14B) outperforms previous GPT models in event-coreference evaluations.
- The method incorporates role-set injection for schema compliance in prompts.
- LoRA fine-tuning is employed for efficient parameter optimization.
- Deterministic decoding includes post-processing to validate JSON outputs.
- Code for reproducing experiments is publicly available.
DeepSignal Analysis
What happened
The study investigates the capabilities of mid-sized open LLMs, particularly Phi-4 (14B), in performing schema-constrained Event Argument Extraction (EAE) at the document level. The model achieved a 42.39% F1 score on the MAVEN-ARG benchmark, indicating its effectiveness in this task. The methodology includes role-set injection, LoRA fine-tuning, and deterministic decoding to improve extraction performance.
Key evidence
- The research focuses on schema-constrained Event Argument Extraction (EAE) using mid-sized open LLMs, specifically Phi-4 (14B).
- The model achieved a 42.39% F1 score on the MAVEN-ARG benchmark, outperforming previous GPT baselines.
- The approach employs role-set injection, LoRA fine-tuning, and deterministic decoding to enhance the reliability of event record extraction.
Why it matters
This study highlights the potential of mid-sized open LLMs in handling complex document-level tasks like EAE, which is crucial for applications in natural language processing. The ability to achieve competitive performance with a 14B parameter model suggests that smaller models can be viable alternatives to larger counterparts, potentially reducing resource requirements for similar tasks. This could democratize access to advanced NLP capabilities.
Paper Resources
Source Excerpt
Event Argument Extraction (EAE) converts documents into structured event records by identifying argument spans and assigning them schema-defined roles. Document-level EAE is challenging due to long-range dependencies between triggers and arguments, cross-sentence context, and strict role constraints, which often lead to boundary errors, uncertainty in roles, and inconsistencies with restricted schemas. In this paper, we study whether mid-sized open can perform schema-constrained EAE relia
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.