Instruction Finetuning DeepSeek-R1-8B Model Using LoRA and NEFTune
Quick Answer
The DeepSeek-R1-8B model, enhanced with LoRA and NEFTune, achieves a micro-F1 score of 0.912 for financial NER, outperforming Llama3-8B and other benchmarks.
Quick Take
This approach effectively addresses misclassification of financial entities in unstructured reports, benefiting financial analysts and researchers.
Key Points
- DeepSeek-R1-8B model utilizes LoRA and NEFTune for financial NER.
- Achieved a micro-F1 score of 0.901 with LoRA, improved to 0.912 with NEFTune.
- Outperformed Llama3-8B, Qwen3-8B, Baichuan2-7B, T5, and BERT-Base.
- Study based on a corpus of 1693 annotated financial samples.
- Addresses misclassification of financial entities in unstructured texts.
Paper Resources
Source Excerpt
Financial named-entity recognition (NER) is essential for translating unstructured financial reports and news into structured knowledge graphs. However, general-purpose (LLMs) often misclassify financial entities or ignore domain-specific patterns. This paper investigates the use of DeepSeek-R1-8B, a recent open-source large language model, combined with Low-Rank Adaptation (LoRA) and Noisy Embedding Fine-Tuning (NEFTune) for financial NER. Each annotated sentence in our co
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →AINTMA: Agentic AI Architecture for Autonomous Test Management with Generative Intelligence, Secure Cloud Communication and Adaptive Quality Analytics
AINTMA, an autonomous test management architecture utilizing six specialized AI agents, achieves 88.4% test prioritization accuracy and reduces defect escape rates from 8.3% to 2.1%. The system demonstrates a 340% ROI within nine months, showcasing the potential of agentic AI in enhancing software quality management in cloud environments.