Building a Multimodal Dataset of Academic Paper for Keyword Extraction
Quick Answer
This study presents a multimodal dataset of 1000 academic papers for keyword extraction, incorporating text, images, and audio.
Quick Take
This study presents a multimodal dataset of 1000 academic papers for keyword extraction, incorporating text, images, and audio. Experiments reveal that combining these modalities significantly enhances keyword extraction performance, highlighting the importance of diverse data sources in model training.
Key Points
- Dataset includes 1000 samples with text, images, audio, and keywords.
- Experiments compare keyword extraction using various modalities.
- Distinct characteristics of text from different modalities were observed.
- Combining paper text, image text, and audio text improves performance.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 31069v1 Announce Type: new Abstract: Up to this point, keyword extraction task typically relies solely on textual data. Neglecting visual details and audio features from image and audio modalities leads to deficiencies in information richness and overlooks potential correlations, thereby constraining the model's ability to learn representations of the data and the accuracy of model predictions.
Furthermore, the currently available multimodal datasets for keyword extraction task are particularly scarce, further hindering the progress of research on multimodal keyword extraction task. Therefore, this study constructs a multimodal dataset of academic paper consisting of 1000 samples, with each sample containing paper text, images, audios and keywords.
Based on unsupervised and supervised methods of keyword extraction, experiments are conducted using textual data from papers, as well as text extracted from images and audio. The aim is to investigate the differences in performance in keyword extraction task with respect to different modal information and the fusion of multimodal information. The experimental results indicate that text from different modalities exhibits distinct characteristics in the model.
The concatenation of paper text, image text and audio text can effectively enhance the keyword extraction performance of academic papers.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.