CaresAI at SMM4H-HeaRD 2026: Predicting TNM Staging
Quick Answer
This paper shows that CaresAI's study at SMM4H-HeaRD 2026 predicts TNM staging using various models, achieving AUROC scores up to 0.9368 with LightGBM and TF-IDF.
Quick Take
CaresAI's study at SMM4H-HeaRD 2026 predicts TNM staging using various models, achieving AUROC scores up to 0.9368 with LightGBM and TF-IDF. Despite strong training results, model generalizability remains a concern, highlighting the need for further optimization before clinical application.
Key Points
- LightGBM with TF-IDF achieved the highest AUROC scores: 0.9368 (T), 0.9524 (N), 0.8311 (M).
- Wide Residual Networks (WRN) showed AUROC scores of 0.839 (T), 0.8502 (N), and 0.803 (M).
- The study reported a Macro-F1 score of 0.978 for test set 1 across T, N, and M categories.
- Performance declined significantly from test set 1 to 2, indicating model generalizability issues.
- Further optimization and validation are necessary before real-world clinical use.
Paper Resources
📖 Reader Mode
~2 min readAbstract:This study aims to predict Tumor, Node, and Metastasis (TNM) stage labels independently, with the Cancer Genome Atlas (TCGA) pathology report as the sixth shared task of SMM4H-HeaRD 2026. The problem is framed as three multi-label classification tasks. We explore both classical and deep learning approaches using Term Frequency-Inverse Document Frequency (TF-IDF) features and embeddings from ClinicalBERT, BioBERT, and PubMedBERT. These representations are used with Logistic Regression (LR), Light Gradient Boosting Machine (LightGBM), Feed-Forward Neural Networks (FFNN), and Wide Residual Networks (WRN). Our results show that individual embeddings perform similarly to the TNM label classification, while their combination improves its predictive ability. WRN achieves AUROC scores of 0.839 (T), 0.8502 (N), and 0.803 (M) with F1-scores of 0.622, 0.702, and 0.9337, respectively, for the training phase. LightGBM with TF-IDF performs best with AUROC scores of 0.9368 (T), 0.9524 (N), and 0.8311 (M) and F1-scores of 0.7559 (T), 0.7384 (N), and 0.7017 (M) during the training phase. Furthermore, the result of the Codabench for the test sets indicates a Macro-F1 score of 0.978, 0.957, and 0.879 for the T, N, and M categories respectively for test set 1; while test set 2 records a Macro-F1 score for T, N, and M is 0.807, 0.767, 1.0 respectively. However, performance declined during the evaluation phase of the test sets, a drop from 0.938 to 0.858 of test set 1 to 2, for the Macro-F1 score across all stages; suggesting limitations in model generalizability, sensitivity to class imbalance, and challenges in processing lengthy clinical documents. Although this study provides an efficient baseline model and a reproducible pipeline, further optimization and validation are required before it can be considered suitable for use in a real-world clinical setting.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.03466 [cs.CL] |
| (or arXiv:2607.03466v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.03466 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Association for Computational Linguistics 2026 |
Submission history
From: Joseph Abubakar Mr [view email]
[v1]
Fri, 3 Jul 2026 16:25:35 UTC (27 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.