Ad Headline Generation using Self-Critical Masked Language Model
Quick Answer
This study presents a novel approach for generating E-commerce advertising headlines using a Reinforcement Learning Policy gradient method applied to Transformer-based Masked Language Models.
Quick Take
This study presents a novel approach for generating E-commerce advertising headlines using a Reinforcement Learning Policy gradient method applied to Transformer-based Masked Language Models. The proposed method significantly outperforms existing models, including LSTM + RL, in both overlap metrics and quality audits, producing headlines that surpass human-generated ones in grammar and creativity.
Key Points
- Utilizes Reinforcement Learning Policy gradient methods on Transformer-based models.
- Outperforms LSTM + RL and existing Transformer methods in quality metrics.
- Generated headlines exceed human submissions in grammar and creativity.
- Addresses the challenge of creating scalable, high-quality E-commerce advertisements.
- Accepted at NAACL-HLT 2021, showcasing its relevance in the industry.
Paper Resources
📖 Reader Mode
~2 min readAbstract:For any E-commerce website it is a nontrivial problem to build enduring advertisements that attract shoppers. It is hard to pass the creative quality bar of the website, especially at a large scale. We thus propose a programmatic solution to generate product advertising headlines using retail content. We propose a state of the art application of Reinforcement Learning (RL) Policy gradient methods on Transformer based Masked Language Models. Our method creates the advertising headline by jointly conditioning on multiple products that a seller wishes to advertise. We demonstrate that our method outperforms existing Transformer and LSTM + RL methods in overlap metrics and quality audits. We also show that our model-generated headlines outperform human submitted headlines in terms of both grammar and creative quality as determined by audits.
| Comments: | Accepted at NAACL-HLT 2021 (Industry Track). 9 pages, 3 tables, 3 figures - ACL Anthology URL: this https URL - Editors of the proceedings: Young-bum Kim, Yunyao Li, Owen Rambow - Bibkey: kanungo-etal-2021-ad |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.06818 [cs.CL] |
| (or arXiv:2607.06818v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.06818 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers, pages 263-271, June 2021 |
| Related DOI: | https://doi.org/10.18653/v1/2021.naacl-industry.33
DOI(s) linking to related resources |
Submission history
From: Yashal Shakti Kanungo [view email]
[v1]
Tue, 7 Jul 2026 21:25:49 UTC (1,351 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.