Effects of Varying LLM Access on Essay Writing Behavior
Quick Take
A study on 24 college students revealed that limited access to LLMs enhances perceived authorship and engagement in essay writing, while unlimited access leads to less creative expression. The findings suggest that constraining LLM use can support student learning outcomes effectively.
Key Points
- 24 college students participated in the study with varying LLM access.
- Limited access led to 62.5% of students claiming essays as independent work.
- Unlimited access resulted in essays resembling LLM outputs and reduced creativity.
- Students with limited access reported stronger organizational skills and strategic prompting.
- Constrained LLM access may preserve authorship confidence while aiding learning.
Article Content
From source RSS / original summaryarXiv:2606. 00250v1 Announce Type: new Abstract: Investigating the degree to which large language models (LLMs) affect teaching and learning in universities can help identify strategies for integrating LLMs in a way that supports, rather than undermines, student learning outcomes. This study examined how varying levels of LLM assistance affect writing performance, engagement, and perceived authorship.
We report a pilot study in which 24 college students were randomly assigned to write a short essay with no LLM access, limited access (<=3 prompts, responses capped at 100 words), or unlimited access. Overall essay quality was statistically indistinguishable across groups. Yet writing behavior and perceived authorship diverged sharply: students with limited access reported higher ownership (62. 5% would submit the essay as independent work, vs.
25% in the unlimited group), stronger organizational gains, and more strategic, revision-focused prompting. The unlimited group spent more time writing, produced essays more similar to LLM output, and reported reduced creative expression. Our findings suggest that constraining, rather than banning, LLM access may preserve authorship confidence while retaining the scaffolding benefits of AI assistance.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The REFLECT benchmark reveals that current LLM judges are unreliable, achieving below 55% accuracy in evaluating reasoning and evidence use, highlighting the need for improved evaluation methods for deep research agents.