[瑞士]乔舒亚·魏德利希1, 2 [澳]德拉甘·伽瑟维克3 [德]亨德里克·德拉克斯勒4, 5 ,6 [荷]保罗·柯施纳6, 7
（1.苏黎世大学，苏黎世 8001，瑞士；2.苏黎世教师教育大学，苏黎世 8090，瑞士；3.莫纳什大学，维多利亚 3800，澳大利亚；4.莱布尼茨教育研究和教育信息研究所，法兰克福 60323，德国；5.歌德大学，法兰克福 60323，德国；6.荷兰开放大学，海尔伦 6401DL，荷兰；7.托马斯摩尔应用科技大学，安特卫普 2860，比利时）
肖俊洪 译
【摘 要】ChatGPT等人工智能工具提升学习效果的潜能是一个当下的研究热点，研究者纷纷加入这一研究，但是长期困扰教育技术研究的那些缺陷同样影响到这些聚焦新兴技术的研究的效度。如果在媒介比较研究中混淆了教学方法和技术能供性，我们可能无法解释促使学习效果发生变化的真正原因。本文重新审视20世纪八九十年代关于“媒介与方法”辩论的主要观点，以期让学界关注到围绕ChatGPT效能的研究中存在常见的概念混淆的问题。本文是一篇概念性文章，通过比较ChatGPT教育应用这一新兴研究和关于智能辅导系统的更加成熟的研究，总结出确保学习效果的可解释的三个不可或缺的要素：①准确描述实验处理的性质；②准确描述对照组的活动；③对学习结果的测量应该能够有效地反映学习的情况。为了初步验证上述三要素，分析了近期ChatGPT教育应用的实验研究，结果表明只有极少数研究满足这三个要素的要求。实验处理缺乏严格界定、对照条件不合适或不清晰且不能肯定结果测量与持久学习相关——这些问题导致我们无法确信这些新兴研究中ChatGPT的使用与学习效果的提升存在因果关系。鉴于此，实验观察到的学习效果的提升不一定是使用ChatGPT的结果，对相关文献进行元分析所得出的效应量可能夸大或低估了ChatGPT促进学习的作用。我们认为必须保证研究设计严谨、相关信息透明且对“快科学”持批判性态度方能促进此研究领域的进步。
【关键词】ChatGPT；人工智能；媒介比较；缺陷；实验；元分析
ChatGPT in Education: An Effect in Search of A Cause
Joshua Weidlich1,2, Dragan Gašević3, Hendrik Drachsler4,5,6 and Paul Kirschner6,7
(1.University of Zurich, Zurich 8001, Switzerland; 2. Zurich University of Teacher Education, Zurich 8090, Switzerland; 3. Monash University, Clayton,Victoria 3800, Australia; 4. DIPF-Leibniz Institute for Research and Information in Education, Frankfurt am Main 60323, Germany; 5. GoetheUniversität, Frankfurt am Main 60323, Germany; 6. Open University of the Netherlands, Heerlen 6401DL, the Netherlands; 7. Thomas More Universityof Applied Sciences, Antwerpen 2860, Belgium)
Abstract: As researchers rush to investigate the potential of AI tools like ChatGPT to enhance learning, well-documented pitfalls threaten the validity of this emerging research. Issues of media comparison research, where the confounding of instructional methods and technological affordances is unrecognized, may render effects uninterpretable. Using a recent meta-analysis by Deng et al. (Computers & Education, 227, 105224) as an example, we revisit key insights from the media/methods debate to highlight recurring conceptual challenges in ChatGPT efficacy studies. This conceptual article contrasts nascent ChatGPT research with the more established literature on Intelligent Tutoring Systems to identify three non-negotiable considerations for interpretable effects: (1) descriptions of the precise nature of the experimental treatment and (2) the activities of the control group, as well as (3) outcome measures as valid indicators of learning.To provide some initial evidence, we audited a subset of primary experiments included in Deng et al.'s meta-analysis, demonstrating that only a small minority of studies satisfied all three non-negotiable considerations. Loosely defined treatments, mismatched or opaque controls, and outcome measures with unclear links to durable learning obscure causal claims of this emerging literature. Observed gains cannot, at this time, be confidently attributed to ChatGPT, and meta-analytics effect sizes may over-or understate its benefits. Progress, we argue, will require rigorous designs, transparent reporting, and a critical stance toward “fast science.”
Keywords: ChatGPT; artificial intelligence; media comparison; pitfalls; experiment; meta-analysis
下载： 
ChatGPT的教育应用：提升学习效果的真正原因是什么？.pdf
