02322nas a2200253   4500000000100000000000100001008004100002260001200043653002500055653001200080653001900092653002300111653002300134653002100157100002500178700002800203700002800231245007000259856005800329300001000387490000600397520165100403022001402054       2024                            d  c03/202410aFormative Assessment10aChatGPT10aLinear Algebra10aMath Word Problems10aPolya’s Strategy10aPrompt Generator1 aNelly Rigaud Téllez1 aPatricia Rayón Villela1 aRoberto Blanco Bautista00aEvaluating ChatGPT-Generated Linear Algebra Formative Assessments  uhttps://www.ijimai.org/journal/bibcite/reference/3420  a75-820 v83 aThis research explored Large Language Models potential uses on formative assessment for mathematical problem-solving process. The study provides a conceptual analysis of feedback and how the use of these models is related in the context of formative assessment for Linear Algebra problems. Particularly, the performance of a popular model known as ChatGPT in mathematical problems fails on reasoning, proofs, model construction, among others. Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve student’s achievement of intended instructional outcomes. The study analyzed and evaluated feedback provided to engineering students in their solutions, from both, instructors and ChatGPT, against fine-grained criteria of a formative feedback model that includes affective aspects. Considering preliminary outputs, and to improve performance of feedback from both agents’ instructors and ChatGPT, we developed a framework for formative assessment in mathematical problemsolving using a Large Language Model (LLM). We designed a framework to generate prompts, supported by common Linear Algebra mistakes within the context of concept development and problem-solving strategies. In this framework, the instructor acts as an agent to verify tasks in a math problem assigned to students, establishing a virtuous cycle of learning of queries supported by ChatGPT. Results revealed potentialities and challenges on how to improve feedback on graduate-level math problems, by which both educators and students adapt teaching and learning strategies.  a1989-1660