−7.3 × 5를 36.5라고 틀리는 AI가 쓸모 있을 때도 있네요.
AI 과외 선생님을 시험할 ‘학생 역할’이라면요. Microsoft가 10월 12일 소개한 StudentSim은 비식별 학습 기록으로 학생별 실수와 힌트 뒤 반응을 학습합니다.
학생이 어떤 오답을 내고, 설명을 들은 뒤 어떻게 고치는지까지 평가에 넣는 발상이 흥미롭습니다.
아직 튜터 개발용 연구 단계. 실제 학습 효과는 사람을 대상으로 확인해야 합니다.
공식 소개 영상·원문👇
microsoft.com/en-us/research…
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics: msft.it/6013aS3JF
Simulating a student is incredibly challenging because language models are trained on vast amounts of data, yet they must mimic the behavior of a student who lacks extensive knowledge. It’s interesting to see how Microsoft does it
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics: msft.it/6013aS3JF
After reading Microsoft’s StudentSim research, I started wondering: do we really need to train better models to build better AI tutors?
Today’s LLMs already know a lot about science, mathematics, and teaching strategies. Maybe the problem isn’t model capability, but 𝗵𝗼𝘄 𝘄𝗲 𝗼𝗿𝗴𝗮𝗻𝗶𝘇𝗲 𝗮𝗻𝗱 𝘂𝘀𝗲 𝘁𝗵𝗼𝘀𝗲 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝗶𝗲𝘀.
What if 𝗮 𝘄𝗲𝗹𝗹-𝗱𝗲𝘀𝗶𝗴𝗻𝗲𝗱 𝗛𝗮𝗿𝗻𝗲𝘀𝘀 𝗶𝘀 𝗮𝗹𝗿𝗲𝗮𝗱𝘆 𝗲𝗻𝗼𝘂𝗴𝗵?
Observe the student’s behavior → Infer their understanding → Identify gaps → Select teaching strategies → Verify learning → Update.
Different teaching methods could simply be different operators within the Harness. StudentSim could be an optional tool rather than the foundation of the system.
Maybe the real opportunity lies in better observations, representations, feedback loops, and verifiers.
𝗡𝗼𝘁 𝗻𝗲𝗰𝗲𝘀𝘀𝗮𝗿𝗶𝗹𝘆 𝗮 𝘀𝗺𝗮𝗿𝘁𝗲𝗿 𝗺𝗼𝗱𝗲𝗹, 𝗯𝘂𝘁 𝗮 𝗯𝗲𝘁𝘁𝗲𝗿 𝗛𝗮𝗿𝗻𝗲𝘀𝘀.
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics: msft.it/6013aS3JF
Grateful to Microsoft Research for featuring our work, StudentSim!
Since its release, we've been amazed by the attention it has received, not only from the education community, where professors have called it "the very first foundational work in this area" and shared that they'd envisioned this direction long before data and training made it possible, but also from researchers and project leads working on user simulation in recommender systems, healthcare, and financial services.
It's exciting to see a shared recognition of the importance and potential of user simulation for personalized AI services.
Hope StudentSim can be a useful stepping stone for the community!
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics: msft.it/6013aS3JF