Rayat Imtiaz Hossain

📊 Why do Large Vision-Language Models struggle with charts? Even the most advanced LVLMs often look at the wrong parts of a chart—fixating on irrelevant axes or labels—leading to wrong answers and making their reasoning hard to interpret. In domains like finance, medicine, or scientific research, this lack of interpretability is a huge barrier to trust. ✨ That’s where our new work ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinement comes in. We introduce ChartGaze, the first eye-tracking dataset for chart reasoning, capturing how humans actually look at charts when answering questions. With this, we: 🔹 Show that LVLM attention diverges significantly from human gaze. 🔹 Propose gaze-guided attention refinement to teach models where to look. 🔹 Improve both accuracy (+2.56%) and interpretability, making attention maps more human-like and reasoning more trustworthy. What’s inside ChartGaze? ✅ 1,620 real-world charts from Statista, Pew, and beyond ✅ 4,638 high-precision human gaze maps ✅ 4,638 chart QA pairs across trend analysis, extremum finding, and comparison reasoning 🚀 The result: LVLMs that don’t just answer better—but reason in a way that’s more aligned with human cognition. 👉 Paper: arxiv.org/pdf/2509.13282 💻 Code: github.com/alisalamatian1/Ch… 📊 Dataset: huggingface.co/datasets/alis… A huge thanks to my amazing co-authors and mentors: Ali Salamatian, @CyuanWan , @RayatHoss , and my supervisors Leonid Sigal and @careninigiusepp — for making this work possible! 🙏 #AI #MachineLearning #DeepLearning #NLP #ComputerVision #MultimodalAI #Interpretability #Charts #Visualization #Research #ChartQA #LVLMs