Which Way Did It Move? Diagnosing and Overcoming Directional Motion Blindness in Video-LLMs
NeurIPS 2026
Research Office
Post-Master Researcher
Vision and Learning Lab, Kyung Hee University
I study how visual experience becomes knowledge of the world, from interpretable video models to video world models.
Keywords: video understanding, video world models, interpretability, multimodal learning
Prefer a plain page? Classic site ↗
You are visitor — thanks for stopping by.
Click a numbered object to open it · drag to look around
I recently completed my M.S. in Computer Science at Kyung Hee University and am continuing my research at the same lab as a Post-Master Researcher, advised by Prof. Jinwoo Choi. I am currently seeking PhD positions in the United States (Fall 2027).
I study how visual experience becomes knowledge of the world, from interpretable video models to video world models. Ultimately, I hope to build an agent that watches the world, understands it, and knows what comes next.
Before graduate school, I earned dual bachelor's degrees in Biomedical Engineering and Electronics Engineering. That interdisciplinary background — signal processing, embedded systems, and AI — still shapes how I approach research today.
Multimodal models that see motion, hear sound and read language together, and answer from what actually happens on screen.
Concept-level explanations of what video models learn, and checks on whether what a model represents is what it says.
Models that learn how the physical world works from observation, so an agent can know what comes next before it acts.
This thread runs across my work: diagnosing failure modes of Video-LLMs in DeltaDirect (NeurIPS 2026), concept-level explanation in DANCE (NeurIPS 2025 Spotlight), audio–visual reasoning in CA²ST (IEEE TPAMI 2026), and continual learning in ESSENTIAL (ICCV 2025 Highlight).
* equally contributed first authors · † corresponding author
NeurIPS 2026
NeurIPS 2026
Project Page Paper coming soon
CVPR 2025 · XAI4CV WorkshopSpotlight · top 16.7%
IEEE TPAMI 2026
arXiv 2024 Preprint, under review
Journal of Appropriate Technology 8(2), 2022
I was selected as a Top Reviewer for NeurIPS 2026.
Two of our papers have been accepted to NeurIPS 2026! “Which Way Did It Move? Diagnosing and Overcoming Directional Motion Blindness in Video-LLMs” (DeltaDirect), which diagnoses and fixes directional motion blindness in Video-LLMs, and “TRACE: Trajectory-Aware Concepts for Explainable Video Understanding” (project page), which extends concept-based explanation to object trajectories over time. Grateful to all my collaborators, including Wooil Lee, Junpyo Hong, Hyuntak Lee, Tuomas Oikarinen, Tsui-Wei Weng, and Jinwoo Choi.
Our paper “CA2ST: Cross-Attention in Audio, Space, and Time for Holistic Video Recognition” has been accepted to IEEE TPAMI, one of the most prestigious journals in computer vision and AI. This work extends our NeurIPS 2023 paper CAST with audio-visual reasoning in a unified cross-attention framework. Huge thanks to my collaborators Joohyun Chang and Jinwoo Choi. arXiv (2503.23447)
Our paper “Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition” has been accepted to NeurIPS 2025 as a Spotlight (3.5% acceptance rate). The work disentangles motion dynamics, objects, and scenes into human-understandable concepts — strong performance with clear explanations of model decisions. Grateful to my collaborators Wooil Lee, Gyeong-Moon Park, Seong Tae Kim, and Jinwoo Choi.
My paper “ESSENTIAL: Episodic and Semantic Memory Integration for Video Class-Incremental Learning” has been accepted to ICCV 2025 as a Highlight (top 2.9%) — after three submission attempts. The journey was filled with revisions and rejections, but it proved to be an incredible learning experience. Thank you to everyone who supported me along the way.
Ultimately, I hope to build an agent that watches the world, understands it, and knows what comes next, before it acts. My position paper argues that world models should learn to perceive before they act.
Models that learn how the physical world works from observation, so an agent can know what comes next before it acts.
Seeking PhD positions in the U.S., Fall 2027
If you think I could be a good fit for your group, I would love to hear from you.
jong980812@khu.ac.krWhether it's about PhD opportunities, research collaboration, or just to say hi — my inbox is always open.
I am always open to collaboration and discussion on video understanding, interpretability, and video world models. Feel free to reach out!