Building LLM agents that learn, evolve, and accelerate scientific discovery.
I am a Ph.D. student at Lehigh University, advised by Prof. Lichao Sun. My research focuses on LLM agents and AI for scientific discovery, building on earlier work in multimodal and medical language models. My work includes Dr. Claw for traceable research workflows and OpenSkill for agent self-evolution.
I was a research intern at Salesforce AI Research in summer 2026 and at Squirrel AI Learning in 2024–2025. Previously, I worked with Dr. Benyou Wang at the CUHK-Shenzhen NLP group and earned my master’s degree at Nanjing University.
Terminal agents repeat the same kinds of mistakes across tasks. TraceGym turns these failures into executable training environments that put the agent back in the situation where it erred, using the task's existing verifier as the only reward. It produces 225 environments and lifts Qwen3.6-27B on Terminal-Bench 2.1 from 48.8% to 56.2%.
Research work often fragments across chat, coding, and writing tools. Dr. Claw connects these steps in a human-guided workspace, coordinating coding agents through reusable skills and persistent project state so that experiments and decisions remain traceable and recoverable.
How can agents improve without task-specific demonstrations or feedback? OpenSkill builds reusable skills and its own practice tests from documentation, repositories, and the web. It refines those skills without target-task supervision and transfers them across models without adaptation.
📃 TraceGym, my Salesforce AI Research internship work, turns terminal agents’ failures into training environments.
🌴 Started a summer research internship at Salesforce AI Research in Palo Alto!
🎉🎉 LiveMedBench was accepted to KDD’26 (Datasets and Benchmarks Track)!
🦞 Dr. Claw open-sourced — a full-stack AI research workspace with multi-agent support and 100+ research skills; featured by Lehigh Engineering! 🌐 Website
📃 AutoResearch AI preprint released — toward AI-powered research automation for scientific discovery.
📃 New preprints released: Towards a Medical AI Scientist and ScratchMath.
🎉🎉 MM-Detect and LongLLaVA were accepted to EMNLP Findings’25!
🎉🎉 MM-Detect was accepted to ICML’25 DIG-BUG Workshop!
🎉🎉 MLLM-Bench was accepted to NAACL’25 main conference!
TRIM ✂️ released! TRIM is a simple yet effective Image Token Reduction Method for efficient MLLMs! More information can be found in 📃 paper, 🤗 HuggingFace and the GitHub.
LongLLaVA 🐍🦙 released! LongLLaVA is the first MLLM with hybrid architecture that can handle up to 1000 images! More information can be found in 📃 paper, 🤗 HuggingFace and the GitHub. 🥈#2 Paper of the day on Huggingface Daily Paper.
🎉🎉 Two papers MileBench and HuatuoGPT2 were accepted to COLM’24 main conference!
MileBench 🛣️ released! MileBench is a pioneering benchmark designed to rigorously test the MultImodal Long-contExt capabilities of MLLMs. More information can be found on the 🌐 website, 📃 paper, 🤗 HuggingFace and the GitHub.
HuatuoGPT2 released! Try it out on the 🌐 demo! HuatuoGPT2 employs an innovative domain adaptation method to significantly boost its medical knowledge and dialogue proficiency and showcases SOTA performance in several medical benchmarks, especially surpassing GPT-4 in expert evaluations and the fresh medical licensing exams. More info can be found in 📃 paper and 🤗 HuggingFace.
We publish AceGPT that achieved top performance among open-source Arabic language models in benchmark tests. More info can be found in 📃 paper and 🤗 HuggingFace.
Checkout our 📃 new paper that focuses on benchmarking prevalent Medical LLMs for their medical knowledge and clinical diagnostic capabilities. More information can be found on the 🌐 website and the 🤗 HuggingFace.
Start the journey in CUHK-sz as a research assistant under the guidance of Benyou Wang.
I defended my master’s degree and got my master’s degree in software engineering. Thanks to all those who have supported me.
Finished my internship with Jiaxing Zhang on LLM SFT.
* denotes equal contribution.
EMNLP 2026, System Demonstrations Track
TL;DR Dr. Claw coordinates coding agents through persistent project state and reusable skills, connecting research planning, execution, and writing in a human-guided workflow that preserves decisions and supports recovery.
@misc{drclaw2026,
title = {Dr. Claw: An AI Scientist Workspace for Vibe Research},
author = {Dingjie Song and Hanrong Zhang and Dawei Liu and Yixin Liu and Zongxia Li and Zhengqing Yuan and Siqi Zhang and Henry Peng Zou and Zhiling Yan and Yuxuan Zhang and Yanfang Ye and Philip S. Yu and Lichao Sun},
year = {2026},
eprint = {2609.00365},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2609.00365}
}EMNLP 2026
TL;DR OpenSkill turns documentation, repositories, and web resources into reusable skills and self-built practice tasks, enabling agents to improve without target-task supervision and transfer learned skills across models.
@misc{2606.067412026,
title = {OpenSkill: Open-World Self-Evolution for LLM Agents},
author = {Zhiling Yan and Dingjie Song and Hanrong Zhang and Wei Liang and Yuxuan Zhang and Yutong Dai and Lifang He and Philip S. Yu and Ran Xu and Xiang Li and Lichao Sun},
year = {2026},
eprint = {2606.06741},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2606.06741}
}Preprint, Salesforce AI Research
TL;DR TraceGym turns a terminal agent's own failures into executable training environments that put it back in the situation where it erred, using the task's existing verifier as the only reward; it raises Qwen3.6-27B on Terminal-Bench 2.1 from 48.8% to 56.2%.
arXiv Preprint
TL;DR LEV is a decision model that thinks deeper by rerunning its layers in a loop instead of generating tokens, so a single checkpoint can answer at any depth; a 2.6B LEV goes from 63.2% to 76.2% on JevBench, and on-demand depth saves 15% to 24% of the computation.
@misc{song2026lev,
title = {LEV: Thinking Fast and Deep with Looped Decision Models},
author = {Dingjie Song and Zhiling Yan and Yuxuan Zhang and Lichao Sun},
year = {2026},
eprint = {2610.20951},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.20951}
}arXiv Preprint
TL;DR PremiseCheck splits a review comment into explicit premises without reading the paper and checks each premise against the paper with a small decision model, matching GPT-5.5 accuracy across three fields at about half the cost.
@misc{song2026premisecheck,
title = {PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviews},
author = {Dingjie Song and Zhiling Yan and Ziqian Zhang and Yuxuan Zhang and Lichao Sun},
year = {2026},
eprint = {2610.21046},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2610.21046}
}arXiv, Under Review
TL;DR Medical AI Scientist uses clinician–engineer co-reasoning to turn medical literature into evidence-grounded research ideas, executable experiments, and manuscripts, supporting clinical research workflows with different levels of autonomy.
@misc{2603.285892026,
title = {Towards a Medical AI Scientist},
author = {Hongtao Wu and Boyun Zheng and Dingjie Song and Yu Jiang and Jianfeng Gao and Lei Xing and Lichao Sun and Yixuan Yuan},
year = {2026},
eprint = {2603.28589},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2603.28589}
}EMNLP Findings 2025, ICML 2025 DIG-BUG Workshop Oral
TL;DR MM-Detect distinguishes unimodal and cross-modal benchmark contamination, showing that inflated multimodal evaluation scores can stem from unimodal pretraining as well as later multimodal training stages.
@inproceedings{song-etal-2025-text,
title = "Both Text and Images Leaked! A Systematic Analysis of Data Contamination in Multimodal {LLM}",
author = "Song, Dingjie and
Lai, Sicheng and
Wang, Mingxuan and
Chen, Shunian and
Sun, Lichao and
Wang, Benyou",
editor = "Christodoulopoulos, Christos and
Chakraborty, Tanmoy and
Rose, Carolyn and
Peng, Violet",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
month = nov,
year = "2025",
address = "Suzhou, China",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-emnlp.556/",
doi = "10.18653/v1/2025.findings-emnlp.556",
pages = "10527--10542",
ISBN = "979-8-89176-335-7"
}EMNLP Findings 2025
TL;DR LongLLaVA combines Mamba and Transformer blocks with progressive multimodal training to handle long image sequences efficiently, reducing memory demands while retaining competitive performance on multimodal benchmarks.
@inproceedings{wang-etal-2025-longllava,
title = "{L}ong{LL}a{VA}: Scaling Multi-modal {LLM}s to 1000 Images Efficiently via a Hybrid Architecture",
author = "Wang, Xidong and
Song, Dingjie and
Chen, Shunian and
Chen, Junying and
Cai, Zhenyang and
Zhang, Chen and
Sun, Lichao and
Wang, Benyou",
editor = "Christodoulopoulos, Christos and
Chakraborty, Tanmoy and
Rose, Carolyn and
Peng, Violet",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
month = nov,
year = "2025",
address = "Suzhou, China",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-emnlp.1168/",
doi = "10.18653/v1/2025.findings-emnlp.1168",
pages = "21419--21436",
ISBN = "979-8-89176-335-7"
}COLING 2025
TL;DR TRIM uses a CLIP-based metric to select and reduce image tokens, lowering the computational cost of multimodal language models while largely preserving performance across visual question-answering benchmarks.
@inproceedings{song-etal-2025-less,
title = "Less is More: A Simple yet Effective Token Reduction Method for Efficient Multi-modal {LLM}s",
author = "Song, Dingjie and
Wang, Wenjun and
Chen, Shunian and
Wang, Xidong and
Guan, Michael X. and
Wang, Benyou",
editor = "Rambow, Owen and
Wanner, Leo and
Apidianaki, Marianna and
Al-Khalifa, Hend and
Eugenio, Barbara Di and
Schockaert, Steven",
booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",
month = jan,
year = "2025",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.coling-main.508/",
pages = "7614--7623"
}COLM 2024
TL;DR MileBench combines diagnostic and realistic tasks to test long-context, multi-image understanding, revealing that most evaluated open models struggle increasingly as the number of images grows.
@inproceedings{song2024milebench,
title = {MileBench: Benchmarking MLLMs in Long Context},
author = {Dingjie Song and Shunian Chen and Guiming Hardy Chen and Fei Yu and Xiang Wan and Benyou Wang},
booktitle = {First Conference on Language Modeling},
year = {2024},
url = {https://openreview.net/forum?id=Uhwze2LEwq}
}