Both Text and Images Leaked! A Systematic Analysis of Multimodal LLM Data Contamination

Dingjie Song*, Sicheng Lai*, Mingxuan Wang, Shunian Chen, Lichao Sun, Benyou Wang

* denotes equal contribution.

TL;DR MM-Detect distinguishes unimodal and cross-modal benchmark contamination, showing that inflated multimodal evaluation scores can stem from unimodal pretraining as well as later multimodal training stages.