LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via Hybrid Architecture
EMNLP Findings 2025
Xidong Wang*, Dingjie Song*, Shunian Chen, Chen Zhang, Benyou Wang
* denotes equal contribution.
TL;DR LongLLaVA combines Mamba and Transformer blocks with progressive multimodal training to handle long image sequences efficiently, reducing memory demands while retaining competitive performance on multimodal benchmarks.