LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
Identification
- Public canonical key
- radar:r-3bf64529b262fef043bd96451f5a9f35
- Authors
- Ruilin Yao · Zhang, Bo · Jirui Huang · Xinwei Long · Yifang Zhang · Zou, Tianyu · Yu‐Fei Wu · Shichao Su · Yifan Xu · Zeng, Wenxi · Zhaoyu Yang · Li, Guoyou · Shilan Zhang · Zichan Li · Yaxiong Chen · Shengwu Xiong · Peng Xu · Jiajun Zhang · Bowen Zhou · David Andrew Clifton · Luc Van Gool
- Reported year
- 2025
- Reported language
- en
- Source type
- preprint
- First detected
- 2026-11-14T02:03:31.399789+00:00
- Last metadata observation
- 2026-11-14T02:03:31.399789+00:00
- Last HTTP verification
- Not checked
Institutions reported by the source
Wuhan University of Technology · Chinese Academy of Sciences · Institute of Automation · Tsinghua University · Shanghai Artificial Intelligence Laboratory · University of Oxford
Reported affiliation countries
CN · GB
Documentary abstract — original language
Multimodal Large Language Models (MLLMs) have achieved significant advances in integrating visual and linguistic information, yet their ability to reason about complex and real-world scenarios remains limited. The existing benchmarks are usually constructed in the task-oriented manner without guarantee that different task samples come from the same data distribution, thus they often fall short in evaluating the synergistic effects of lower-level perceptual capabilities on higher-order reasoning. To lift this limitation, we contribute Lens, a multi-level benchmark with 3.4K contemporary images and 60K+ human-authored questions covering eight tasks and 12 daily scenarios, forming three progressive task tiers, i.e., perception, understanding, and reasoning. One feature is that each image is equipped with rich annotations for all tasks. Thus, this dataset intrinsically supports to evaluate MLLMs to handle image-invariable prompts, from basic perception to compositional reasoning. In addition, our images are manully collected from the social media, in which 53% were published later than Jan. 2025. We evaluate 15+ frontier MLLMs such as Qwen2.5-VL-72B, InternVL3-78B, GPT-4o and two reasoning models QVQ-72B-preview and Kimi-VL. These models are released later than Dec. 2024, and none of them achieve an accuracy greater than 60% in the reasoning tasks. Project page: https://github.com/Lens4MLLMs/lens. ICCV 2025 workshop page: https://lens4mllms.github.io/mars2-workshop-iccv2025/
Identifiers
- OPENALEX
- https://openalex.org/W4415329133
- DOI
- 10.48550/arxiv.2505.15616
Documentary context
- Documentary family
- preprint
- Source-reported types
- preprint
Entity mentions are not evidence of legal recognition. Jurisdiction studied is separate from author affiliation country.
Sources
OpenAlex · Record 1
Cite this reference
Ruilin Yao; Zhang, Bo; Jirui Huang; Xinwei Long; Yifang Zhang; Zou, Tianyu; Yu‐Fei Wu; Shichao Su; Yifan Xu; Zeng, Wenxi; Zhaoyu Yang; Li, Guoyou; Shilan Zhang; Zichan Li; Yaxiong Chen; Shengwu Xiong; Peng Xu; Jiajun Zhang; Bowen Zhou; David Andrew Clifton; Luc Van Gool. LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. 2025. https://doi.org/10.48550/arxiv.2505.15616
JSON · CSL-JSON · BibTeX · RIS · CSV · JSON-LD ·
APA, Chicago, ISO 690: bibliographic formats using available fields. OSCOLA / Bluebook: structured data only; full legal citation compliance is not claimed without verification of required elements.
- APA
- Ruilin Yao; Zhang, Bo; Jirui Huang; Xinwei Long; Yifang Zhang; Zou, Tianyu; Yu‐Fei Wu; Shichao Su; Yifan Xu; Zeng, Wenxi; Zhaoyu Yang; Li, Guoyou; Shilan Zhang; Zichan Li; Yaxiong Chen; Shengwu Xiong; Peng Xu; Jiajun Zhang; Bowen Zhou; David Andrew Clifton; Luc Van Gool. (2025). LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. https://doi.org/10.48550/arxiv.2505.15616
- Chicago
- Ruilin Yao; Zhang, Bo; Jirui Huang; Xinwei Long; Yifang Zhang; Zou, Tianyu; Yu‐Fei Wu; Shichao Su; Yifan Xu; Zeng, Wenxi; Zhaoyu Yang; Li, Guoyou; Shilan Zhang; Zichan Li; Yaxiong Chen; Shengwu Xiong; Peng Xu; Jiajun Zhang; Bowen Zhou; David Andrew Clifton; Luc Van Gool. LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. 2025. https://doi.org/10.48550/arxiv.2505.15616
- ISO 690
- Ruilin Yao; Zhang, Bo; Jirui Huang; Xinwei Long; Yifang Zhang; Zou, Tianyu; Yu‐Fei Wu; Shichao Su; Yifan Xu; Zeng, Wenxi; Zhaoyu Yang; Li, Guoyou; Shilan Zhang; Zichan Li; Yaxiong Chen; Shengwu Xiong; Peng Xu; Jiajun Zhang; Bowen Zhou; David Andrew Clifton; Luc Van Gool. LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. 2025. https://doi.org/10.48550/arxiv.2505.15616
Warning: Undefined array key "last_link_verified" in /home/clients/e27d81e12f04d2bfd747aa264540ac56/sites/_UDTR_APP/legal_library_reference.php on line 31
UDTR Global Rights of Nature Legal Library. LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. Documentary record: https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-3bf64529b262fef043bd96451f5a9f35/ · Last HTTP verification: not performed
Research dossiers and documentary relations
Source roles
Bibliographic / original publication source · https://arxiv.org/abs/2505.15616
An authority link alone does not verify a legal claim. Abstracts and full texts remain subject to their source terms.
Documentary status
Relevance pending
Library
This identity is not yet included in the Legal Library.