LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
Identification
- Clé publique
- radar:r-3bf64529b262fef043bd96451f5a9f35
- Auteurs
- Ruilin Yao · Zhang, Bo · Jirui Huang · Xinwei Long · Yifang Zhang · Zou, Tianyu · Yu‐Fei Wu · Shichao Su · Yifan Xu · Zeng, Wenxi · Zhaoyu Yang · Li, Guoyou · Shilan Zhang · Zichan Li · Yaxiong Chen · Shengwu Xiong · Peng Xu · Jiajun Zhang · Bowen Zhou · David Andrew Clifton · Luc Van Gool
- Année déclarée
- 2025
- Langue déclarée
- en
- Type source
- preprint
- Première détection
- 2026-11-14T02:03:31.399789+00:00
- Dernière observation des métadonnées
- 2026-11-14T02:03:31.399789+00:00
- Dernière vérification HTTP
- Non effectuée
Institutions déclarées par la source
Wuhan University of Technology · Chinese Academy of Sciences · Institute of Automation · Tsinghua University · Shanghai Artificial Intelligence Laboratory · University of Oxford
Pays d’affiliation déclarés
CN · GB
Résumé documentaire — langue originale
Multimodal Large Language Models (MLLMs) have achieved significant advances in integrating visual and linguistic information, yet their ability to reason about complex and real-world scenarios remains limited. The existing benchmarks are usually constructed in the task-oriented manner without guarantee that different task samples come from the same data distribution, thus they often fall short in evaluating the synergistic effects of lower-level perceptual capabilities on higher-order reasoning. To lift this limitation, we contribute Lens, a multi-level benchmark with 3.4K contemporary images and 60K+ human-authored questions covering eight tasks and 12 daily scenarios, forming three progressive task tiers, i.e., perception, understanding, and reasoning. One feature is that each image is equipped with rich annotations for all tasks. Thus, this dataset intrinsically supports to evaluate MLLMs to handle image-invariable prompts, from basic perception to compositional reasoning. In addition, our images are manully collected from the social media, in which 53% were published later than Jan. 2025. We evaluate 15+ frontier MLLMs such as Qwen2.5-VL-72B, InternVL3-78B, GPT-4o and two reasoning models QVQ-72B-preview and Kimi-VL. These models are released later than Dec. 2024, and none of them achieve an accuracy greater than 60% in the reasoning tasks. Project page: https://github.com/Lens4MLLMs/lens. ICCV 2025 workshop page: https://lens4mllms.github.io/mars2-workshop-iccv2025/
Identifiants
- OPENALEX
- https://openalex.org/W4415329133
- DOI
- 10.48550/arxiv.2505.15616
Repères documentaires
- Famille documentaire
- preprint
- Types déclarés par les sources
- preprint
Les mentions d’entités ne constituent pas une preuve de reconnaissance juridique. La juridiction étudiée est distincte du pays d’affiliation des auteurs.
Sources
OpenAlex · Document 1
Citer cette référence
Ruilin Yao; Zhang, Bo; Jirui Huang; Xinwei Long; Yifang Zhang; Zou, Tianyu; Yu‐Fei Wu; Shichao Su; Yifan Xu; Zeng, Wenxi; Zhaoyu Yang; Li, Guoyou; Shilan Zhang; Zichan Li; Yaxiong Chen; Shengwu Xiong; Peng Xu; Jiajun Zhang; Bowen Zhou; David Andrew Clifton; Luc Van Gool. LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. 2025. https://doi.org/10.48550/arxiv.2505.15616
JSON · CSL-JSON · BibTeX · RIS · CSV · JSON-LD ·
APA, Chicago, ISO 690 : formats bibliographiques fondés sur les champs disponibles. OSCOLA / Bluebook : données structurées uniquement ; aucune conformité juridique complète n’est revendiquée sans vérification des éléments nécessaires.
- APA
- Ruilin Yao; Zhang, Bo; Jirui Huang; Xinwei Long; Yifang Zhang; Zou, Tianyu; Yu‐Fei Wu; Shichao Su; Yifan Xu; Zeng, Wenxi; Zhaoyu Yang; Li, Guoyou; Shilan Zhang; Zichan Li; Yaxiong Chen; Shengwu Xiong; Peng Xu; Jiajun Zhang; Bowen Zhou; David Andrew Clifton; Luc Van Gool. (2025). LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. https://doi.org/10.48550/arxiv.2505.15616
- Chicago
- Ruilin Yao; Zhang, Bo; Jirui Huang; Xinwei Long; Yifang Zhang; Zou, Tianyu; Yu‐Fei Wu; Shichao Su; Yifan Xu; Zeng, Wenxi; Zhaoyu Yang; Li, Guoyou; Shilan Zhang; Zichan Li; Yaxiong Chen; Shengwu Xiong; Peng Xu; Jiajun Zhang; Bowen Zhou; David Andrew Clifton; Luc Van Gool. LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. 2025. https://doi.org/10.48550/arxiv.2505.15616
- ISO 690
- Ruilin Yao; Zhang, Bo; Jirui Huang; Xinwei Long; Yifang Zhang; Zou, Tianyu; Yu‐Fei Wu; Shichao Su; Yifan Xu; Zeng, Wenxi; Zhaoyu Yang; Li, Guoyou; Shilan Zhang; Zichan Li; Yaxiong Chen; Shengwu Xiong; Peng Xu; Jiajun Zhang; Bowen Zhou; David Andrew Clifton; Luc Van Gool. LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. 2025. https://doi.org/10.48550/arxiv.2505.15616
Warning: Undefined array key "last_link_verified" in /home/clients/e27d81e12f04d2bfd747aa264540ac56/sites/_UDTR_APP/legal_library_reference.php on line 31
UDTR — Bibliothèque juridique mondiale des droits de la Nature. LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. Notice documentaire: https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-3bf64529b262fef043bd96451f5a9f35/ · Dernière vérification HTTP : non effectuée
Données de cette version figée
Dossiers et relations documentaires
Rôles des sources
Source bibliographique / publication d’origine · https://arxiv.org/abs/2505.15616
Un lien vers une autorité ne prouve pas à lui seul une affirmation juridique. Les résumés et les textes intégraux restent soumis aux conditions de leur source.
Statut documentaire
Pertinence à examiner
Bibliothèque
Cette identité n’est pas encore intégrée à la Bibliothèque juridique.