UDTR Global Radar

UDTR Global Rights of Nature Radar

Documentary registry of records detected across the scholarly, legal and institutional infrastructures monitored by UDTR.

A global documentary infrastructure designed to achieve the broadest verifiable and scientifically defensible coverage of the Rights of Nature.

← Back to Radar

EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks

Identification

Public canonical key
radar:r-875fb71644bc99e84d6391c35c32d274
Authors
Arnal, Pau · Denfir, Khaled · Smahliuk, Danylo · Avhad, Amrut · Castro, Marcus A.
Reported year
2026
Reported language
und
Source type
Preprint
First detected
2026-11-13T08:56:58.903006+00:00
Last metadata observation
2026-11-13T08:56:58.903006+00:00
Last HTTP verification
Not checked

Documentary abstract — original language

Frontier LLMs are increasingly put to use on open-ended complex questions, different in nature from the ones they are typically evaluated on. We dedicate more than 4,000 human expert hours to evaluate a selection of six frontier LLMs on a member of this class of problems: EuroExec, our introduced human expert-based benchmark composed of 413 open-ended long-form European executive tasks authored by 47 vetted domain experts, each question drawn from experience in a real case. Every response is manually evaluated through a multi-attribute rubric, an item-specific checklist of requirements, and a preference rank ordering, extracting an aggregate metric "Solve Rate". The strongest model solves only 56.9% of tasks, while expert-written reference answers judged blindly are solved at near-ceiling levels and are preferred over every model response in 74% of direct rankings, placing frontier generative systems well below the professional standard of work they are already used for. We see that the best way to extract this kind of conclusion is by employing human evaluators, carefully checking their consistency through rigorous statistical analysis, and observe that automatic measurements also fall short when evaluating on this case of real-world open-ended problems with a subjective ground truth.

Identifiers

DOI
10.48550/arxiv.2608.04549
ARXIV
2608.04549

Documentary context

Documentary family
unknown
Source-reported types
Preprint
Publishers
arXiv

Entity mentions are not evidence of legal recognition. Jurisdiction studied is separate from author affiliation country.

Sources

DataCite · Record 1

Cite this reference

Arnal, Pau; Denfir, Khaled; Smahliuk, Danylo; Avhad, Amrut; Castro, Marcus A.. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. 2026. https://doi.org/10.48550/arxiv.2608.04549

JSON · CSL-JSON · BibTeX · RIS · CSV · JSON-LD ·

APA, Chicago, ISO 690: bibliographic formats using available fields. OSCOLA / Bluebook: structured data only; full legal citation compliance is not claimed without verification of required elements.

APA
Arnal, Pau; Denfir, Khaled; Smahliuk, Danylo; Avhad, Amrut; Castro, Marcus A.. (2026). EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. https://doi.org/10.48550/arxiv.2608.04549
Chicago
Arnal, Pau; Denfir, Khaled; Smahliuk, Danylo; Avhad, Amrut; Castro, Marcus A.. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. 2026. https://doi.org/10.48550/arxiv.2608.04549
ISO 690
Arnal, Pau; Denfir, Khaled; Smahliuk, Danylo; Avhad, Amrut; Castro, Marcus A.. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. 2026. https://doi.org/10.48550/arxiv.2608.04549

Warning: Undefined array key "last_link_verified" in /home/clients/e27d81e12f04d2bfd747aa264540ac56/sites/_UDTR_APP/legal_library_reference.php on line 31

UDTR Global Rights of Nature Legal Library. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. Documentary record: https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-875fb71644bc99e84d6391c35c32d274/ · Last HTTP verification: not performed

Report a correction

Frozen-version metadata

Research dossiers and documentary relations

Source roles

Bibliographic / original publication source · https://arxiv.org/abs/2608.04549

An authority link alone does not verify a legal claim. Abstracts and full texts remain subject to their source terms.

Documentary status

Relevance pending

Library

This identity is not yet included in the Legal Library.

UDTR Global Rights of Nature Legal Library