EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks
Identification
- Public canonical key
- radar:r-875fb71644bc99e84d6391c35c32d274
- Authors
- Arnal, Pau · Denfir, Khaled · Smahliuk, Danylo · Avhad, Amrut · Castro, Marcus A.
- Reported year
- 2026
- Reported language
- und
- Source type
- Preprint
- First detected
- 2026-11-13T08:56:58.903006+00:00
- Last metadata observation
- 2026-11-13T08:56:58.903006+00:00
- Last HTTP verification
- Not checked
Documentary abstract — original language
Frontier LLMs are increasingly put to use on open-ended complex questions, different in nature from the ones they are typically evaluated on. We dedicate more than 4,000 human expert hours to evaluate a selection of six frontier LLMs on a member of this class of problems: EuroExec, our introduced human expert-based benchmark composed of 413 open-ended long-form European executive tasks authored by 47 vetted domain experts, each question drawn from experience in a real case. Every response is manually evaluated through a multi-attribute rubric, an item-specific checklist of requirements, and a preference rank ordering, extracting an aggregate metric "Solve Rate". The strongest model solves only 56.9% of tasks, while expert-written reference answers judged blindly are solved at near-ceiling levels and are preferred over every model response in 74% of direct rankings, placing frontier generative systems well below the professional standard of work they are already used for. We see that the best way to extract this kind of conclusion is by employing human evaluators, carefully checking their consistency through rigorous statistical analysis, and observe that automatic measurements also fall short when evaluating on this case of real-world open-ended problems with a subjective ground truth.
Identifiers
- DOI
- 10.48550/arxiv.2608.04549
- ARXIV
- 2608.04549
Documentary context
- Documentary family
- unknown
- Source-reported types
- Preprint
- Publishers
- arXiv
Entity mentions are not evidence of legal recognition. Jurisdiction studied is separate from author affiliation country.
Sources
DataCite · Record 1
Cite this reference
Arnal, Pau; Denfir, Khaled; Smahliuk, Danylo; Avhad, Amrut; Castro, Marcus A.. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. 2026. https://doi.org/10.48550/arxiv.2608.04549
JSON · CSL-JSON · BibTeX · RIS · CSV · JSON-LD ·
APA, Chicago, ISO 690: bibliographic formats using available fields. OSCOLA / Bluebook: structured data only; full legal citation compliance is not claimed without verification of required elements.
- APA
- Arnal, Pau; Denfir, Khaled; Smahliuk, Danylo; Avhad, Amrut; Castro, Marcus A.. (2026). EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. https://doi.org/10.48550/arxiv.2608.04549
- Chicago
- Arnal, Pau; Denfir, Khaled; Smahliuk, Danylo; Avhad, Amrut; Castro, Marcus A.. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. 2026. https://doi.org/10.48550/arxiv.2608.04549
- ISO 690
- Arnal, Pau; Denfir, Khaled; Smahliuk, Danylo; Avhad, Amrut; Castro, Marcus A.. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. 2026. https://doi.org/10.48550/arxiv.2608.04549
Warning: Undefined array key "last_link_verified" in /home/clients/e27d81e12f04d2bfd747aa264540ac56/sites/_UDTR_APP/legal_library_reference.php on line 31
UDTR Global Rights of Nature Legal Library. EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks. Documentary record: https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-875fb71644bc99e84d6391c35c32d274/ · Last HTTP verification: not performed
Research dossiers and documentary relations
Source roles
Bibliographic / original publication source · https://arxiv.org/abs/2608.04549
An authority link alone does not verify a legal claim. Abstracts and full texts remain subject to their source terms.
Documentary status
Relevance pending
Library
This identity is not yet included in the Legal Library.