BERT, are you paying attention? Attention regularisation with human-annotated rationales
Identification
- Public canonical key
- radar:r-4fcde613087c246da09b52622fd9e086
- Authors
- Elize Herrewijnen · Dong Nguyen · Floris Bex · Albert Gatt A · Sub Natural Language Processing · Recht innovatie en technologie · Sub Responsible AI · AI and Law Data · Montaigne Centrum voor Rechtsstaat en Rechtspleging · Vera Demberg · Kentaro Inui · Lluís Màrquez Villodre
- Reported year
- 2026
- Reported language
- en
- Source type
- book
- First detected
- 2026-11-14T02:38:36.181702+00:00
- Last metadata observation
- 2026-11-14T02:38:36.181702+00:00
- Last HTTP verification
- Not checked
Documentary abstract — original language
Attention regularisation aims to supervise the attention patterns in language models like BERT. Various studies have shown that using human-annotated rationales, in the form of highlights that explain why a text has a specific label, can have positive effects on model generalisability. In this work, we ask to what extent attention regularisation with human-annotated rationales improve model performance and model robustness, as well as susceptibility to spurious correlations. We compare regularisation on human rationales with randomly selected tokens, a baseline which has hitherto remained unexplored. Our results suggest that often, attention regularisation with randomly selected tokens yields similar improvements to attention regularisation with human-annotated rationales. Nevertheless, we find that human-annotated rationales surpass randomly selected tokens when it comes to reducing model sensitivity to strong spurious correlations.
Identifiers
- OPENALEX
- https://openalex.org/W7204976876
Documentary context
- Documentary family
- Book
- Source-reported types
- book
Entity mentions are not evidence of legal recognition. Jurisdiction studied is separate from author affiliation country.
Book bibliographic description
- Reported publication date
- 2026-03-01
Access status unknown
Sources
OpenAlex · Record 1
Cite this reference
Elize Herrewijnen; Dong Nguyen; Floris Bex; Albert Gatt A; Sub Natural Language Processing; Recht innovatie en technologie; Sub Responsible AI; AI and Law Data; Montaigne Centrum voor Rechtsstaat en Rechtspleging; Vera Demberg; Kentaro Inui; Lluís Màrquez Villodre. BERT, are you paying attention? Attention regularisation with human-annotated rationales. 2026. https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-4fcde613087c246da09b52622fd9e086/
JSON · CSL-JSON · BibTeX · RIS · CSV · JSON-LD ·
APA, Chicago, ISO 690: bibliographic formats using available fields. OSCOLA / Bluebook: structured data only; full legal citation compliance is not claimed without verification of required elements.
- APA
- Elize Herrewijnen; Dong Nguyen; Floris Bex; Albert Gatt A; Sub Natural Language Processing; Recht innovatie en technologie; Sub Responsible AI; AI and Law Data; Montaigne Centrum voor Rechtsstaat en Rechtspleging; Vera Demberg; Kentaro Inui; Lluís Màrquez Villodre. (2026). BERT, are you paying attention? Attention regularisation with human-annotated rationales. https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-4fcde613087c246da09b52622fd9e086/
- Chicago
- Elize Herrewijnen; Dong Nguyen; Floris Bex; Albert Gatt A; Sub Natural Language Processing; Recht innovatie en technologie; Sub Responsible AI; AI and Law Data; Montaigne Centrum voor Rechtsstaat en Rechtspleging; Vera Demberg; Kentaro Inui; Lluís Màrquez Villodre. BERT, are you paying attention? Attention regularisation with human-annotated rationales. 2026. https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-4fcde613087c246da09b52622fd9e086/
- ISO 690
- Elize Herrewijnen; Dong Nguyen; Floris Bex; Albert Gatt A; Sub Natural Language Processing; Recht innovatie en technologie; Sub Responsible AI; AI and Law Data; Montaigne Centrum voor Rechtsstaat en Rechtspleging; Vera Demberg; Kentaro Inui; Lluís Màrquez Villodre. BERT, are you paying attention? Attention regularisation with human-annotated rationales. 2026. https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-4fcde613087c246da09b52622fd9e086/
Warning: Undefined array key "last_link_verified" in /home/clients/e27d81e12f04d2bfd747aa264540ac56/sites/_UDTR_APP/legal_library_reference.php on line 31
UDTR Global Rights of Nature Legal Library. BERT, are you paying attention? Attention regularisation with human-annotated rationales. Documentary record: https://www.declarationuniverselledesdroitsdelarbre.org/global-rights-of-nature-radar/reference/r-4fcde613087c246da09b52622fd9e086/ · Last HTTP verification: not performed
Research dossiers and documentary relations
Source roles
Bibliographic / original publication source · https://dspace.library.uu.nl/handle/1874/486067
An authority link alone does not verify a legal claim. Abstracts and full texts remain subject to their source terms.
Documentary status
Relevance pending
Library
This identity is not yet included in the Legal Library.