It is all in the [MASK]: Simple instruction-tuning enables BERT-like masked language models as generative classifiers

It is all in the [MASK]: Simple instruction-tuning enables BERT-like masked language models as generative classifiers

While encoder-only models such as BERT and ModernBERT are ubiquitous in real-world NLP applications, their conventional reliance on task-specific classification heads can limit their applicability compared to decoder-based large language models (LLMs). In this work, we introduce ModernBERT-Large-Ins...

Full description

Saved in:
Bibliographic Details
Main Authors: Benjamin Clavié, Nathan Cooper, Benjamin Warner
Format: Article
Language:English
Published: Elsevier 2025-06-01
Series:Natural Language Processing Journal
Subjects:
Zero-shot classification
Multiple-choice question answering
Encoder models
BERT
ModernBERT
Masked language modeling
Online Access:http://www.sciencedirect.com/science/article/pii/S2949719125000263
Tags: Add Tag
No Tags, Be the first to tag this record!

Internet

http://www.sciencedirect.com/science/article/pii/S2949719125000263

Similar Items