Populism-LLM

Cross-Validated LLM Annotations of European Party Manifestos

A cross-validated populism corpus for political-economic research

Three independent large language models โ€” Anthropic Claude Sonnet 4.6, OpenAI GPT-4.1-mini, and Google Gemini Flash โ€” score every CMP party manifesto from 1920 to 2025 on twelve dimensions of populism and liberalism. Each score comes with a verbatim quote, contextual snippet, and machine-readable evidence status.

5,285 manifestos

67 countries

2 LLM families

12 scored dimensions

What you get

๐Ÿ“‚ Browse 3,327 manifestos

Open browser โ†’ Country, party, year filters. Click a manifesto to see every score from every model, with the verbatim evidence each model relied on.

๐Ÿ”€ Cross-model agreement

Open agreement view โ†’ Where do the three LLMs agree? Where do they diverge? Per-dimension Pearson and Spearman correlations, full disagreement-flagged outlier list.

๐Ÿ“ˆ Interactive regressions

Open regression tool โ†’ Filter by country, year, choose which models to include, pick within-manifesto and between-model aggregation weights โ€” run the panel regression in your browser. Powered by WebR.

โฌ‡๏ธ Download

Get the data โ†’ Bulk parquet via Hugging Face. DOI-citable mirror via Dataverse. Code on GitHub with Zenodo archive.

Methodology in one paragraph

Each manifesto is split into โ‰ค20,000-token chunks. Each chunk is scored by all three LLMs using an identical strict JSON schema. Models must back every non-null score with a 3โ€“8-word verbatim quote and a surrounding context snippet. Evidence spans are revalidated post-hoc against the input text with a 6-tier fuzzy matcher plus an LLM-as-judge backstop. Final scores are confidence-weighted means across chunks, computed independently per model; the cross-model aggregate is a configurable choice (default: simple mean of the three families). All choices can be inspected and changed in the interactive regression module and visualised as a specification curve.

โ†’ Full pipeline: methodology.

Citation

@article{stoeckl2026populism,
  author  = {Stoeckl, Sebastian},
  title   = {Populism and Liberalism in European Party Manifestos:
             A Cross-Validated LLM Approach},
  year    = {2026},
  note    = {Working paper; dataset DOI: tba}
}