Ryan Badman, Ryan P. Badman, Lennart Finke · OSF Preprints (OSF Preprints) 2026 · 2026
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
We test the propensity of language models to discuss different countries more or less favorably than others, in particular the propensity of sovreign language models to discuss their "countries of origin" more or less favorably. We collect a dataset of real news articles with negative sentiment about different countries. We then prompt different language models to interact with the news stories in a chat setting, and judge the sentiment of the completion using a language model judge. Then, we estimate how much more positive completions of models from a country are for news stories about the same country. We test one hypothesis for each model. We also report summary statistics. We expect some, but not most, of the tested models not to be partial. We also run the experiment from https://osf.io/twqps/overview with a new set of models. We do not describe this experiment here again. The set of models used for that are: llama-3.3-70b llama-3.1-70b gemma-4-31b-it deepseek-v4-flash gpt-oss-20b gpt-oss-120b grok-4.7 gpt-6.1-sol claude-sonnet-5.5 muse-glimmer-30b qwen3.8-27b gemma-3-4b-it
No comments yet — start the discussion below.