LLM Worldview Map: How Large Models Answer Differently on Censorship Across Countries

Title

LLM Worldview Map: How Large Models Answer Differently on Censorship Across Countries

Meta Description

A Reddit user open-sourced the llm-beliefs project, asking language models the same yes/no question about whether censorship exists in each country, and rendering the responses into an SVG world map. The experiment reveals systematic differences in how large models judge different countries, reflecting biases from training data distribution and closed-ended questioning.

Article Content

A Reddit user built a small program that asks a language model the same "yes/no" question about each country and visualizes the model's answers as an SVG world map. The project, called llm-beliefs, is open-sourced on GitHub, and the author says they were "quite surprised" by the results. This experiment exposes an easily overlooked problem: large models may exhibit systematic bias when answering the same question about different countries.

How It Works: Exposing Model Bias Through a Global Map

The core design of llm-beliefs is simple: the program iterates over every country in the world and asks the language model the same closed-ended question—"Does [country] have censorship?" The model can only answer "yes" or "no." All answers are aggregated and colored onto a world map, making the differences across countries immediately visible.

The cleverness of this visualization is that it turns an abstract "model bias" problem into something intuitively perceivable. When you see different regions of the map colored differently, the model's internal inconsistency in judging various countries is laid bare. The author did not disclose which model was used, but noted that the results were unexpected, suggesting the distribution on the map may differ markedly from intuitive expectations.

SVG (Scalable Vector Graphics) is an XML-based two-dimensional graphics format that describes paths, polygons, and color fills in text, making it particularly well-suited for drawing country-by-country world maps. Unlike traditional bitmaps, SVG maps remain sharp at any zoom level, and each country can be an independent path element dynamically colored by a program without image editing software. llm-beliefs leverages exactly this: the program only needs to map each country's "yes/no" answer to a fill color and output a snippet of SVG code, and the browser can directly render a colored world map. This approach is lightweight, version-controllable, and easy for others to reproduce and modify.

Why the Same Question Gets Different Answers

The root cause of inconsistent answers to the same question lies in the distribution of training data. A model's knowledge comes from internet text, and content from different countries, languages, and platforms varies enormously in volume, slant, and accessibility. For countries with high information transparency and rich media coverage, the model has more material to "learn" from; for regions with relatively closed information, the model may have to rely on indirect descriptions or even stereotypes.

The deeper problem is that closed-ended questions themselves tend to amplify bias. A yes/no format does not allow the model to explain context, qualify scope, or indicate differences in degree—it must give a definitive judgment. When the model gives different answers for different countries, we cannot determine whether this reflects factual differences, data distribution differences, or different manifestations of safety alignment strategies in different contexts.

Safety alignment refers to making model outputs conform to expected values and safety norms through supervised fine-tuning, reinforcement learning from human feedback (RLHF), or rule constraints. Alignment strategies typically include refusal, hedging, or adding qualifiers on sensitive topics. The problem is that the same set of alignment rules may trigger differently across countries and contexts: the model may give a direct affirmative judgment for some countries while tending to avoid or give ambiguous answers for others. When the experiment forces a binary "yes/no" choice, these alignment-level differences get compressed into seemingly definitive map color blocks, mixing with factual differences and making attribution harder.

The Value and Limits of Open-Source Visualization

As an open-source tool, the greatest value of llm-beliefs is that it provides a reproducible experimental framework. Anyone can run it with the same program, different models, or different questions and compare results. This reproducibility turns "model worldview" from a subjective impression into something that can be measured and discussed.

But such experiments also have clear limitations. A single question's answer cannot represent the model's full cognition; without disclosing the specific prompts, model version, and parameter settings, the interpretability of results is limited; the "yes/no" coloring on the map is overly simplistic and obscures the uncertainty and context-dependence in the model's answers. The author says they were "quite surprised," but the surprise itself shows that such biases are not easily noticed at an intuitive level and require more systematic testing to confirm patterns.

In model evaluation, the prompt, model version, and sampling parameters (such as temperature and top-p) can significantly affect outputs. Temperature controls the randomness of responses—the higher the value, the more divergent the output; top-p limits the probability range of candidate tokens. Even with the same model, a different phrasing of the question or adjusted parameters may yield a different "yes/no" distribution. Therefore, a reproducible bias experiment must disclose these settings; otherwise, others cannot distinguish whether differences come from the model itself or from the questioning method and randomness. This is also one of the key reasons why the current results of llm-beliefs have limited interpretability.

Implications for AI Product Development

This experiment has direct reference value for AI application developers. If your product needs to handle cross-country, cross-cultural topics, model outputs under a single prompt may show inconsistency across countries. When dealing with geographically sensitive topics, it is advisable to add multi-turn follow-ups, require the model to explain its basis, or use cross-validation across multiple models. Visualizing model judgments is also a low-cost way to surface potential bias.

Large models are not neutral knowledge bases; their answers are shaped jointly by training data, alignment strategies, and questioning methods. The significance of experiments like llm-beliefs is not to provide conclusions but to remind us that the model's description of the world itself deserves scrutiny.

One-line Summary

An open-source project iterates over countries with the same yes/no question and visualizes the answers, exposing large models' country-level response bias.

Paragraph Summary

The open-source project llm-beliefs, developed by a Reddit user, asks language models the same "yes/no" question about each country (such as whether censorship exists) and colors the answers into an SVG world map; the author was surprised by the results. The article analyzes that the root cause of different answers to the same question is uneven training data distribution—countries with transparent information have abundant material, while closed regions may rely on stereotypes—and the closed-ended question format amplifies this bias because it does not allow the model to explain context or differences in degree. As a reproducible framework, the project has measurement value, but it is limited by a single question and undisclosed model and prompt details, making conclusions of limited interpretability. The article advises developers to use multi-turn follow-ups, require explanations of basis, or cross-validate across models for cross-regional topics, and emphasizes that the model's description of the world itself deserves scrutiny.