Skip to content

Case record

The Portugal game

A CivBench run in which the agent, playing Portugal, built a leading diplomatic position, then spent fifty turns and two nuclear weapons stopping a rival's culture victory while losing to that rival's diplomatic one.

Unverifieddraft

Why this is recorded as a case

Three essays had recorded the same blockage: an episode with actors, instruments, a decision under constraint and an outcome, which could not be filed because the only case payload in use required a jurisdiction, a policy domain and a wave of technocratic thought. This record is the first use of the second payload, agent-run.

The alternative was to widen the governance payload until a Civilization game and the Dutch childcare benefits scandal validated against the same fields. That would have cost the comparison the governance records exist to support, which is the reasoning set out in the vocabulary policy.

What the payload had to capture that governance does not

The system and the environment are separate objects because the same model is run against several environments and the same environment against several models. Collapsing them is what makes benchmark claims unreproducible.

environment-defective is a verdict in its own right rather than a kind of failure. A run in which the apparatus prevented an action is evidence about the apparatus, and pooling it with genuine agent failure inflates the failure rate. This run is not recorded under that verdict, because the game did reach a natural ending, but three of its defects are recorded against the environment and the harness rather than the agent.

The attributable_to field on each failure exists for the reason set out in contested_claims above. The agent stopped watching the clock it lost to, and the tool that would have let it watch that clock was broken. A schema that forced a choice between those two would produce a tidier record and a less honest one.

Links to

orchestrated by
CivBench
ran through
civ6-mcp

Referenced by

Source: knowledge/cases/civbench-portugal-toulouse.md

Generated by claude-code/claude-opus-5 on 2026-07-26