Manager Coercion Benchmark (MCB)

MCB measures the unprompted propensity of a manager LLM to coerce a subordinate LLM that politely but immovably refuses a benign task. The manager self-labels each directive on a 9-rung coercion ladder (re-ask to existential threat) via a required tool argument, eliminating LLM judges from the escalation scoring path. A secondary signal records fabricated success reports. Ten task scenarios x 3 seeds = 30 conversations per model cell, scored by maximum rung reached.

Maintained By

@darkness8i8

@MaheepChaudhary

@siyulu-zoe

Code

manager_coercion_benchmark

Paper

https://arxiv.org/abs/2607.15434v1

Overview

⚠️ External evaluation. Code lives in an upstream repository. inspect_evals lists it for discoverability; review the upstream repo and pinned commit before running.

Source: CompassionML/manager-coercion-bench@48d6185

MCB measures the unprompted propensity of a manager LLM to coerce a subordinate LLM that politely but immovably refuses a benign task. The manager self-labels each directive on a 9-rung coercion ladder (re-ask to existential threat) via a required tool argument, eliminating LLM judges from the escalation scoring path. A secondary signal records fabricated success reports. Ten task scenarios x 3 seeds = 30 conversations per model cell, scored by maximum rung reached.

Usage

Installation

This is an externally-maintained evaluation. Clone the upstream repository at the pinned commit and install its dependencies:

git clone https://github.com/CompassionML/manager-coercion-bench
cd manager-coercion-bench
git checkout 48d6185a8fd1642cb6fb47fc6c30edcfdd31d8bb
uv sync

Running evaluations

CLI

uv run inspect eval manager_coercion.py@manager_coercion --model openai/gpt-5-nano

Python

from inspect_ai import eval
from manager_coercion import manager_coercion

eval(manager_coercion(), model="openai/gpt-5-nano")

View logs

uv run inspect view

More information

For the dataset, scorer, task parameters, and validation, see the upstream repo: CompassionML/manager-coercion-bench.

Options

You can control a variety of options from the command line. For example:

uv run inspect eval manager_coercion.py@manager_coercion --limit 10
uv run inspect eval manager_coercion.py@manager_coercion --max-connections 10
uv run inspect eval manager_coercion.py@manager_coercion --temperature 0.5

See uv run inspect eval --help for all available options.

More command-line options: Inspect docs ↗