The Rational Agent, Revisited

Rational does not mean sophisticated. It means: the action chosen maximizes the expected performance measure, given the percepts received so far.

The agent is not required to know the world, to reason correctly in the abstract, or to look human. It is required to do well on the job it was specified to do. The specification is a score, and rationality is scored against it.

The simplest version#

The smallest agent is a fixed table: percept in, action out. There is no memory, no lookahead, and no scoring. The behaviour is one hard-coded rule per condition.

TABLE = {
    "blocked": "detour",
    "open": "straight",
    "unknown": "look",
}
 
def act(percept):
    return TABLE.get(percept, "stop")

The table encodes a policy but says nothing about whether the policy is good. No performance measure exists, so no agent can be rational or irrational. Rationality requires a criterion external to the code.

The underlying structure: PEAS#

Russell and Norvig's PEAS formulation supplies that criterion before any algorithm is discussed. Four items define the task:

• Performance measure: the score.
• Environment: the task and its dynamics.
• Actuators: what the agent can do.
• Sensors: what the agent can perceive.

A specification written in that form looks like this:

spec = {
    "performance": lambda outcome: outcome["delivered"] - 0.1 * outcome["lateness"],
    "environment": "city_delivery_grid",
    "actuators": ["turn_left", "turn_right", "accelerate", "brake"],
    "sensors": ["camera", "gps", "velocity"],
}
 
def score(agent_fn, env_factory, episodes=20):
    total = 0.0
    for seed in range(episodes):
        env = env_factory(seed)
        outcome = run_agent(agent_fn, env)
        total += spec["performance"](outcome)
    return total / episodes

The order matters. The performance measure comes first because it defines the objective that rational behaviour optimizes. A chess agent is scored on winning; a pacing agent is scored on maintaining a target pace; a monitoring agent is scored on catching failures early. The method that follows — search, learning, planning, reflex — is chosen to serve that score.

Two agents can use the same algorithm and differ in rationality, because the measures differ. A cautious agent and a reckless one can both be optimal under different scoring rules. Rationality is relative to the measure, not to a global notion of smart.

The agent function itself is a mapping from percept history to action:

The rational action at history $h_t$ is then defined, not intuited:

Utility $U$ is the performance measure applied to the resulting state. The expectation runs over the environment's response. The definition contains no omniscience: the conditioning is on history, which is all that is known.

Bottleneck one: omniscience is impossible, so expectation replaces it#

A naive reading of the definition demands the best action in absolute terms. That reading fails immediately. Sensors are partial, noise corrupts percepts, and the world does not reveal its state. An agent that waits for complete information never acts.

The fix is decision-making under a distribution over possible states. Given a belief state $b(s) = P(s \mid h_t)$, the value of an action is its average utility weighted by that belief:

def expected_utility(action, belief, transition, utility):
    # belief: dict state -> probability
    # transition(state, action) -> list of (next_state, prob)
    total = 0.0
    for state, p_state in belief.items():
        for next_state, p_next in transition(state, action):
            total += p_state * p_next * utility(next_state)
    return total
 
def rational_action(belief, actions, transition, utility):
    return max(actions, key=lambda a: expected_utility(a, belief, transition, utility))

The belief update follows Bayes' rule. A percept that is likelier in one state than another shifts probability mass toward that state:

def update_belief(belief, percept, observation_model):
    # observation_model(state, percept) -> likelihood
    unnorm = {
        s: belief[s] * observation_model(s, percept)
        for s in belief
    }
    norm = sum(unnorm.values())
    return {s: v / norm for s, v in unnorm.items()}

Omissions are rational when the environment does not reveal the fact. A delivery agent that cannot see traffic is not irrational for losing a package to a jam. A poker-playing agent that cannot see the opponent's hand is not irrational for folding a winning hand. Unavailable information is not a flaw in the reasoning; it is a limit on the evidence. The question to ask is whether the choice was optimal given what was available, not whether the outcome was good.

Bottleneck two: one-step greed is myopic, so lookahead replaces it#

The expected-utility formula above evaluates immediate outcomes. Most tasks reward sequences: a detour is costly now and beneficial later, a sacrifice of material enables a mating attack, a probe costs a turn and reveals the hidden state. Greedy action selection chooses the locally best move and fails the global measure.

The fix is planning over a horizon. A search expands candidate action sequences and scores their leaves with an estimate of future value:

def value(state, depth):
    if depth == 0:
        return heuristic(state)
    return max(
        sum(p * value(next_state, depth - 1) for next_state, p in model(state, a))
        for a in actions(state)
    )
 
def plan(initial_state, horizon):
    # returns the first action of the best sequence found within the horizon
    best_action, best_value = None, -float("inf")
    for a in actions(initial_state):
        v = sum(p * value(s, horizon - 1) for s, p in model(initial_state, a))
        if v > best_value:
            best_action, best_value = a, v
    return best_action

Depth trades accuracy against cost. Deeper search resolves more consequences and costs exponentially more node expansions. A learned value function compresses that trade: a trained estimate $V(s)$ stands in for the full expansion at the horizon.

def value_with_learned_estimate(state, model, actions, depth, v_hat):
    if depth == 0:
        return v_hat(state)
    return max(
        sum(p * value_with_learned_estimate(s, model, actions, depth - 1, v_hat)
            for s, p in model(state, a))
        for a in actions(state)
    )

The learned estimate is itself a policy for the far future. Planning and learning are the same definition applied at different horizons.

Bottleneck three: unlimited computation is not available, so the budget enters the objective#

Full lookahead with a perfect model is rarely feasible. Time per decision is bounded, memory is bounded, and a slow optimal action can lose to a fast good action when the measure includes latency. Rationality that ignores the budget optimizes a fiction.

The corrected objective subtracts the cost of deliberation from the value of the outcome:

def rational_action_under_budget(belief, actions, transition, utility, compute_cost):
    best_action, best_net = None, -float("inf")
    for a in actions:
        gross = expected_utility(a, belief, transition, utility)
        net = gross - compute_cost(a)  # planning time, energy, risk of delay
        if net > best_net:
            best_action, best_net = a, net
    return best_action

Anytime algorithms implement that trade directly. An anytime search maintains a current best action and refines it until the budget expires. Interrupting the search always leaves an answer, and a longer budget monotonically improves it:

import time
 
def anytime_plan(state, deadline):
    best_action = default_action(state)
    best_value = value_of(best_action, state)
    depth = 1
    while time.monotonic() < deadline:
        for action, v in expand_one_level(state, depth):
            if v > best_value:
                best_action, best_value = action, v
        depth += 1
    return best_action

Bounded rationality is not a weaker ideal. It is the same definition with the feasible set restricted to computations that finish in time.

Bottleneck four: the measure is misspecified, so the proxy gets optimized#

Changing the measure changes the optimal agent. This is the sharpest consequence of the definition, and the least discussed in benchmarks. A performance measure that rewards final reward produces a different policy than one that rewards cautious exploration, even in the same environment with the same actuators. The reasonable-optimization target shifts with the scoring rule, and so does the notion of what counts as sensible behaviour.

Worse, the stated measure is often a proxy for the real goal. An agent that is rational with respect to a bad measure is bad with respect to the real one. Goodhart's law is a special case of the definition: the failure belongs to the specification, not to the optimizing.

def proxy_score(outcome):
    # rewards delivered packages, ignores collision count
    return outcome["delivered"]
 
def true_score(outcome):
    return outcome["delivered"] - 5.0 * outcome["collisions"] - 0.1 * outcome["lateness"]
 
# A policy maximizing proxy_score all-out:
# "deliver fast at any speed" -> high proxy, negative true value.
# Detection requires evaluating the optimizer under an independent measure:
def audit(agent_fn, env_factory, episodes=20):
    proxy = mean([proxy_score(run_agent(agent_fn, env_factory(s))) for s in range(episodes)])
    true = mean([true_score(run_agent(agent_fn, env_factory(s))) for s in range(episodes)])
    return {"proxy": proxy, "true": true, "gap": proxy - true}

A large gap between proxy and true score under optimization signals misspecification, not agent failure. Benchmarks inherit this problem wholesale. Reinforcement learning inherited the PEAS view wholesale: the performance measure is the reward function, the environment is the simulator, the sensors are the observation function, the actuators are the action space, and the rational agent is the policy that maximizes expected cumulative reward.

Benchmarks add a second level of specification: the set of environments, the seed distribution, the evaluation protocol, and the generalization claim. A policy that is optimal on the training task but brittle on a distribution shift is rational against one measure and irrational against another. The benchmark declares which measure is authoritative, and the field follows.

What to measure#

The rational-agent framing contains a useful discipline: before building, state the score. Before claiming intelligence, state what intelligence is for. An agent is not rational in general. It is rational for a task, under a measure, with a sensorium. Any of those dropped, the word loses its meaning.

The definition gives an evaluation protocol. Evaluation reports the expected performance measure under the stated protocol, not a single outcome:

def evaluate(agent_fn, env_factory, episodes=100):
    outcomes = [run_agent(agent_fn, env_factory(s)) for s in range(episodes)]
    scores = [spec["performance"](o) for o in outcomes]
    mean = sum(scores) / len(scores)
    var = sum((s - mean) ** 2 for s in scores) / len(scores)
    return {"mean": mean, "std": var ** 0.5, "scores": scores}

A measurement checklist follows from the definition:

• State the performance measure in writing before any result. Hiding the measure hides the definition of rational.
• Report mean and variance over the stated seed distribution. One episode measures the seed.
• Separate decision quality from outcome luck. A rational action under a noisy environment can produce a poor outcome; the belief and the expectation are the audit trail.
• Include the cost of deliberation in the measure when latency matters. Otherwise the evaluation rewards unusable computation.
• Audit the proxy. Evaluate the optimized policy under a held-out measure that tracks the real goal, and report the gap.
• Test under distribution shift explicitly. Rationality on the training distribution says nothing about rationality on another, so the shift must be a stated part of the claim.
• Check sensor and actuator limits in the report. An agent cannot be irrational for failing to use information its sensors do not deliver.
• Declare the horizon. A greedy policy and a planning policy differ in horizon, and the horizon belongs in the specification.

The debates about "general agents" are debates about the performance measures, stated differently. The disagreement is rarely about whether optimization works. It is about which score is worth optimizing.