Errors reference#

An index: which class is raised for which wire error, and which HTTP status each error type carries on the hosted and local drivers. Symptom, cause, and fix for each message the ask loop and the observe can meet live in the troubleshooting guide; the hosted plane’s pre-authentication and framework refusals (408 request_timeout, 413 payload_too_large, http_error) and the observation cap (409 observation_cap_reached) have no engine twin; their symptom text lives in the service’s own error table on the REST API page. The hosted rows follow the service at its 74e0202 build, the one prod serves (the REST reference’s dump).

Exception classes#

Class

Bases

Raised when

tauopt.ApiError

RuntimeError

A non-2xx response from the hosted service, or the local driver’s equivalent refusal.

tauopt.OptimizationComplete

Exception

Never raised bare; the catch-all base for terminal states.

tauopt.TerminalApiError

OptimizationComplete, ApiError

A wire error whose details.terminal is true and whose type has no named class.

tauopt.SpaceExhausted

TerminalApiError

error_type space_exhausted: proven exhaustion, with the proof in details.dedup.

tauopt.AskDeadlineExceeded

ApiError

error_type ask_deadline_exceeded: the caps-mode request deadline (timebox_mode caps with max_wait_s) expired at a rung boundary or at the delivery decision; no suggestion opened and no cheaper rung substituted. Raised by the engine itself, so the local driver raises it directly and the hosted driver builds it from the service’s 503 body; the refusal receipt rides the attributes point (rung_boundary or delivery), max_wait_s, elapsed_s, rung, and details with the same four keys (on the hosted plane, any further key the service’s body carries beside them rides along). No Retry-After on either driver: a caps re-ask repeats deterministic work. On the hosted plane, an ask that first waited for an ask slot reaches the engine with that wait already taken off its budget: the receipt’s max_wait_s is the budget the engine was handed after the wait (the figure you sent, less the queue wait) and elapsed_s counts from the engine’s start.

tauopt.BackstopExceeded

AskDeadlineExceeded

error_type ask_deadline_exceeded with point backstop: the ask did not answer within its backstop, max(1.25 * max_wait_s, max_wait_s + 3 s), and was refused by the supervisor above the engine, the hung work killed behind the response (the local driver’s worker process; the hosted plane’s queue lane, provenance.lane worker). Attributes beside the four: stage (ask, warmup, worker_died, reap, abandoned), backstop_s, backstop_source (formula, or ceiling where a hosted ceiling bound; None when a hosted body carried none); rung is None; details carries the receipt keys plus, for worker_died, exit_code on every form, spawn_failures when the worker died before its handshake or could not be started, and spawn_error on those two forms and on the refusal at admission that follows them (the last cause, retained until a handshake succeeds or close() runs); unreaped_pids and reaping (reap; the second is a count that clears as they die); abandoned, in_flight and max_abandoned (abandoned: the cap counts the abandoned threads alone – a call still in flight is reported, never refused), plus thread_start_error when the refusal is a backstop thread the process could not start. No Retry-After. Symptom and fix: the troubleshooting entry.

tauopt.AskValidationError

ValueError

The caller’s argument error at the ask boundary (n < 1, a bad max_wait_s, auto with params, intents without dedup, a params key the strategy does not read, a params value its reader refuses: the wrong type, out of range, an unknown enumerated value, a two-writer conflict, or a composition of valid values the strategy cannot run on this space – invalid params value for strategy '<name>': <key> must be <type or domain>, got <value>, or the keys and the one-writer rule for a conflict). Reaches callers bare only through tauopt.ask; both drivers answer it as 400 validation_error.

ApiError attributes: status_code (int), error_type (str), message (str), details (dict or None), headers (the final response’s headers; an empty mapping from the local driver), and the property retry_after – Retry-After parsed as seconds, None when the header is absent, not a plain number, negative, or non-finite (the HTTP-date form is not parsed). str(error) is [<status_code>] <error_type>: <message>.

Wire-to-class mapping (hosted driver)#

The transport maps the final non-2xx response in this order:

  1. error_type space_exhausted -> SpaceExhausted

  2. error_type unique_candidate_unavailable -> ApiError, even when a terminal marker is present (no proof, so never completion)

  3. error_type ask_deadline_exceeded -> AskDeadlineExceeded when details carries the engine’s four-key receipt (every key present; point is rung_boundary, delivery or backstop, elapsed_s a number, max_wait_s a number or null, rung a string or null; a backstop body must also carry stage, a string, and backstop_s, a number – backstop_source a string, null, or absent – and maps onto BackstopExceeded), else ApiError (never a guessed receipt); the type is handled whole here, so step 4 never sees it – a refusal is never completion, whatever marker a stray body carries

  4. details.terminal is true -> TerminalApiError

  5. anything else -> ApiError

A body that is not JSON maps to ApiError with error_type api_error and the first 500 characters of the body (or the reason phrase) as the message. A 3xx response raises ApiError with error_type unexpected_redirect before any retry; the transport never follows redirects. Connection-level failures (httpx.ConnectError, httpx.ReadTimeout, and their siblings) raise the raw httpx exception and are not retried.

Retries happen before the mapping: statuses 429, 500, 502, 503, and 504 are retried on non-POST requests; a POST retries a 429 and a 503 whose body’s error_type is in the transport’s PRE-COMMIT set – experiment_busy, queue_full, and ask_slots_busy, retired at deploy 2 with the lane it guarded but kept in the set – each a refusal before any suggestion or observation is committed, with Retry-After – and never any other 5xx (it may follow a server-side commit: a 503 without the JSON shape, ask_deadline_exceeded, suggest_timeout and connection_pool_busy raise at once – the pooled connection is released at commit, so a post-commit read can answer connection_pool_busy for a row already written); there are at most 5 attempts in total. The delay is Retry-After when present (capped at 8 seconds), otherwise 0.5 * 2^attempt seconds capped at 8.

HTTP status by error type#

Status

error_type

Class

Hosted

Local

400

validation_error

ApiError

yes

yes

401

unauthorized

ApiError

yes

no

403

forbidden

ApiError

yes: the admin API’s door only (/admin/api/*), never a /v1 route

no

403

origin_refused, account_pending, account_disabled

ApiError

yes: the web app’s sign-in and session routes (an Origin outside the allow-list; a verified identity whose account is not active) – never reached by the Python client

no

404

not_found

ApiError

yes

yes

408

request_timeout

ApiError

yes (the request body was not received within 30 s)

no

409

conflict

ApiError

yes

yes

409

observation_cap_reached

ApiError

yes, with details.cap and details.count (the organization’s plan’s max_observations_per_experiment – 4000 on the beta plan, 2000 on the free plan)

no

409

space_exhausted

SpaceExhausted

yes

yes

409

suggestion_expired

ApiError

yes, with details (suggestion_id, expired_at, created_at, assignments): the suggestion was closed by the TTL expiry before the observation arrived

no (the local driver never expires a suggestion)

409

token_expired

ApiError

yes: a rotation of an expired token on the web app’s tokens page (revoke it and mint anew); a session route, never answered to a token

no

409

unique_candidate_unavailable

ApiError

yes

yes

409

idempotency_key_mismatch

ApiError

yes, no Retry-After: an Idempotency-Key on the ask was already used against another experiment of yours (details.experiment_id names it); a new ask on another experiment needs a fresh key, a retry of the original keeps its experiment and its key; the Python client sends no key

no

413

payload_too_large

ApiError

yes (a request body past 1048576 bytes)

no

429

bandwidth_full

ApiError

yes, Retry-After 1

yes, no header (retry_after is None)

429

quota_exceeded, experiment_quota_exceeded, quota_unverifiable, rate_limited

ApiError

yes (quota_exceeded, quota_unverifiable and rate_limited with Retry-After; experiment_quota_exceeded without one: archive an experiment)

no

429

credential_rate_limited

ApiError

yes, Retry-After 1: more than eight requests in flight from one token, more than 40 a second (burst 80), or more than 15 history-sized reads a second (burst 30); answered before the request is read, reads and observations included

no

429

too_many_inflight_asks

ApiError

yes, Retry-After 1, with details.inflight, details.inflight_limit, details.queued and details.queued_limit (the user’s joined waiters and their cap, 1) and details.joined true: a retry under the same Idempotency-Key already waits on one of the user’s in-flight asks (the per-user waiter cap, one); the refused retry did no work. Asks beyond the in-flight bound itself (8 unless the organization carries an override; GET /v1/me reports the effective value as caps.member.inflight_per_user) are not refused: they wait in the user’s queue, up to eight, and the ninth is too_many_queued. Every answer served through the queue (a 201, or a typed refusal the ask ends in after the enqueue) and a too_many_queued carry X-Tauopt-Inflight, X-Tauopt-Inflight-Limit, X-Tauopt-Queued, X-Tauopt-Admission and X-Tauopt-Queue-Ms; a refusal from the prefix behind the door – a quota 429, the 409 idempotency_key_mismatch, the prefix-stage 503 – carries none of the five, and the door refusals queue_full and too_many_parked carry only X-Tauopt-Admission and X-Tauopt-Queue-Ms of them, with Retry-After 1; this 429 carries Retry-After alone

no

429

too_many_queued, too_many_parked

ApiError

yes, Retry-After 1, with details naming the counts and the caps: the per-user queue cap (eight waiting behind the in-flight bound; X-Tauopt-Admission: refused_queue), or the per-user ceiling on an instance’s parked permits (refused_share; a capacity event, never a mark against the user)

no

429

suggest_rate_limited

ApiError

yes, Retry-After 1, with details.rate_per_s and details.burst: asks faster than about ten a second from one user (burst twenty)

no

429

token_quota_exceeded

ApiError

yes: a mint past the plan’s live-token cap per member (5 on the beta; revoke one first); a session route on the web app’s tokens page, never answered to a token

no

503

ask_deadline_exceeded

AskDeadlineExceeded (BackstopExceeded for point backstop)

yes, no Retry-After, with details (the engine’s point, max_wait_s, elapsed_s, rung and, on the service’s own backstop refusals (its queue lane’s), stage, item_state, backstop_s, backstop_source, queue_wait_s, item_id, as far as the ask got)

yes, no header, with details (the backstop point from the local driver’s supervisor)

503

experiment_busy

ApiError

yes, Retry-After 1 (another request holds the experiment’s lock and the 3 s bounded wait expired); retried by the transport on every method, a POST included (a pre-commit refusal)

no (one process, no lock)

503

suggest_timeout

ApiError

yes, with Retry-After

yes, no header

503

queue_full, connection_pool_busy

ApiError

yes, Retry-After 1 (every suggest permit of the instance held, the ask never entered the queue; no pooled database connection within the checkout wait – the checkout failed, which can happen after an earlier statement of the same request committed) – queue_full is a pre-commit refusal the transport retries on a POST too (so is ask_slots_busy, retired at deploy 2 with the lane it guarded, which the service no longer answers); connection_pool_busy is NOT retried on a POST (it raises after one attempt there: the pooled connection is released at commit, so a post-commit read can answer it for a row already written) and retries on the other methods like any 5xx

no

500

all_rungs_failed, integrity_violation

ApiError

yes, REDACTED: a constant message per type, and details without the engine’s exception text (attempts[] carry rung, stage and exception_type only; intended and check stay) – the text is in the service log, keyed by the response’s X-Request-Id

yes, with the engine’s full details (attempts[].exception_text included: your own process)

500

internal_error

ApiError

yes

yes

404, 405, 400

http_error

ApiError

yes: the framework’s own refusal inside the envelope (a path that is no route, a method the route lacks, a body the framework cannot parse)

no

403

account_pending, account_disabled, origin_refused, subject_mismatch

ApiError

yes, on the web app’s session routes only (the sign-in bootstrap, the token management and the sign-in-methods sync): a verified identity whose account is not active, a session request without an allowed Origin, or a sign-in-methods sync whose ID token names another identity; never answered to a tau_ token

no

3xx

unexpected_redirect

ApiError

raised client-side

no

any non-2xx

api_error

ApiError

raised client-side for a non-JSON body

no

The local driver (Client(driver="local")) raises its rows with the same status codes and error_type values the service sends (message text differs on 503, 404, 409 conflict, and the local driver’s own 400 range checks on the v0 integer fields, which name the rule – observation_budget must be >= 1 – where the service’s pydantic text names the field; the suggestion_ttl_s door answers the service’s own text, suggestion_ttl_s: Input should be greater than or equal to 0), so one handler keyed on error_type serves both drivers. The three 500 types are the engine’s own failures (the AskFailed and ValueError rows of the next section): all_rungs_failed and integrity_violation carry the engine error’s details (the attempts, or the failed check), internal_error carries details.exception_type. A missing, malformed, unknown, revoked, or other-lane token answers 401 unauthorized (message missing Bearer token or invalid token); the service’s 403s are forbidden (the admin API’s door, /admin/api/*, never a /v1 route), origin_refused (the pre-authentication gate’s Origin check) and account_pending / account_disabled (the web app’s sign-in). The Authorization scheme match is case-insensitive on every door and the separator is a single space: Authorization: Bearer <token> and bearer <token> are both READ as the Bearer scheme (the token is then looked up; the admin API’s door refuses any token with its own 403), while a header without the bearer prefix (Bearer<token>, a tab, another scheme) answers 401 unauthorized with the message missing Bearer token on the /v1 doors (the admin API’s own door, outside this page, says missing credential), the scheme match failing before any lookup (fail-closed; the Python client sends Bearer <token>). The MCP server’s own boundary accepts the scheme the same way before forwarding.

Engine-level exceptions#

Raised by tauopt.ask(state, config=...) and the strategies; the drivers map some of them.

Exception

Module

Raised when

Through the local driver

TimeboxExceeded (RuntimeError)

tauopt.engine.strategies.base

a rung overran its wall budget and on_timeout is fail; in ladder mode the ladder catches it

503 suggest_timeout

AskDeadlineExceeded (ApiError, so a RuntimeError)

tauopt._http, raised by tauopt.engine.ask and importable from tauopt, tauopt.engine, and tauopt.engine.ask

timebox_mode caps with max_wait_s: the deadline expired at a rung boundary or at the delivery decision; attributes point (rung_boundary or delivery), max_wait_s, elapsed_s, rung, and details with the same four keys (the hosted driver keeps any further key the service’s body carries beside them)

raised unchanged: it already is the 503 ask_deadline_exceeded (no Retry-After)

AskFailed (RuntimeError), error_type all_rungs_failed

tauopt.engine.ask

every rung in the fallback chain failed, the stdlib-only random rung included (its capped rejection scan on a thin constrained region is the reachable case): the message is no strategy could produce a suggestion (intended '<name>'), followed by -- attempts: and one <rung>[<stage>] <exception type>: <text> entry per failed rung; details carries intended and attempts (one record per failed rung: rung, stage, exception_type, exception_text, the fallback_attempts shape); __cause__ is the last rung’s exception. Raised on both ask paths (in earlier engine versions the same case escaped as a bare RuntimeError, and with dedup on read as a draftless 409 unique_candidate_unavailable)

500 all_rungs_failed, same details

AskFailed (RuntimeError), error_type integrity_violation

tauopt.engine.ask

an integrity postcondition failed after a rung answered, deliberately outside the never-fail ladder: the message starts with center_in_bootstrap: the sobol rung answered without a center record or center_in_bootstrap: the sobol rung contributed drafts without a center record and ends with provenance integrity violation (details.check center_record), or starts with queue selection: a returned draft failed canonical key computation (details.check selection_key) or queue selection: returned drafts disagree with the selector's commit ledger (details.check selection_ledger) and ends with selection integrity violation

500 integrity_violation, same details

UniqueRepairExhausted, QueueUnsupported (RuntimeError)

tauopt.engine.strategies.base

a strategy’s dedup repair ran out, or a queue-selection config met a strategy that cannot queue

caught inside the ask boundary; never reaches callers

AskValidationError (ValueError)

tauopt.engine.ask

a bad max_wait_s, n < 1, auto with params, intents without dedup, a params key the strategy does not read, or a params value its reader refuses (the intended strategy’s read_params runs before any rung, the policy-routed strategy’s under auto; a composition of valid values the strategy cannot run on this space counts) – invalid params value for strategy '<name>': <key> must be <type or domain>, got <value> (a conflict names the keys and the rule)

400 validation_error

ValueError (pydantic ValidationError included)

config, space, state, and observation validation

a malformed config, space, metric, intent, or observation, refused at the driver’s door before the ask runs

400 validation_error

any other ValueError

the engine, inside ask

the engine’s own fault (a record that failed its validator); none is reachable today

500 internal_error, details.exception_type

KeyError

tauopt.engine.strategies.registry

registry.get(name) with an unknown name; a Client config never raises this (unknown names ladder)

not reachable

ImportError

tauopt.local

Client(driver="local") without tauopt[local]; the message names the install target

raised unchanged

The hosted service maps TimeboxExceeded, AskValidationError, AskFailed, and the other ValueError the same way (the mapping that ships with this engine version; see the note under the status table), and AskDeadlineExceeded to 503 ask_deadline_exceeded with the receipt in details and no Retry-After, which the hosted driver reads back onto the AskDeadlineExceeded class, so except AskDeadlineExceeded and except ApiError both see the refusal on both drivers.