TauOpt
Troubleshooting#
The errors a run meets, by symptom: every error the client itself raises, plus
the engine errors the guides document (the how-to pages quote the space and
metric refusals they exercise). API-surface errors arrive as
tauopt.ApiError with the message shape [<status>] <type>: <message>, the
parsed fields on .status_code / .error_type / .message, and any
machine-readable payload under .details. Terminal completion states
additionally subclass tauopt.OptimizationComplete (see “The ask loop”
below). Runnable examples for most of these flows live in
python/examples/ (see the
index).
The documentation tree (quickstart, how-to guides, reference, explanation) starts at the documentation index; this page is its error catalog.
tauopt-bench refusals: Run the benchmark suite.
Install and import#
ImportError: the local driver needs the math extras#
ImportError: the local driver needs the math extras: pip install 'tauopt[local]'
(or 'tauopt[all]' to include the GP strategies via 'tauopt[botorch]')
Cause: Client(driver="local") with only the featherweight install. The base
package is httpx + pydantic by design; the in-process engine needs numpy/scipy.
Fix, from a clone: pip install "./python[local]" for the space-filling
strategies, or pip install "./python[all]" to include the Gaussian-process
(GP) strategies (the PyPI name holds a placeholder during the closed beta).
Every suggestion reports strategy sobol/random (or the quickstart output is off)#
Cause: the GP extras are missing, and the never-fail ladder degrades SILENTLY.
An unavailable rung (no tauopt[botorch] installed) is skipped without an
error; the suggestion’s provenance records fallback_from but nothing raises.
The same signature appears under sustained load in the default wall-clock
budget mode: GP fit cost grows with observations and dimensions, and asks past
the default timebox_s (10 seconds) degrade down the ladder. ucb_cl
models the objective only, and sobol/random model nothing –
outcome-constraint metrics are not enforced by fallback points. Fix: check
tauopt.engine.strategies.registry.available() and
suggestion.provenance.fallback_from; install ./python[all] from a clone; for load-driven
fallbacks raise params.timebox_s, or pin timebox_mode="caps"/"off".
To tell the causes apart, read the failed-rung records; through the Client
they are plain dicts (None when no rung ran and failed):
# docs: no-run
for a in sug.provenance.fallback_attempts or []:
print(a["rung"], a["exception_type"], a["exception_text"])
An unavailable rung (the extra not installed) is skipped without a record, so
fallback_from names it while the list stays None; a rung that ran and
failed leaves its TimeboxExceeded, its ValueError (for example GP strategies require at least one successful observation), or its numerics
error in exception_text.
Guide: Confirm the GP strategies run.
ValueError: hosted driver requires base_url and token#
Cause: Client() defaults to the hosted driver, which needs both connection
arguments. Fix: pass base_url= and token=, or ask for the in-process engine
explicitly with Client(driver="local").
ValueError: unknown driver 'x' (hosted|local)#
Cause: Client(driver=...) with a value other than "hosted" or "local".
The Client constructor raises it before any request. Fix: one of the two
names.
KeyError: unknown strategy 'x' (known: [...])#
Cause: a DIRECT tauopt.engine.strategies.registry.get() call with an
unregistered name – this error does NOT fire through the Client. A typo’d
strategy in config={"strategy": ...} never raises: the ask silently falls
back through the whole ladder and answers from a lower rung, with the typo’d
name recorded in suggestion.provenance.fallback_from. If your “gp_ei” run
behaves like space-filling, check that field for a misspelling. The known set
is random, sobol, gp_ei, ucb_cl, gp_nehvi (GP strategies also need
tauopt[botorch] installed to run).
unknown params key 'num_restrats' for strategy 'gp_ei'#
[400] validation_error: unknown params key 'num_restrats' for strategy
'gp_ei' (did you mean 'num_restarts'?); accepted: acq_maxiter,
categorical_mode, failure_mode, fit_maxiter, noise_mode, num_restarts,
raw_samples, timebox_mode, timebox_s
Cause: a params key the named strategy does not read – a typo, or a key
of another strategy. Unlike a misspelled strategy name, this refuses before
any rung runs, on either driver and on every path (an explicit config,
strategy: "auto", or a custom policy’s routed config): an unread key used to ride silently into the
canonical config and move config_hash, so two runs meant to be identical
replayed as different configs. sobol and random read no params at all
(sobol reads no params). Fix: use a key from the accepted set (the
per-strategy tables in the
strategies reference),
or move a field that exists only at the top level (seed, dedup, base_noise,
center_in_bootstrap, duplicate_intents, queue_selection) out of
params.
invalid params value for strategy 'gp_ei': num_restarts must be an integer, got 'eight'#
[400] validation_error: invalid params value for strategy 'gp_ei':
num_restarts must be an integer, got 'eight'
Cause: a params value the named strategy’s reader refuses – the wrong type
("eight", a bool where a count is expected, 1.9 for an integer key), a
value out of range (num_restarts outside [1, 64], timebox_s outside
(0, 300], mc_samples outside [16, 1024]), an unknown enumerated value
(noise_mode, failure_mode, categorical_mode, timebox_mode), or a
two-writer conflict (timebox_mode or categorical_mode set both top-level
and in params; timebox_mode="off" with an explicit timebox_s; a cap
field outside timebox_mode="caps"), or a composition of valid values the
strategy cannot run on this space (categorical_mode mixed or mixed_capped
on a conditional space; failure_mode="feasibility_gp_fixed" with either on a
space with categoricals). Under strategy: "auto" the named strategy is the
policy’s pick and the refused value is the top-level flag carried into it, and
the refusal lands on the first ask the policy routes to a GP rung (the
bootstrap’s sobol asks carry the flag inertly). Like an unread key, this refuses at the
ask boundary before any rung runs, on either driver (on the hosted service a
non-exempt user’s timebox-class refusal – timebox_s, timebox_mode, the
timebox_mode two-writer pair, timebox_mode="caps" with timebox_s, and the
params-form caps pointer; a cap field outside caps, the categorical pair and the
caps-mode bundle reach ask() and answer the engine text on both planes – is answered by the service’s own config pre-check
as config: <the same reader text>, without the strategy name): a refused value used to
raise inside the never-fail ladder, and the ask was served by the next rung
with provenance.fallback_from as the only tell, so a run that asked for
gp_ei with eight restarts silently got another strategy. The ladder now
descends on runtime failures only. Fix: a typed, range or enumerated refusal
names the key, the value and the expected domain; a two-writer or posture
conflict names the keys and the one-writer rule; the per-strategy tables in the
strategies reference
carry every domain.
Creating an experiment (400 validation_error)#
Guide: Add constraints and Optimize multiple metrics.
provide exactly one of metric | metrics#
Cause: both or neither of metric= / metrics= given. Single-objective uses
metric={"name": ..., "objective": ...}; anything with thresholds, storage, or
multiple objectives uses metrics=[...].
metric="loss" is refused: the bare string names no objective#
[400] validation_error: metric="loss" is refused: the bare string names no
objective, so pass metric={"name": "loss", "objective": "minimize"} or
{"name": "loss", "objective": "maximize"}
Cause: metric= given as a bare string. The shorthand used to expand to
{"name": "loss"} and maximize the loss with no signal, so both drivers now
refuse it before anything is created (the Client refuses it before any
request leaves; the local driver’s own door refuses it too). Fix: pass the
dict with an explicit objective. A dict without objective still
maximizes, as before.
metric strategy/threshold require metrics mode: use metrics=[...]#
Cause: metric={..., "strategy": "constraint"} or a threshold in
single-metric mode. Those semantics only exist in metrics mode; pass the same
dict inside metrics=[...] instead.
constraint region too small: 0/4096 sobol probe points feasible#
Cause: the linear constraints exclude (essentially) the whole box, detected by a feasibility probe at creation so the failure is immediate rather than at the first ask. Fix: widen bounds or relax constraint thresholds. Note linear constraints range over plain (non-log, non-grid, unconditional) double parameters only.
num_solutions is single-objective only#
Cause: num_solutions > 1 with metrics mode. Multimetric best is the Pareto
frontier (already a set); num_solutions selects k diverse top solutions for a
single objective.
parameter 'batch': log_scale integers are not supported#
Arrives wrapped in pydantic’s report ([400] validation_error: 1 validation error for Parameter ...). Cause: log_scale=True on an integer parameter;
log scale applies to doubles only. Fix: spell the parameter as an integer
grid of the values you want
({"name": "batch", "type": "integer", "grid": [16, 32, 64, 128, 256]}), or
as a log-scale double that the objective rounds (int(round(a["batch"]))).
Basic bounds#
name must be 1-255 characters, observation_budget must be >= 1,
parallel_bandwidth must be >= 1, num_solutions must be >= 1 mean exactly
what they say. Space and metric STRUCTURE are immutable after creation:
update_experiment can change only name, budget, bandwidth, metadata, and
metric_thresholds.
The ask loop (suggest)#
Guide: Run parallel workers and Pin a strategy and seed.
429 bandwidth_full#
[429] bandwidth_full: N open suggestions >= parallel_bandwidth=N; observe or
close one first
Cause: every bandwidth slot holds an open suggestion. This is the EXPECTED
backpressure signal under parallelism (not a fault). Fix: observe or close an
open suggestion, or create the experiment with a larger parallel_bandwidth.
The hosted transport auto-retries 429 with backoff (honoring Retry-After,
capped at 8 seconds per attempt); workers that want to wait longer implement
their own retry loop (see
python/examples/parallel_workers.py).
Hosted beta quotas (also 429)#
The hosted beta additionally enforces a suggestion quota (default 1,000 per
rolling 24 hours; these 429s, quota_exceeded, carry Retry-After – after the
transport’s own bounded retries give up, the raised error exposes the final
response’s headers as e.headers and the parsed numeric delay as
e.retry_after, so your own backoff loop can honor it) and an
active-experiment cap (default 25; these 429s, experiment_quota_exceeded,
carry NO Retry-After – waiting does not help). The 429 error types a client
loop meets most are five (bandwidth_full, quota_exceeded, experiment_quota_exceeded,
quota_unverifiable, rate_limited); their meanings are in the REST reference’s
error-type table (quota_unverifiable: the quota
store could not be read, so the request was refused rather than admitted unchecked; wait the
30 s Retry-After).
The quotas never throttle reads or observations; the beta lane’s
pre-authentication rate limit (150 requests per second, burst 300, every
caller together) applies to every request on that lane and answers
rate_limited with Retry-After 1; its per-credential sibling answers
credential_rate_limited with Retry-After 1 past eight requests in flight
from one token, 40 a second (burst 80), or 15 history-sized reads a second
(burst 30). Fix
for the experiment cap: archive finished experiments with
client.archive_experiment(exp.id) (hosted: DELETE /v1/experiments/{id}).
An archived experiment frees its cap slot but stays readable – experiment()
(state becomes "deleted"), best(), and suggestion dismissals keep working;
suggest/observe/update_experiment on it refuse with a 404.
Guide: Archive experiments.
Admission and rate limits (hosted)#
At its 74e0202 build (the prod since 2026-10-31) the hosted service answers
twelve more typed refusals a package-free loop or a worker fleet
can meet, none of which the local driver has. Most are one line of
When the service says no
with its remedy, the rest are only here; the exact bodies are the service’s.
429 with Retry-After: 1: too_many_inflight_asks (a retry
under the same Idempotency-Key already waits on one of your in-flight asks;
the waiter cap is one), too_many_queued (eight of your asks already wait in
your queue behind your in-flight bound – 8 unless your organization carries an
override, which GET /v1/me reports), too_many_parked (your parked suggest permits reached your
ceiling on the instance), suggest_rate_limited (asks faster than about ten a
second), credential_rate_limited (more than eight requests in flight from one
token, more than 40 a second, or more than 15 history-sized reads a second),
rate_limited (the lane’s bucket, every caller together); and, with no
Retry-After, token_quota_exceeded (a mint past the plan’s five live tokens;
a web-app route: revoke one first). 503 with Retry-After: 1: queue_full
(every suggest permit of the instance held),
connection_pool_busy (every pooled database connection in use). 409:
observation_cap_reached (the experiment holds its 4,000 observations),
token_expired (a rotation of an expired token; revoke it and mint anew, a
web-app route), idempotency_key_mismatch (an Idempotency-Key reused
against another experiment: a new ask on another experiment needs a fresh key,
a retry of the original ask keeps its experiment and its key). A package-free script sees these as the helper’s
ApiError with the service’s error.type; the Python client raises its
ApiError with e.error_type set to the same name.
3xx unexpected_redirect#
[301] unexpected_redirect: unexpected redirect (301) to 'https://...'; the
transport does not follow redirects - check base_url for a scheme/host that
redirects (e.g. http vs https)
Cause: the configured base_url answers with a redirect – typically an
http:// URL bouncing to https://, a proxy in front of the service, or a
moved host. The TauOpt API itself never redirects, and the transport refuses
to follow one silently: a followed 301/302/303 would convert a POST into a
GET, and cross-origin redirects drop the auth header.
Fix: point base_url at the final URL (usually: use https:// directly).
api_error (the reply was not the service’s JSON)#
ApiError with error_type api_error: the response body was not the service’s JSON
error shape (a proxy’s HTML page, a plain-text 500), so the client reports the status with
the first 500 characters of the body (or the reason phrase) as the message. Check the base
URL (https://api.tauopt.com on the beta) and the service’s health before retrying; the
message is the upstream text, not an engine error.
409 space_exhausted (raises tauopt.SpaceExhausted)#
[409] space_exhausted: the search space is fully explored -- every
input-feasible point has been suggested or observed (proof rides
details.dedup). Continue via an authorized replication intent, or read the
best point: the optimization is complete
Cause: COMPLETION (not failure). Every input-feasible point has been suggested
or observed, and the engine PROVED it – the receipt is in details.dedup
(exhaust_proof, proof_method, universe_size). Two proof methods exist:
finite spaces up to 1024 raw points get an enumeration proof (conditional
parameters included); larger all-finite spaces that are unconditional and
unconstrained get a cardinality proof. Fix: nothing is broken. Catch
tauopt.OptimizationComplete in the ask loop and read the best point; on
noisy objectives, continue through an authorized replication intent (see
“Intents” below).
503 suggest_timeout#
Cause: on_timeout="fail" and either a GP rung’s timebox_s or the ask’s
max_wait_s expired (budget or off mode; in caps mode the deadline is
AskDeadlineExceeded, see below) – an honest timeout so you can run your
own backoff. Hosted responses carry Retry-After: the hosted client raises the
first 503 (a POST is not retried on this 503), and the raised ApiError exposes
that response’s headers as e.headers and the parsed delay as
e.retry_after, so your own backoff loop can honor it. max_wait_s accepts a number in (0, 300]. The default
on_timeout="ladder" instead degrades down the fallback ladder
(gp_ei -> ucb_cl -> sobol -> random; multimetric: gp_nehvi -> sobol -> random)
and records the fallback in the suggestion’s provenance rather than raising.
503 ask_deadline_exceeded (raises tauopt.AskDeadlineExceeded)#
[503] ask_deadline_exceeded: ask exceeded max_wait_s=50.0 before gp_ei could
run (caps mode: rung-boundary refusal, no substitution)
Cause: a config carrying timebox_mode: "caps" and an expired max_wait_s.
In caps mode the deadline is a refusal, not a ladder step: it fires at a rung
boundary (before the rung runs) or at the single delivery decision (after the
work, the computed drafts discarded), in either on_timeout mode, and no
cheaper rung substitutes. No suggestion opens and no bandwidth slot is taken.
The raised AskDeadlineExceeded is an ApiError on both drivers; the refusal receipt rides
e.point (rung_boundary or delivery), e.max_wait_s, e.elapsed_s,
e.rung, and e.details with the same four keys (plus, on the hosted plane,
any further key the service’s body carries beside them). On the hosted plane an
ask that first waited for an ask slot reaches the engine with that wait already
taken off its budget: the
receipt’s max_wait_s is the budget the engine was handed after the wait (the
figure you sent, less the queue wait) and elapsed_s counts from the engine’s
start. There is deliberately no
Retry-After (e.retry_after is None): a caps re-ask repeats the same
deterministic work, so waiting fixes nothing. Fix: raise max_wait_s, lower
the work caps (fit_maxiter, acq_maxiter), or pin a cheaper strategy.
Guide: Pin a strategy and seed.
A third e.point, backstop, is the supervisor’s refusal
(tauopt.BackstopExceeded, a subclass): the ask did not answer within
max(1.25 * max_wait_s, max_wait_s + 3 s) – a kernel that never returns, not
the engine’s own clocks – so the local driver’s worker process was killed and
replaced behind the response (the hosted plane does the same on its queue
lane’s worker pool, provenance.lane worker; POSIX only – on Windows the local driver runs
the ask inline with no backstop). e.stage says where: ask (the worker was
running the ask), warmup (the worker’s imports and handshake had not finished;
the replacement warms behind the refusal, and the next ask waits for it inside
its own budget – it can refuse at warmup again if the imports take longer
than that backstop), worker_died (the worker crashed or was killed,
e.details["exit_code"], or died three times before its handshake or could not
be started, e.details["spawn_failures"] and e.details["spawn_error"]; its
stderr is on your terminal; close() then ask again retries), reap (three
retired workers refuse to die, e.details["unreaped_pids"] and
e.details["reaping"]: restart the process), abandoned (eight pipe threads
abandoned, e.details["abandoned"], or a backstop thread that could not be
started: restart the process). No suggestion opened
and the state is unchanged. Fix: raise max_wait_s if the ask is slow rather
than hung, pin a cheaper strategy, or Client(driver="local", supervisor="off") to run the ask inline as before.
503 experiment_busy (hosted)#
[503] experiment_busy: another request holds this experiment; retry
Cause: another request holds the experiment’s advisory lock – a running ask on
a worker, usually (every ask runs under it, one at a time; so do an update, an
archive and the two dismissal routes – the four requests that can be refused
this way; by contrast, an ask that finds the lock held waits for it on its
worker and is re-admitted, never refused, and an observe never answers it) –
and the 3 s bounded wait expired.
Hosted only (the local driver is one process with no lock).
The answer carries Retry-After: 1 (e.retry_after). The hosted transport
retries it for you on every method (it is in the transport’s pre-commit set, so a
POST retries it too: experiment_busy, queue_full, and ask_slots_busy,
retired at deploy 2 with the lane it guarded but kept in the set) – up to five
attempts honoring the
Retry-After, so the ApiError surfaces only when those run out. Fix: wait and
re-send the same call – the refused update, archive or dismissal did nothing, so
there is nothing to reconcile.
500 all_rungs_failed, integrity_violation, internal_error#
[500] all_rungs_failed: no strategy could produce a suggestion (intended
'gp_ei') -- attempts: gp_ei[fit] ValueError: ...; ucb_cl[fit] ValueError: ...;
sobol[-] ValueError: ...; random[-] ValueError: linear constraints rejected
100000 consecutive random draws -- feasible region too small to sample
Cause: the engine’s own failure, never the request’s. all_rungs_failed
means every rung of the fallback ladder raised, the stdlib-only random rung
included (its one refusal is the rejection cap on a pathologically thin
linear-constrained region); e.details["attempts"] carries one record per
failed rung (rung, stage, exception_type, exception_text). Fix: for
the constrained-region case, widen the bounds or relax the constraint and
re-create the experiment; otherwise read the attempts. integrity_violation
means a postcondition failed after a rung answered (e.details["check"]
names it) and internal_error means an engine-side ValueError that is not
the caller’s (e.details["exception_type"]); both are defects to report with
the message. Both drivers answer these shapes; the hosted transport does not
retry a POST 500 (a POST retries only a 429 and the transport’s pre-commit
set), so the error arrives at once. In
earlier engine versions the AskFailed cases escaped the local driver as a raw
RuntimeError and answered a bare 500 api_error on the hosted plane, an
engine-internal ValueError was blamed on the caller as a 400, and with
dedup on (the default) the all-rungs-failed case read as a 409
unique_candidate_unavailable.
ValueError: on_timeout must be 'ladder' or 'fail', got 'x'#
Cause: client.suggest(..., on_timeout=...) with a value other than
"ladder" or "fail". The Client raises it before any request is sent. Fix:
one of the two names ("ladder" is the default).
“suggest keeps returning the same point”#
Applies to direct tauopt.ask() users and
config={"dedup": False} asks (strategy defaults to "auto"):
suggest is a pure function of (state, config, the ask’s intents), so until
the state changes –
an observation lands, or an open suggestion is closed – the same ask returns
the same answer by design. Fix: report or close before re-asking. (Through the
Client with the dedup default, this symptom does not occur: every suggest
opens a suggestion, which advances state, and pending points are excluded –
the visible signals there are 429/409 instead – with one edge: a suggestion
you close WITHOUT observing leaves the dedup universe (the set of observed and
open points the engine never re-suggests), so the same point can lawfully
return on a later ask.)
Reporting results (observe)#
Guide: Report failures and noise.
provide exactly one of suggestion_id | assignments#
Cause: both or neither given. Observe a suggestion by its id, or report a free-form point by its assignments.
provide value(s) XOR failed=true#
Cause: a result and a failure flag together (or neither). A failed evaluation
carries failed=True and no value; a successful one carries value= (single
metric) or values={...} (metrics mode, every optimize/constraint metric
required).
observation value must be finite, got nan#
Arrives wrapped in pydantic’s report ([400] validation_error: 1 validation error for ObservationRecord ...); in metrics mode the line names the metric
(observation values['loss'] must be finite, got nan). Cause: a NaN or inf
result, the usual signature of a diverged run. Values are checked at the door,
never discovered inside a fit. Fix: report the evaluation as failed instead
(failed=True, no value), which keeps the point in the dedup universe and out
of the model.
409 conflict: suggestion '...' already has an observation#
Cause: double-reporting the same suggestion (the local driver mirrors the
hosted unique index). Fix: one observation per suggestion; use a fresh ask, or
report a free-form point via assignments=.
409 suggestion_expired (hosted)#
[409] suggestion_expired: suggestion 11 expired at 2026-10-23T00:00:00+00:00
after 86401 s open (the experiment's suggestion_ttl_s; a later suggest expired
it and reissued its slot). Report the result as an unsolicited observation
with details.assignments, or raise suggestion_ttl_s for longer evaluations
Cause: the suggestion stayed open longer than the experiment’s
suggestion_ttl_s (the service default is 86400 s), a later ask on the
experiment closed it (its expired_at is set; suggestions(exp.id) shows it
closed) and reissued its bandwidth slot, and then the observation arrived.
Hosted only: the local driver records suggestion_ttl_s and never expires a
suggestion (expired_at is always None there). e.details carries
suggestion_id, expired_at, created_at, and the assignments. Fix:
report the result as a free-form point, observe(exp.id, assignments=e.details["assignments"], value=...), which keeps the evidence;
for long evaluations create the experiment with a larger suggestion_ttl_s
or move it with update_experiment(exp.id, suggestion_ttl_s=...) (0 never
expires; the new value applies from the next ask).
Guide: Run against the hosted service.
404 not_found#
no experiment '...' / no suggestion '...' (the local driver’s wording; the
service’s 404 messages differ): the id does not exist in this driver
instance. The local driver’s state lives in the process – ids do not
survive a restart (recovery recipe below).
After a restart (local driver)#
Local state is in-memory only. To survive a crash or a wall-time kill, log
the FULL payload of every observe you send – assignments plus value/values,
any stddevs, and failed=True rows (failed points are part of the dedup
universe too) – then rebuild: re-create the experiment with the same
definition and replay the log through observe(assignments=..., ...) in
order (no suggestion ids needed). The rebuilt experiment carries the same
evidence and the next suggest is deterministic on it; open suggestions and
dismissals from the dead process are not part of such a log, so the
continuation is equivalent-in-evidence, not byte-identical to a run that
never died.
Guide: Replay a suggestion for the
rebuild, and Archive experiments for
what stays readable while the process lives.
Reproducibility#
Guide: Replay a suggestion and Pin a strategy and seed.
Same machine + same dependency versions => bit-identical runs: suggest is
deterministic per (state, config), seeds derive from state hashes, and every
suggestion’s provenance records (engine_version, engine_sha, deps_fingerprint, strategy, config_hash, seed). The fine print:
Across CPU classes or torch/BoTorch builds, expect numeric drift in GP-heavy paths – pin the machine class when exact replay matters.
The default wall-clock budget mode is pure GIVEN a non-binding timebox: on a loaded machine a binding timebox can trigger a recorded ladder fallback. Pin
"timebox_mode": "caps"(or"off"with no ask-levelmax_wait_s) for load-independent replay.GP fit/acquisition compute is single-threaded BY DESIGN: the engine pins torch to one thread for the duration of the suggest step and restores it after (a fixed BLAS schedule for bit-determinism), so thread count does not enter the engine’s bit-identity envelope – and a suggest never oversubscribes a shared node. Your own evaluation code still uses whatever threads you allow it.
To replay a suggestion exactly: rebuild the experiment state up to that ask (your recorded observations, in order), verify the environment against the receipt’s
deps_fingerprint, and re-ask with the same config. The receipt alone identifies a run; replay needs your recorded state.For policy-free research runs pin everything explicitly:
config={"strategy": "gp_ei", "seed": 7, "timebox_mode": "caps"}.
Engine-level exceptions (direct tauopt.ask users)#
Driving tauopt.ask(state, config=...) directly (no Client) surfaces typed
engine errors instead of ApiError: TimeboxExceeded (wall budget exceeded
with on_timeout="fail") and ValueErrors
from config/space validation plus STATE-READ preconditions (single-objective
reads refuse multimetric states: use pareto_frontier(), not best).
The one left out of that list is AskDeadlineExceeded (a caps-mode request
deadline made the answer unavailable – refused, never substituted): an engine
error that already IS the 503 ask_deadline_exceeded ApiError both drivers
raise.
UniqueRepairExhausted and QueueUnsupported never reach you: the ask
boundary catches both. A dedup repair that runs dry keeps the slots it
filled and descends the ladder for the rest, leaving a fallback_attempts
entry and the hop in provenance.dedup.ladder_hops (per_draft_origin
names the rung that produced each returned draft); a strategy that cannot
serve a queue_selection block answers through its plain repair path, and
each such slot appears in provenance.selection with origin
pass_through. Strategy-level preconditions do NOT surface through ask()
either: a GP rung with zero successful observations raises inside the rung,
the ladder catches it and descends (recorded in fallback_attempts). The
same catch takes the final rung’s one refusal: on a
pathologically thin constrained region, random’s capped rejection scan raises
linear constraints rejected 100000 consecutive random draws, and the
creation-time feasibility probe screens out most such regions. What climbs
out when every rung has failed is one tauopt.engine.AskFailed (a
RuntimeError) with error_type all_rungs_failed:
AskFailed: no strategy could produce a suggestion (intended 'gp_ei') -- attempts: ...
followed by an attempts summary (one rung[stage] ExceptionType: text entry
per failed rung, the same evidence fallback_attempts carries); its
details carries intended and the attempts records, and its __cause__
is the last rung’s exception. Through the local driver it is the 500
all_rungs_failed ApiError described under “The ask loop” (and the same
through the hosted plane). It means every
rung failed; read the attempts, and for the constrained-region case widen the
bounds or relax the constraint and re-create the experiment. The same class
carries error_type integrity_violation when a postcondition fails after a
rung answered (a center-flagged sobol answer without a center record, or
queue-selection drafts that disagree with the selector’s ledger); details
names the check. The caller’s own argument errors (n < 1, a bad
max_wait_s, auto with params, intents without dedup, a params key the
strategy does not read) are tauopt.AskValidationError (a ValueError), the
one class the drivers answer as 400. AskDeadlineExceeded reaches
Client(driver="local").suggest callers as the 503 ask_deadline_exceeded
ApiError it is, exactly as the hosted driver raises it – one
except ApiError sees it, or catch the class itself alongside
OptimizationComplete in caps-mode loops.
Hosted transport#
Guide: Run against the hosted service.
Retries are bounded and honor Retry-After (capped at 8 seconds per attempt):
429 always retries (the server rejected the request; nothing committed); 500,
502, 503, and 504 retry on non-POST methods; a POST also retries a 503 whose
typed body names one of the transport’s pre-commit set – experiment_busy,
queue_full, and ask_slots_busy, retired at deploy 2 with the lane it guarded
but kept in the set – each a refusal before any suggestion or observation is
committed, with Retry-After – and no other 5xx (connection_pool_busy included: the pooled
connection is released at commit, so a post-commit read can answer it for a row
already written). A POST that dies with any other 5xx, or with no answer at all,
may have committed server-side – check state before re-sending rather than
blind retrying:
client.suggestions(exp.id, state="open") and client.observations(exp.id)
list what the server holds (both work on archived experiments), and
client.experiments() lists your active experiments (newest first, count
is the active total) after a lost create_experiment. client.health()
and client.auth_check() probe the service and the token first.
Cannot connect / auth failed#
Connection-level failures raise RAW httpx exceptions and are NOT retried:
httpx.ConnectError means the base_url or network is wrong, or the TLS
handshake failed (CERTIFICATE_VERIFY_FAILED in its text, with the reason
after it: unable to get local issuer certificate means the host’s chain is
not in your Python’s trust store, which a python.org macOS build fixes by
running its Install Certificates.command once; an expired certificate or a
hostname mismatch is the service’s to fix, so report it; the package-free
scripts surface the same failure as urllib.error.URLError);
httpx.ReadTimeout means the server did not answer within the read timeout:
timeout_s (default 60 seconds) on every route but the suggest POST, which
listens for the backstop of its max_wait_s plus 40 s whenever that exceeds
timeout_s (415 s with no deadline; at the default timeout_s a hung ask with
no deadline, or one above 240 s, then ends in the platform’s 504, an ApiError, on a service without the
queue and worker supervisor; a smaller deadline still ends here, at the derived
bound), so a long ask never needs a raised timeout_s; siblings
(WriteError, PoolTimeout, protocol errors) escape the same way. The
transport does not follow redirects: an http:// base_url against an
https-only service raises [301]/[302] unexpected_redirect (see “3xx
unexpected_redirect” above) – use the exact scheme and host you were given.
A [401] unauthorized ApiError means the token was rejected: it is unknown,
revoked, or issued for another lane (beta tokens are lane-scoped and are
shown once, when minted on the web app’s tokens page). The service answers
every token failure with 401; its 403s are forbidden (the admin API’s door only, never a /v1 route),
origin_refused (the pre-authentication gate’s Origin check) and
account_pending / account_disabled (the web app’s sign-in).