TauOpt
TauOpt documentation#
TauOpt picks the next configuration to try when each evaluation is expensive: hyperparameter tuning, simulator calibration, experimental design. You ask for a point (a suggestion), evaluate it yourself, and report the result (an observation); the engine fits a model (a Gaussian process, GP) to everything reported so far and picks the next point. That is Bayesian optimization (BO); TauOpt is a Python 3.10+ package for it from Talaria Scientific, Apache-2.0. One Client runs the engine in-process (Client(driver="local")) or talks to a hosted service (Client(base_url=..., token=...)) with the same create_experiment, suggest, observe, and best calls.
The loop. (1) Your code asks for a point, (2) evaluates it, and (3) reports the result; (4) the engine fits a model to every result so far and picks the next point, keeping every suggestion and observation in the experiment record. Step 2 is the one step TauOpt never does.#
Closed beta: the hosted service is invite-only and the package is not distributed until the public release (the repository is private; pip install tauopt installs a placeholder), so request access first if you have no invite. Beta participants need no install: you sign in at app.tauopt.com, mint a tau_ token there, and drive the service at api.tauopt.com over plain HTTP from Python’s standard library or from an agent through the MCP server at mcp.tauopt.com – start with Hosted beta: getting started, which walks the whole path once, then the hosted examples.
These pages describe the repository’s main branch: version 0.4.0 plus the changes listed under Unreleased in the changelog; terms the pages use without defining them are in the glossary at the end of this page. No page makes a benchmark or convergence claim; comparisons against other optimizers are the job of tauopt-bench, the benchmark suite in the repository. Questions and bug reports: a GitHub issue on the repository.
Start here#
The closed beta’s whole path once: sign in at app.tauopt.com, mint a token, run the first loop against api.tauopt.com with curl (every answer pasted), see it in the web app, connect an agent through mcp.tauopt.com, and read what the service answers when it refuses a request.
Run a 30-observation tuning loop and read the result: in the closed beta over plain HTTP against the hosted service, with a token minted on the web app and nothing installed; with repository access, in-process from a clone, then the same loop against the service with one constructor change.
The mental model in one page: the experiment, the ask/observe loop, what best returns, and completion as a terminal state rather than a failure.
Already ran it? Install TauOpt covers uv, conda, and air-gapped hosts; Run against the hosted service takes your beta token; Use the MCP server drives experiments from Claude Code.
How-to guides#
One task per page; most assume a working [all] install (the quickstart or the install guide). During the closed beta, start with the hosted service and MCP pages; the other pages show the client form, and each flow runs hosted through its *_rest.py twin in the hosted examples, except replay, which needs the engine.
Nothing to install during the closed beta (the hosted service and the MCP server need only your token); with repository access, the clone install with pip, uv, or conda, the extras tiers, avoiding the CUDA torch download, air-gapped wheelhouses, and how to confirm the Gaussian-process (GP) strategies run.
Drive the service with your token and nothing installed (the REST calls one by one), or point the client at it: set its two timeouts, handle the 429 replies (wait out a rate limit or a busy slot; archive an experiment when the active-experiment cap answers without a Retry-After), and recover after a lost response.
Drive experiments from Claude Code or any client of the Model Context Protocol (MCP) through the hosted server’s twelve tools.
Declare metrics=[...], set thresholds, and read the Pareto frontier from best.
Linear constraints over the input space and outcome constraints on metrics.
Share one experiment across workers with parallel_bandwidth; a 429 bandwidth_full reply means every slot is taken, so wait and retry (backpressure).
Parameters that exist only when another parameter takes a given value.
failed=True for evaluations that crashed, value_stddev for measurement noise, and assignments= to report a point you evaluated without a suggestion.
Ask best for several distinct good configurations (num_solutions) instead of a single winner.
Choose the strategy and seed yourself (config=) so a run is reproducible and the policy stays out of it; a load-independent timebox mode and max_wait_s bound the wait.
Rebuild the experiment state, check the provenance receipt, re-derive the same point.
Free one of the hosted active-experiment slots the quota allows (the wire’s DELETE; the state then reads deleted); the experiment stays readable.
Re-evaluate a point the engine would otherwise refuse as a duplicate: variance_replication repeats an observed point to measure its noise; liar_verification re-asks a still-open point whose placeholder value (the constant liar) may be hiding a worker that will never report.
Install tauopt-bench from the clone (needs Python 3.12 and uv: its own uv sync, not the [all] install), list its problems, methods, and frozen presets, run one cell and read its receipt, bound the cores a run uses, and know what the fleet directories are for.
Reference#
Signatures, fields, status codes, error strings.
Client and its methods, the wire models, the exception classes, and the engine entry points.
One page per module (tauopt.client, tauopt.models, tauopt.local, the engine) and one for the exceptions module (tauopt._http), generated from the docstrings; HTML only, the Python API page lists the same surface as markdown. Start on the Python API page for any public name; these pages add the per-module view.
Every /v1 route the hosted service serves, with request and response shapes, the error types, and authentication.
Every subcommand of the benchmark suite’s command line, with its usage, flags, defaults, and help text.
The twelve tools, their arguments and annotations, and the error payload.
random, sobol, gp_ei, ucb_cl, gp_nehvi, and every OptimizerConfig field with its default.
Every error_type and exception class with its status code.
The errors a run meets, by symptom: the client’s own errors and the engine errors the guides document, each with its cause and fix.
Release history by version.
Explanation#
Why the system is shaped the way it is.
The experiment as the unit of work, the ask/observe loop, budgets as hints, and completion as a terminal state (raised as SpaceExhausted, an OptimizationComplete) rather than a failure.
Double, integer, and categorical parameters; bounds, log scale, grids, and conditions.
A single metric versus a metrics list (metrics mode): the optimize, store, and constraint roles, thresholds, and what best returns in each mode.
Open and closed suggestions, how many may be open at once (parallel_bandwidth), dismissals, and what an observation carries.
What each strategy does, and how an ask falls to the next strategy on a timeout or numerical failure instead of raising an error.
How the default policy routes each ask from an initial space-filling phase into the GP strategies.
The receipt on every suggestion and what “same machine, same dependencies, same answer” does and does not promise.
Why the engine never re-suggests an observed or open point, proven exhaustion, authorized duplicates for the slots that remain, and dedup: false as the escape hatch.
How the cost of an ask grows with parameters and observations inside the default time budget (the timebox), how many categories and how much noise the models tolerate, how integers are handled, and what the engine does not do.
What tauopt-bench measures and why it exists, problems and rosters, seeds and replicates, what a receipt proves, the campaign model and its claims gate, and why no result appears on this site.
Examples#
Runnable scripts in two variants, every flow in both: the hosted, package-free scripts the closed beta runs (the quickstart loop, Branin, parallel workers, exhaustion, a mixed space, multiple metrics, an outcome constraint, an experimental design, the model-tuning use cases, an ODE calibration, and a convergence plot over plain HTTP) and the local scripts in-process (the same flows plus replay, which needs the engine); each takes --help.
The same ask/observe loop on the Branin test function, run in-process and then against the hosted lane over plain HTTP behind TAUOPT_TOKEN.
Glossary#
One line per term; the linked page explains it.
experiment: the unit of work: a search space, one metric or a metrics list, an observation-budget hint, and a bandwidth, plus the record of every suggestion and observation – Experiments and the ask loop.
driver: where the engine runs:
Client(driver="local")in your own process,Client(base_url=..., token=...)on a hosted service, with the same calls either way – Run against the hosted service.ask: one request for a suggestion (
client.suggest); the policy routes each ask on its own – Experiments and the ask loop.suggestion: the point an ask returns, as assignments for every active parameter plus a provenance receipt; it stays open until observed or closed – Suggestions and observations.
bandwidth slot: one open suggestion;
parallel_bandwidthsets how many may be open at once (default 1), and at the capsuggestanswers 429bandwidth_fulluntil an observation or dismissal frees one – Suggestions and observations.observation: the reported result for a point: a value (or
valuesin metrics mode), an optional noise estimate, orfailed=True– Suggestions and observations.best: the best feasible, non-failed observed point so far (never a model prediction), or the Pareto frontier when two or more metrics are optimized; empty before the first successful observation – Metrics and objectives.
strategy: one way of picking the next point:
random,sobol,gp_ei(GP, expected improvement),ucb_cl(GP, upper confidence bound with constant-liar fill),gp_nehvi(GP, noisy expected hypervolume improvement, for two or more metrics); the GP strategies fit a Gaussian process, a statistical model of the results so far (unrelated to thestrategyfield on a metric: optimize, store, or constraint) – Strategies and the never-fail ladder.acquisition: the model’s score for a candidate point; a GP strategy picks the candidate with the highest score – Strategies and the never-fail ladder.
policy: the rule that picks a strategy for each ask from the experiment’s history; an explicit
config=bypasses it – Policies.ladder, rung: the ordered list of fallback strategies an ask descends when one fails (each strategy on it is a rung); the bottom rung is standard-library random – Strategies and the never-fail ladder.
timebox: in the default budget mode, the wall-clock budget each GP strategy gets per ask (
timebox_s, 10 s by default, 300 at most); overrunning it descends the ladder – Limits and when not to use TauOpt.provenance: the receipt on every suggestion: the engine identity triple (version, sha, dependency fingerprint), the strategy that answered, the config hash, the seed, and any fallback – Provenance and determinism.
dedup, universe: the engine never re-suggests a point in its universe (every observed point, failed ones included, and every open suggestion); exhausting a finite universe is completion, raised as
SpaceExhausted(anOptimizationComplete) – Dedup and completion.intent: a request, by kind, to re-evaluate a point that dedup would otherwise refuse – Use duplicate intents.
lane: one deployment of the hosted service (the invite-only beta is one); a token works only on the lane that issued it – Run against the hosted service.
token: the
tau_credential you mint on the web app’s tokens page after signing in; shown once, lane-scoped, sent asAuthorization: Bearerto the REST API and the MCP server alike – Hosted beta: getting started.hosted plane: the hosted service and its MCP server, as opposed to the local driver in your process; a behavior called hosted-only is the service’s and the local driver does not have it.
hypervolume: the volume of metric space between a frontier and the reference point; a larger one is a better frontier, and
gp_nehvipicks the point expected to grow it most.429: the service’s wait-and-retry reply: every bandwidth slot taken, a quota reached, or the rate limit hit;
Retry-Aftersays how long when there is one; the active-experiment quota (experiment_quota_exceeded) sends none (archive an experiment instead) – Run against the hosted service.MCP: the Model Context Protocol, through which the hosted service exposes the experiment surface as tools for Claude Code and other MCP clients – Use the MCP server.
cell: one problem, one method, one replicate seed; the unit a benchmark receipt, budget, and grid are counted in – Run the benchmark suite.
protocol, preset: the data that drives a benchmark run (problem ids, method ids, a seed policy, a budget rule, an optional batch geometry), identified by its hash; a preset is a protocol frozen as a file, additive and never revised – The benchmark suite.
campaign: one frozen preset run to completion, its receipts merged into one file and rendered as one report – The benchmark suite.
claims gate: the checks a publishable benchmark report must pass (
tauopt-bench report --claims): a frozen preset, a complete and clean grid, planned budgets, one substrate, one fingerprint and one adapter version per method, and on ans8-floorpreset a passing self-audit table – Run the benchmark suite.receipt (benchmark): one JSON row per cell: identity, seed chain, environment fingerprint, trace, running best, measurements; on native rows the trace carries the per-suggestion provenance receipts – Run the benchmark suite.
pin: a test that fails the moment an invariant stops holding (the suite’s determinism, worker-invariance, and shard-merge pins) – The benchmark suite.
space-filling: a strategy that spreads points over the box without a model (
random;sobol, a low-discrepancy sequence that covers the box more evenly than random draws); the policy’s first asks are space-filling – Strategies and the never-fail ladder.bootstrap: the first few asks of an experiment, answered by a space-filling strategy before a GP has enough observations to fit (four by default) – Policies.
Pareto frontier: with two or more optimized metrics, the observed points no other observed point beats on every optimized metric; what
bestreturns in that case – Metrics and objectives.constant liar:
ucb_cl’s rule for open suggestions: each is treated as already observed at a pessimistic value so parallel asks spread out instead of repeating – Strategies and the never-fail ladder.reference point: the worst-acceptable corner of metric space that
gp_nehvimeasures hypervolume from; a metric threshold sets it – Metrics and objectives.Branin: a standard two-dimensional test function with three global minima (0.397887), used by the quickstart notebook and several examples because it is cheap and its answer is known – Examples.
battery: the module that registers one family of benchmark problems (
evalset,bbob,botorch,bayesmark,lasso,mixed,cond,moo) – Run the benchmark suite.