TauOpt
tauopt-bench CLI reference#
Every line below comes from the checkout’s argparse tree (tauopt_bench.cli.build_parser(), imported from the repository’s bench/python), rendered by tools/gen_cli_reference.py; bench/python/tests/test_cli_reference_is_current.py fails when this page and the suite disagree. The page states no result: it lists the subcommands, their flags, defaults, and help text. Run the benchmark suite installs the suite and walks the commands. python -m tauopt_bench.cli is the same entry point as the tauopt-bench console script.
tauopt-bench#
With no arguments the command prints its banner and usage:
tauopt-bench -- benchmark suite for Bayesian-optimization methods.
This release claims the package name; the real package ships from
https://github.com/talariasci/tauopt -- see https://talariasci.com
usage: tauopt-bench [-h]
{list,plan,run,fanout,fanout-bake,fanout-bake-candidate,skyseam-emit,merge,canonical,report}
...
tauopt-bench list#
Registered problems, methods, and frozen presets.
usage: tauopt-bench list [-h]
options:
-h, --help show this help message and exit
tauopt-bench plan#
Split a protocol into shard specs.
usage: tauopt-bench plan [-h] --protocol PROTOCOL --shards SHARDS --out OUT
[--isolate-adapter NAME]
options:
-h, --help show this help message and exit
--protocol PROTOCOL preset name or protocol JSON path
--shards SHARDS
--out OUT
--isolate-adapter NAME
pull this adapter's cells into one dedicated trailing
shard (repeatable) -- the shard-by-environment seam
for isolated-venv adapters (hebo: see bench/hebo-
venv.sh)
tauopt-bench run#
Run a full protocol or one shard spec.
usage: tauopt-bench run [-h] [--protocol PROTOCOL] [--shard SHARD] --out OUT
[--workers WORKERS] [--torch-threads N]
[--max-tasks-per-worker N]
options:
-h, --help show this help message and exit
--protocol PROTOCOL preset name or protocol JSON path
--shard SHARD path to a shard-NNNN.json spec
--out OUT run directory (created). One that already holds this
protocol's receipts.jsonl is RESUMED: its cells are
skipped and the rest appended; one holding another
protocol's, another shard's, or another environment's
receipts, or a cell twice, is refused, as is one
another run holds (its .run.lock)
--workers WORKERS parallel cell workers: N or 'auto' (all cores;
default). Receipts are worker-count-invariant; dial
down to leave cores free.
--torch-threads N pinned torch intra-op threads per cell (default 1;
recorded in the env fingerprint -- changing it changes
numerics, i.e. a different environment)
--max-tasks-per-worker N
recycle each pooled worker after N cells (default 1 =
a fresh process per cell: resets per-process map
counts and library arenas -- the S9 fleet-OOM fix; 0 =
legacy long-lived workers). Receipt-safe: recorded in
the run manifest, never in receipts
tauopt-bench fanout#
Fan a protocol out as one managed cloud job per shard (SkyPilot backend; –emit-yaml-only for BYO schedulers).
usage: tauopt-bench fanout [-h] --protocol PROTOCOL [--shards SHARDS] --out
OUT --bucket gs://... [--image docker:...]
[--source] [--campaign CAMPAIGN]
[--instance-type INSTANCE_TYPE] [--infra INFRA]
[--spot] [--isolate-adapter NAME]
[--max-restarts MAX_RESTARTS]
[--workers-per-shard WORKERS_PER_SHARD]
[--torch-threads TORCH_THREADS]
[--max-tasks-per-worker MAX_TASKS_PER_WORKER]
[--poll-interval POLL_INTERVAL]
[--poll-timeout POLL_TIMEOUT]
[--registry-token-auth] [--reuse-attempts]
[--emit-yaml-only]
[--cost-tables {none,auto,require}]
[--boot-image auto|none|projects/...]
options:
-h, --help show this help message and exit
--protocol PROTOCOL preset name or protocol JSON path
--shards SHARDS shard count, or 'auto' (default) = one wave:
ceil(general cells / 44) so every cell has a worker at
t=0 -- wall = provisioning + the slowest single cell
--out OUT
--bucket gs://... GCS bucket (receipts handoff: write-local in the
fleet, upload-last, pull back + hash-verify)
--image docker:... docker mode (BYO/k8s substrates): image + machine type
ARE the env identity -- one of each per fleet, or
claims refuse. Broken under org-enforced OS Login on
GCP; use --source there.
--source setup mode (plain VM -- works under org-enforced OS
Login): git-archive THIS checkout's HEAD (refused
dirty), sha256-verified in-VM, pinned uv + frozen lock
build; env identity = the recorded fingerprint.
Exactly one of --image | --source.
--campaign CAMPAIGN bucket prefix (default: <protocol_id>-<launch stamp>)
--instance-type INSTANCE_TYPE
default: by roster (the 2026-09-01 fleet-memory
ruling) -- c3-highmem-44 when the protocol's roster
carries ax or vizier (the profiled wall: 44 workers x
~5-6 GB), else c3-standard-44 (fixed Sapphire Rapids:
one cpu_model across regions and the whole c3 family,
so mixes stay one substrate; standard allocates
fastest on-demand). An explicit value overrides the
table (ad-hoc rosters). The 2026-07-09 'OOM' was map-
count exhaustion (setup sysctl = the fix) -- never
escalate for a costumed wall
--infra INFRA 'gcp' roams regions; 'gcp/us-central1' pins one
--spot spot instead of on-demand (ON-DEMAND is the ruled
campaign default, D9.23 -- never spot for campaigns;
spot + restart-from-scratch recovery suits cheap non-
campaign experiments)
--isolate-adapter NAME
pull this adapter's cells into a dedicated trailing
shard with NO fleet task -- run it locally from its
isolation venv and merge after (hebo: bench/hebo-
venv.sh)
--max-restarts MAX_RESTARTS
managed-jobs retries for INFRA failures (adapter
errors become receipt rows and never retry)
--workers-per-shard WORKERS_PER_SHARD
tauopt-bench run --workers inside each shard VM
--torch-threads TORCH_THREADS
--max-tasks-per-worker MAX_TASKS_PER_WORKER
tauopt-bench run --max-tasks-per-worker inside each
shard VM (default 1 = recycle every cell; 0 = legacy
long-lived workers)
--poll-interval POLL_INTERVAL
--poll-timeout POLL_TIMEOUT
seconds before the poll loop fails LOUD (completed
shards stay pullable from the bucket)
--registry-token-auth
private-registry pulls: mint a short-lived docker
login from YOUR gcloud auth into the task YAMLs
(Artifact Registry oauth2accesstoken; ~1h -- a shard
restarting after expiry fails its pull loudly;
relaunch re-mints)
--reuse-attempts allow a NON-EMPTY campaign bucket prefix -- only for
resuming the same code + protocol; a rerun never
adopts prior attempts silently
--emit-yaml-only write plan/ + tasks/ and stop (no cloud deps needed)
--cost-tables {none,auto,require}
cell-cost priors for shard packing: auto (the default
-- Scott ruling 2026-07-12) = LPT + longest-first from
committed campaign timing tables (lpt-cost-v1, an
unscaled mixed-table heuristic), falling back to
legacy round-robin WITH the fallback recorded when a
checkout has no tables; require = refuse instead of
falling back; none = legacy round-robin, bit-exact
(scheduling-only either way; receipts are partition-
invariant, PB1; provenance -> plan/cost-model.json)
--boot-image auto|none|projects/...
setup-mode VM boot image: 'auto' (default) uses the
baked fleet image matching the current lockhash when
one exists (setup self-heals to a no-op, ~10 min ->
~2-3); 'none' forces the full env build; or an
explicit projects/<proj>/global/images/<name>
tauopt-bench fanout-bake#
Bake the GCP fleet image for the current lock (idempotent; fanout –source then auto-detects it).
usage: tauopt-bench fanout-bake [-h] [--instance-type INSTANCE_TYPE] --project
PROJECT [--out OUT] [--timeout TIMEOUT]
options:
-h, --help show this help message and exit
--instance-type INSTANCE_TYPE
bake VM size (x86 only -- the bake asserts X86_64 and
deletes an ARM image rather than let auto-detection
bind it to c3 fleets); the env build is not CPU-bound
--project PROJECT exact GCP project; must match the effective Sky
workspace
--out OUT receipt + task-yaml directory
--timeout TIMEOUT
tauopt-bench fanout-bake-candidate#
Bake + STAMP a sealed code-bound candidate image for an envelope’s activation (never idempotent: one mint per (envelope, commit, variant); an existing name refuses).
usage: tauopt-bench fanout-bake-candidate [-h] --envelope ENVELOPE
[--variant VARIANT]
[--instance-type INSTANCE_TYPE]
--project PROJECT [--out OUT]
[--timeout TIMEOUT]
options:
-h, --help show this help message and exit
--envelope ENVELOPE envelope id (e.g. e55r); names the image + family
--variant VARIANT rebake variant -- currently REFUSED for any non-empty
value: the warm prewarm recipe is a future reviewed
lane (a variant that bakes nothing extra must not
mint)
--instance-type INSTANCE_TYPE
bake VM size (x86 only, same law as fanout-bake)
--project PROJECT exact GCP project; must match the effective Sky
workspace
--out OUT receipt + task-yaml directory
--timeout TIMEOUT
tauopt-bench skyseam-emit#
Emit a self-contained sky task YAML via the I3 seam (BYO schedulers; consumable by bare sky launch <file> --dryrun with zero tauopt imports).
usage: tauopt-bench skyseam-emit [-h] --name NAME --run RUN --infra INFRA
[--instance-type INSTANCE_TYPE] [--cpus CPUS]
[--image projects/...|docker:...] [--env K=V]
[--setup SETUP] [--num-nodes NUM_NODES]
[--use-spot] [--managed] [--out OUT]
options:
-h, --help show this help message and exit
--name NAME task/cluster name (^[a-z][a-z0-9-]*$)
--run RUN run command
--infra INFRA repeatable; ORDER = failover preference (gcp/us-
central1, gcp/us-east1/us-east1-b, k8s, ssh/mynode)
--instance-type INSTANCE_TYPE
provider-shape sizing (gcp tiers)
--cpus CPUS cpu sizing for k8s/local/ssh tiers (e.g. 0.5, 4+)
--image projects/...|docker:...
--env K=V
--setup SETUP
--num-nodes NUM_NODES
--use-spot user tier only (the sealed tier's never-spot law lives
upstream of the seam)
--managed stamp the managed-tier job_recovery default
(EAGER_NEXT_REGION)
--out OUT output path; '-' = stdout
tauopt-bench merge#
Merge shard receipts (one protocol only).
usage: tauopt-bench merge [-h] --inputs INPUTS [INPUTS ...] --out OUT
[--protocol PROTOCOL]
options:
-h, --help show this help message and exit
--inputs INPUTS [INPUTS ...]
receipt files or run directories
--out OUT
--protocol PROTOCOL optional: verify the merge is EXACTLY this protocol's
cell set
tauopt-bench canonical#
Print canonical receipt rows (sorted; volatile fields stripped).
usage: tauopt-bench canonical [-h] path
positional arguments:
path receipts.jsonl or a run directory
options:
-h, --help show this help message and exit
tauopt-bench report#
Render receipts into the campaign report (md + csv tables).
usage: tauopt-bench report [-h] --receipts RECEIPTS --out OUT [--claims]
[--companion-receipts COMPANION_RECEIPTS]
options:
-h, --help show this help message and exit
--receipts RECEIPTS receipts.jsonl or a run directory
--out OUT report output directory
--claims claims mode: require a frozen preset, a complete grid,
and zero error rows
--companion-receipts COMPANION_RECEIPTS
receipts of the companion campaign carrying the strong
baselines (required for the s8-floor audit on presets
with a declared companion)