Parameters and search spaces

Three parameter types

A space is an ordered list of parameters, and the order is part of its meaning. double is a continuous value in a bounded range. integer is an integer-valued box: the optimizer works on its continuous relaxation and rounds on the way out, so the model never sees the integer structure and on a small range a whole interval of continuous optima rounds to one value; for a handful of admissible values, declare an integer grid or a categorical instead. categorical takes one of at least two distinct, non-empty strings – no bounds, grid, or log scale.

The strategies see the space through two views. The space-filling rungs draw one unit coordinate per parameter and decode categories and grid values by equal-width buckets; the GP rungs decode the model view to the nearest grid value (ties to the lower). The Gaussian-process (GP) rungs embed categoricals one-hot and numerics on their unit (or log) scale, so a categorical with five values contributes five model dimensions – the reason many-valued categoricals cost more model capacity than a numeric of the same range.

Bounds, log scale, and grids

A numeric parameter takes exactly one of bounds or grid. Bounds are a finite (min, max) with min < max; integer bounds must be whole numbers. log_scale=True optimizes a double in log10 space and requires values above zero; it is not supported on integers (the refusal reads log_scale integers are not supported), so a log-spaced integer such as a batch size is either an integer grid of the values you want or a log-scale double that your evaluation rounds. A grid restricts a numeric parameter to an explicit list of at least two finite, distinct values; the bounds derive from the grid’s extremes. Grids are stored ascending whatever order you wrote them in (so the GP rungs’ nearest-value decoding is deterministic, ties to the lower; the space-filling rungs decode by equal-width bucket), and an observation on a grid parameter must land on a grid value exactly.

Choose a grid when the evaluation can only run at those values; choose bounds plus integer when any value in the range is meaningful.

Conditional parameters

A parameter with conditions is active only when every named parent takes one of the listed values. The shape is a mapping from parent name to a tuple of that parent’s categorical values, and two rules make it well defined: each parent must be a categorical parameter earlier in the list, and the listed values must belong to it. Chains resolve in one forward pass, so an inactive parent deactivates its children.

Inactive parameters are absent from assignments rather than set to a placeholder. Validation requires exactly the active set, which is why reporting a momentum for an adam run below is refused:

from tauopt import Parameter, Space

space = Space(parameters=(
    Parameter(name="optimizer", type="categorical", categorical_values=("sgd", "adam")),
    Parameter(name="lr", type="double", bounds=(1e-4, 1e-1), log_scale=True),
    Parameter(name="momentum", type="double", bounds=(0.0, 0.99),
              conditions={"optimizer": ("sgd",)}),
    Parameter(name="layers", type="integer", grid=(8, 2, 4)),
))
print([p.name for p in space.active_parameters({"optimizer": "adam"})])
print(space.from_unit([0.9, 0.5, 0.5, 0.5]))
print(space.from_unit([0.1, 0.5, 0.5, 0.5]))
print(space.canonical_json())
try:
    space.validate_assignments({"optimizer": "adam", "lr": 0.01, "momentum": 0.9, "layers": 4})
except ValueError as e:
    print("refused:", e)

['optimizer', 'lr', 'layers']
{'optimizer': 'adam', 'lr': 0.0031622776601683794, 'layers': 4}
{'optimizer': 'sgd', 'lr': 0.0031622776601683794, 'momentum': 0.495, 'layers': 4}
{"parameters":[{"categorical_values":["sgd","adam"],"log_scale":false,"name":"optimizer","type":"categorical"},{"bounds":[0.0001,0.1],"log_scale":true,"name":"lr","type":"double"},{"bounds":[0.0,0.99],"conditions":{"optimizer":["sgd"]},"log_scale":false,"name":"momentum","type":"double"},{"grid":[2.0,4.0,8.0],"log_scale":false,"name":"layers","type":"integer"}]}
refused: assignments keys ['layers', 'lr', 'momentum', 'optimizer'] != active parameters ['layers', 'lr', 'optimizer']

In the GP view an inactive numeric parameter imputes at mid-range (unit 0.5), which on that axis is indistinguishable from an active parameter sitting at mid-range; the parent’s one-hot block is what separates the branches. An inactive categorical imputes to an all-zero one-hot block, a region no category occupies.

Linear constraints

A linear constraint is a weighted sum of parameter values compared with a right-hand side: terms maps names to finite, nonzero weights, op is >= or <=, and rhs is finite. Constraints may cover plain double parameters only – no integers, categoricals, log-scaled, grid, or conditional parameters. The restriction keeps the transform between raw and unit coordinates affine, so the GP optimizer can satisfy the constraint exactly and every candidate it returns is feasible by construction; the space-filling rungs honor constraints by rejection instead. Satisfaction allows a relative tolerance of 1e-8 so a point the solver considers feasible never bounces off validation. At experiment creation a fixed-seed probe of 4,096 points rejects a constraint region with no feasible hits.

from pydantic import ValidationError

from tauopt import LinearConstraint, Parameter, Space

box = (Parameter(name="a", type="double", bounds=(0.0, 1.0)),
       Parameter(name="n", type="integer", bounds=(1, 10)))
ok = Space(parameters=box, linear_constraints=(
    LinearConstraint(terms={"a": 1.0}, op="<=", rhs=0.5),))
print(ok.linear_constraints[0].satisfied({"a": 0.25, "n": 3}))
try:
    Space(parameters=box, linear_constraints=(
        LinearConstraint(terms={"a": 1.0, "n": 1.0}, op="<=", rhs=5.0),))
except ValidationError as e:
    print("refused:", e.errors()[0]["msg"])
plain = Space(parameters=(Parameter(name="a", type="double", bounds=(0.0, 1.0)),))
explicit = Space(parameters=(Parameter(name="a", type="double", bounds=(0.0, 1.0),
                                       log_scale=False, grid=None, conditions=None),))
print(plain.canonical_json() == explicit.canonical_json(), plain.canonical_json())

True
refused: Value error, linear constraint on 'n': constraints support plain double parameters only (no integer/categorical/log_scale/grid/conditions)
True {"parameters":[{"bounds":[0.0,1.0],"log_scale":false,"name":"a","type":"double"}]}

Canonical form and why it matters for hashing

A space serializes to one canonical string: parameter order preserved, keys sorted within each object, compact separators, and fields added after the original format dropped when they sit at their default (bounds, categorical_values, grid, and conditions at null, linear_constraints when empty). Original fields such as log_scale always serialize. An omitted default and an explicit default therefore produce the same bytes, as the last line above shows.

The space hash over those bytes is the scope the Sobol rung derives its scramble seed from, so the sequence stays one sequence as observations accrue; the state hash (space plus history) is what the point strategies seed from. Reordering parameters changes the space hash and therefore the stream; an observation changes the state hash but not the space hash:

from tauopt import ExperimentState, Parameter, Space
from tauopt.engine import Objective, ObservationRecord

a = Parameter(name="a", type="double", bounds=(0.0, 1.0))
b = Parameter(name="b", type="integer", bounds=(1, 4))
ab = ExperimentState(space=Space(parameters=(a, b)), objective=Objective(metric="m"))
ba = ExperimentState(space=Space(parameters=(b, a)), objective=Objective(metric="m"))
print(ab.space_hash(), ba.space_hash())
later = ab.with_observation(ObservationRecord(assignments={"a": 0.5, "b": 2}, value=1.0))
print(ab.space_hash() == later.space_hash(), ab.state_hash() == later.state_hash())

blake2b_26da538ff46fa778 blake2b_d12de99b61f9c2d7
True False

The drop-at-default rule is a compatibility promise: a space written before a field existed keeps its bytes and hashes after an upgrade, so an in-flight experiment never re-scrambles and an old receipt still re-derives. The field-level schemas are in the Python API reference.

Do it: Use conditional parameters and Add constraints.