Add constraints#

TauOpt has two constraint kinds. A linear constraint restricts the parameter space itself, and every suggestion satisfies it: the Gaussian-process (GP) strategies by construction, the space-filling strategies by rejection. An outcome constraint is a measured metric with a threshold; it decides which observations can be reported as best and steers gp_ei and gp_nehvi toward feasible regions. Background: Parameters and search spaces and Metrics and objectives. The outputs below were produced on one machine with one dependency set and will differ on another.

Closed beta (nothing installed): the same constraints over plain HTTP are mixed_space_rest.py (a linear constraint in linear_constraints of the POST /v1/experiments body) and outcome_constraints_rest.py (a strategy: "constraint" metric with a threshold in metrics; the feasibility check reads GET /v1/experiments/{id}/observations).

Add a linear constraint#

A linear constraint may only reference plain double parameters: no integer, categorical, log_scale, grid, or conditions on any term (the refusal reads constraints support plain double parameters only). Constraints are part of the immutable space, so declare them at creation.

  1. Write each constraint as {"terms": {"<name>": <coefficient>, ...}, "op": "<=" | ">=", "rhs": <number>}. Coefficients must be finite and nonzero, rhs finite, and terms non-empty.

  2. Pass the list as linear_constraints=[...] to create_experiment.

  3. If the constraints leave no room, creation fails at once with constraint region too small: 0/4096 sobol probe points feasible (a fixed-seed Sobol probe of 4096 points found none feasible). Widen the bounds or relax rhs.

  4. Evaluate suggestions as usual: every one satisfies every constraint, on every ladder rung, to the engine’s feasibility tolerance of 1e-8 * (1 + |rhs|) (a Gaussian-process candidate on a constraint’s boundary may sit inside that slack). Reports you send by assignments are validated the same way, with the same tolerance.

from tauopt import ApiError, Client

client = Client(driver="local")
exp = client.create_experiment(
    name="mixture-weights",
    parameters=[
        {"name": "w1", "type": "double", "bounds": (0.0, 1.0)},
        {"name": "w2", "type": "double", "bounds": (0.0, 1.0)},
    ],
    # 1.0 * w1 + 1.0 * w2 <= 1.0
    linear_constraints=[{"terms": {"w1": 1.0, "w2": 1.0}, "op": "<=", "rhs": 1.0}],
    metric={"name": "score", "objective": "maximize"},
)
print("declared:", exp.linear_constraints)

for i in range(10):
    sug = client.suggest(
        exp.id, config={"strategy": "auto", "seed": 3, "timebox_mode": "caps"}
    )
    w1, w2 = sug.assignments["w1"], sug.assignments["w2"]
    # the engine's feasibility tolerance, 1e-8 * (1 + |rhs|): the slack its validator allows a report
    tol = 1e-8 * (1.0 + abs(1.0))
    assert w1 + w2 <= 1.0 + tol, f"a suggestion violated the constraint by {w1 + w2 - 1.0:.3e} (tolerance {tol:.1e})"
    client.observe(exp.id, suggestion_id=sug.id, value=-((w1 - 0.3) ** 2) - (w2 - 0.6) ** 2)
    print(i, sug.provenance.strategy, "w1 + w2 =", round(w1 + w2, 4))

# an off-suggestion report must satisfy the constraint too
try:
    client.observe(exp.id, assignments={"w1": 0.8, "w2": 0.8}, value=0.0)
except ApiError as e:
    print(e)

# a region the constraints exclude is refused at creation, not at the first ask
try:
    client.create_experiment(
        name="empty-region",
        parameters=[
            {"name": "w1", "type": "double", "bounds": (0.0, 1.0)},
            {"name": "w2", "type": "double", "bounds": (0.0, 1.0)},
        ],
        linear_constraints=[{"terms": {"w1": 1.0, "w2": 1.0}, "op": ">=", "rhs": 3.0}],
        metric={"name": "score", "objective": "maximize"},
    )
except ApiError as e:
    print(e)

declared: (LinearConstraintSpec(terms={'w1': 1.0, 'w2': 1.0}, op='<=', rhs=1.0),)
0 sobol w1 + w2 = 0.2931
1 sobol w1 + w2 = 0.9351
2 sobol w1 + w2 = 0.4346
3 sobol w1 + w2 = 0.735
4 gp_ei w1 + w2 = 0.4184
5 gp_ei w1 + w2 = 1.0
6 gp_ei w1 + w2 = 0.8304
7 gp_ei w1 + w2 = 1.0
8 gp_ei w1 + w2 = 0.9595
9 gp_ei w1 + w2 = 0.8872
[400] validation_error: linear constraint violated: {'w1': 1.0, 'w2': 1.0} <= 1.0 at {'w1': 0.8, 'w2': 0.8}
[400] validation_error: constraint region too small: 0/4096 sobol probe points feasible

Add an outcome constraint#

Outcome constraints need metrics mode. A single metric= carrying a non-optimize strategy or a threshold is refused with metric strategy/threshold require metrics mode: use metrics=[...].

  1. Declare the constrained metric inside metrics=[...] with "strategy": "constraint" and a threshold (required). With "objective": "minimize" a point is feasible when the value is at most the threshold; with "maximize", at least the threshold.

  2. Report the constraint metric on every successful observe; it is a required name in values (a failed=True observation carries no values).

  3. Read best(exp.id): only feasible observations qualify, and the result is empty (count == 0) until one exists.

  4. Read suggestion.provenance.strategy when feasibility matters: gp_ei and gp_nehvi model the constraint metric and weight the acquisition by feasibility, while ucb_cl models the objective only and sobol and random ignore the constraint, so a suggestion answered from one of those rungs (during the bootstrap, or with fallback_from set) can land in the infeasible region.

from tauopt import ApiError, Client

client = Client(driver="local")
exp = client.create_experiment(
    name="loss-under-a-memory-cap",
    parameters=[
        {"name": "width", "type": "double", "bounds": (0.0, 1.0)},
        {"name": "depth", "type": "double", "bounds": (0.0, 1.0)},
    ],
    metrics=[
        {"name": "loss", "objective": "minimize"},
        # feasible when mem_gb <= 4.0 ("minimize" makes the threshold an upper bound)
        {"name": "mem_gb", "objective": "minimize", "strategy": "constraint", "threshold": 4.0},
    ],
)


def evaluate(a):
    loss = (1.0 - a["width"]) ** 2 + (1.0 - a["depth"]) ** 2  # best at the top corner
    mem_gb = 8.0 * a["width"] * a["depth"]                      # the corner costs 8 GB
    return {"loss": loss, "mem_gb": mem_gb}


for i in range(12):
    sug = client.suggest(
        exp.id, config={"strategy": "auto", "seed": 5, "timebox_mode": "caps"}
    )
    values = evaluate(sug.assignments)
    client.observe(exp.id, suggestion_id=sug.id, values=values)
    print(i, sug.provenance.strategy, round(values["loss"], 4), round(values["mem_gb"], 2),
          "feasible" if values["mem_gb"] <= 4.0 else "infeasible")

# an infeasible point with a better loss never becomes best
client.observe(exp.id, assignments={"width": 1.0, "depth": 1.0}, values={"loss": 0.0, "mem_gb": 8.0})
best = client.best(exp.id).data[0]
print("best loss:", round(best.value, 4), "at", {k: round(v, 3) for k, v in best.assignments.items()})

# every successful observe must carry the constraint metric
try:
    client.observe(exp.id, assignments={"width": 0.5, "depth": 0.5}, values={"loss": 0.5})
except ApiError as e:
    print(e)

0 sobol 0.7183 1.2 feasible
1 sobol 0.5917 1.64 feasible
2 sobol 0.8474 0.93 feasible
3 sobol 0.2494 3.58 feasible
4 gp_ei 0.1828 4.58 infeasible
5 gp_ei 0.1046 4.76 infeasible
6 gp_ei 1.0 0.0 feasible
7 gp_ei 0.0 8.0 infeasible
8 gp_ei 0.187 3.87 feasible
9 gp_ei 0.173 3.99 feasible
10 gp_ei 0.2136 4.3 infeasible
11 gp_ei 0.1701 4.01 infeasible
best loss: 0.173 at {'width': 0.719, 'depth': 0.694}
[400] validation_error: values missing required metrics ['mem_gb']

With one optimized metric, best returns that metric under value. To tighten or relax an outcome constraint later, call update_experiment(exp.id, metric_thresholds={"mem_gb": 2.0}); the mechanics are the same as for frontier thresholds in Optimize multiple metrics.