Request multiple solutions#

Ask the experiment for k solutions when you want several good candidates from different regions of the space rather than one winner. Optimization runs exactly as before; only best changes. Background on single-objective best lives in Metrics and objectives. The numbers below are one machine’s; another dependency set prints others.

Closed beta (nothing installed): the same request over plain HTTP is experimental_design_rest.py: num_solutions: 3 in the JSON body of POST /v1/experiments, and GET /v1/experiments/{id}/best_assignments returns count entries in data.

Ask for k solutions#

  1. Use a single objective: metric=..., or metrics=[...] with exactly one optimized metric.

  2. Pass num_solutions=k (an integer of at least 1) to create_experiment. The value is fixed for the life of the experiment; update_experiment cannot change it.

  3. Run the ask-observe loop as usual.

  4. Read best(exp.id): data[0] is the best observation; each further entry is the candidate farthest (by its minimum distance in the unit cube of the search space) from the entries already chosen, drawn from a pool of the top 2k scored observations by value, or the top 20 percent of them when that is more. The result holds fewer than k entries while fewer scored observations exist, and only feasible ones when an outcome constraint is declared.

  5. Read experiment(exp.id).progress.best_observation when you want the single best alone.

from tauopt import ApiError, Client

client = Client(driver="local")
exp = client.create_experiment(
    name="three-candidates",
    parameters=[
        {"name": "x1", "type": "double", "bounds": (0.0, 1.0)},
        {"name": "x2", "type": "double", "bounds": (0.0, 1.0)},
    ],
    metric={"name": "score", "objective": "maximize"},
    num_solutions=3,
)


def score(a):
    # two good basins: around (0.2, 0.8) and around (0.8, 0.2)
    left = -((a["x1"] - 0.2) ** 2) - (a["x2"] - 0.8) ** 2
    right = -((a["x1"] - 0.8) ** 2) - (a["x2"] - 0.2) ** 2
    return max(left, right)


for i in range(14):
    sug = client.suggest(
        exp.id, config={"strategy": "auto", "seed": 4, "timebox_mode": "caps"}
    )
    client.observe(exp.id, suggestion_id=sug.id, value=score(sug.assignments))

best = client.best(exp.id)
print("count:", best.count)
for entry in best.data:
    print(round(entry.value, 4), {k: round(v, 3) for k, v in entry.assignments.items()})
print("scalar best:", round(client.experiment(exp.id).progress.best_observation.value, 4))

# num_solutions is single-objective only
try:
    client.create_experiment(
        name="frontier-and-k",
        parameters=[{"name": "x", "type": "double", "bounds": (0.0, 1.0)}],
        metrics=[
            {"name": "a", "objective": "minimize"},
            {"name": "b", "objective": "minimize"},
        ],
        num_solutions=3,
    )
except ApiError as e:
    print(e)

count: 3
-0.0 {'x1': 0.799, 'x2': 0.196}
-0.0199 {'x1': 0.06, 'x2': 0.815}
-0.0027 {'x1': 0.252, 'x2': 0.801}
scalar best: -0.0
[400] validation_error: num_solutions is single-objective only

After data[0], distance orders the entries: the second is the point farthest from the first (in the other basin), and the third, despite its better value, is the point farthest from both of them.

Combine with metrics mode#

num_solutions > 1 with two or more optimized metrics is refused at creation with num_solutions is single-objective only, as the end of the run shows. With two or more optimized metrics best already returns a set, the Pareto frontier; see Optimize multiple metrics. A metrics-mode experiment with one optimized metric plus constraint or stored metrics is single-objective and accepts num_solutions.