hillclimb

Hillclimbing on verifiable rewards. Use Claude Code and Codex to autonomously search for python programs that optimize a score you define.

hillclimb is a CLI and modular Python framework for building, testing and benchmarking hillclimbing algorithms powered by coding agents. It provides reusable components and a common way to evaluate different search strategies across different problems — from speeding up code and solving optimisation problems to improving prediction accuracy. The goal of hillclimb to create a framework to enable discovery of better autoresearch methods, rather than build a single best autoresearcher.

There are two main parts: the problem and the climber. A problem defines what counts as better through a verifier script, verifier.sh, and optionally includes a starting solution, solution.py. A climber is a search algorithm that drafts initial solutions and then decides which solutions to build on and what to try next. Start with built-in problems and climbers, or write your own ones.

hillclimb run starts the search in the background and returns control of your terminal. Within the compute budget you set, the climber dispatches headless coding agents to write, debug and improve solutions. Your verifier scores each attempt, and hillclimb keeps the best result.

On this page#

Installation#

hillclimb needs Python 3.12+ and installs from PyPI. Make a virtual environment with uv and install it there:

uv venv --python 3.12 source .venv/bin/activate uv pip install hillclimb

Or install it as a standalone tool, so the hillclimb command works everywhere without activating an environment:

uv tool install hillclimb

Quickstart#

With six CLI commands you can get up and running with your first hillclimb search, and see it climb in the TUI. The walkthrough below, goes through every step in more details.

uv tool install hillclimb # the CLI, Python 3.12+ hillclimb connect claude # or codex, openrouter hillclimb init # make this folder a hillclimb dir hillclimb problem get heilbronn-convex-13 # the problem to climb hillclimb climber get greedy # the climber: policy + prompts hillclimb run heilbronn-convex-13 --budget 30m # start climbing hillclimb watch # the live TUI

What is hillclimb for?#

Anything you can phrase as: a program or artifact in, a number out, and a verifier that computes a number/score (potentially using data the agents never see). Some problem types that are well suited for hillclimb:

Problem typeWhat Hillclimb improves
OptimizationPacking, routing and scheduling solutions, scored by solution quality, cost or constraint violations.
Prediction and forecastingTraining and prediction code, evaluated on hidden data using metrics such as MAE, pinball loss or CRPS.
Performance engineeringCode for a fixed workload, scored by runtime, memory use, binary size or another resource constraint.
Parameter fittingEstimation code that recovers unknown parameters, tested against cases with known ground truth.
Strategies and policiesDispatch, bidding and cache-eviction strategies, evaluated by replaying historical or simulated scenarios.
Generated artifactsSQL queries, regular expressions, solver configurations and prompts—anything that can be generated and scored.
Mathematical discoveryConstructions, counterexamples and bounds, scored by a programmatically verifiable mathematical objective.

Where it fits poorly. The search loop needs many candidates per budget so verifiers that take hours to run are not a great match with hillclimb. Objectives without a scalar score, like UX, prose or "nicer code". Pass/fail verifiers with no partial credit, since a 0/1 score gives the search nothing to climb. Low-dimensional continuous optimisation, where a numerical optimiser is the better tool.

What else is out there?#

hillclimb doesn't aim to build the best single algorithm. Rather, it is a modular framework for evaluating different methods for your problem and building your own. Below are some of the hillclimbing algorithms out there. Our ambition is to integrate the best and most promising of these, and of those still to come.

Full walkthrough#

Check out the hillclimb commands (and enjoy our ASCII art intro 🏔️😂).

hillclimb --help

Connect your coding agent, hillclimb will use it in the background.

hillclimb connect claude # or codex, openrouter

Initialise hillclimb to make your working directory a hillclimb dir. By default, everything hillclimb reads and writes lives here: hillclimb.yaml, problems/, climbers/ and runs/ (with the run defaults in runs/config.yaml).

hillclimb init

The command hillclimb problem get [problem] gets the problem definition into your working directory. The following command loads the Heilbronn triangle problem for this walkthrough (get a feel for this problem here).

hillclimb problem get heilbronn-convex-13
problems/heilbronn-convex-13/
├── problem.yaml           # metric, direction — the problem's identity
├── description.md         # what the coding agents read before drafting
├── interface.py           # the output format, machine-checked (hillclimb spaces)
├── verifier.sh            # the ONLY process hillclimb starts: drives solution.py and reports the score
├── verify.py              # the scorer — writes the score to $HILLCLIMB_RESULT
├── baseline.py            # the starting solution scored at t=0
└── sample_submission.csv  # a valid, weak submission: the floor the search starts from

Get a climber the same way. A climber is how hillclimb climbs: the policy that picks the next move, and the prompt templates its operators send to the coding agents. hillclimb climber get greedy copies the pre-packaged greedy climber into climbers/greedy/ and makes it this folder's climber, so you can go through and modify the climber's policies and prompts before it runs.

hillclimb climber get greedy
climbers/greedy/
├── policy.py           # the climber, as Python: the selector policy (which candidate next), the operator policy (which operator on it), their defaults, and the Climber(...) that wires them to the operators, tuner, memory and prompts
└── prompts/
    ├── README.md       # when each template is used, what fills each {{token}}
    ├── draft.md        # the templates: the words the coding agents get
    ├── research_cue.md
    ├── debug.md
    ├── improve.md
    ├── ablation_cue.md
    └── ensemble.md

Each template is a skeleton of {{tokens}}. For every attempt the harness fills them with that moment's context: the problem, the candidate the attempt builds on and its score, what was already tried from it, what memory knows from earlier searches. Edit a template and the next run climbs with it.

climbers/greedy/prompts/improve.md show
You are improving the current best solution for the problem below.

# Problem

{{description}}

# Objective

Optimize **{{metric_name}}** ({{direction}}).

# Current best solution

`solution.py` in your working directory — validation {{metric_name}}: **{{best_score}}**.

Its execution stdout (tail):
```
{{stdout_tail}}
```

{{evaluation_report}}
# Already tried from this solution (do NOT repeat)

{{sibling_summaries}}

{{live_experience}}
{{prior_ablations}}
# Instruction

{{ablation_cue}}
Make **exactly ONE measurable change** to improve validation {{metric_name}}: one engineered feature, one model swap, one hyperparameter adjustment, one validation-scheme fix — a single attributable modification. State your hypothesis as the first line of `notes.md`. Keep everything else identical so the score change is attributable to your change.

{{contract}}

Start climbing. Parallelism in hillclimb has two levels: --parallel-searches sets how many independent climber searches (exploration trees) to run on the problem (each its own engine process, all in the same "run", so they share what they learn), --parallel-agents sets how many coding agents each search uses to draft, debug and improve candidate solutions.

# --budget: or CPU time, tokens, evaluations hillclimb run heilbronn-convex-13 \ --budget 30m \ --parallel-searches 2 \ --parallel-agents 3 \ --agent claude-code \ --model claude-sonnet-5

Then run hillclimb watch to open the TUI and enjoy your coffee as the agents search for solutions ☕. Some other useful commands are hillclimb chart, which gives the chart with successive progression (see More visualisations) and hillclimb stop --all stops the run.

hillclimb watch

At the end of the run, or at any point mid-climb, run hillclimb summit to extract the best solution found so far into the root of your working directory. It adds two files there: solution.py, the code the agents wrote, and submission.csv, its output, which is what the verifier scores. Add --plot to also draw it into solution.png.

hillclimb summit
solution.py show
"""Heilbronn (convex-hull normalised), n=13.

Symmetry-reduced smoothing homotopy: the max-min objective is replaced by a
log-sum-exp soft-min of |triangle area| / hull area, maximised with L-BFGS
(analytic gradients) while the sharpness beta is annealed upward.  Starts are
drawn in symmetry-reduced coordinates (mirror, 3-fold rotation, or none), the
symmetry is then released, and promising results get an exact epigraph polish.
"""
import os
import time
from itertools import combinations
from multiprocessing import Pool

import numpy as np
from scipy.optimize import minimize
from scipy.spatial import ConvexHull

try:
    from hillclimb import spaces
    P = spaces.params()
except Exception:  # noqa: BLE001
    P = {}
TIME_BUDGET = float(P.get("time_budget", 45.0))
BETA0 = float(P.get("beta_start", 300.0))
BETA_GROWTH = float(P.get("beta_growth", 3.0))
SYM_FRAC = float(P.get("sym_frac", 0.6))
EXPLORE_FRAC = float(P.get("explore_frac", 0.5))  # restarts, then basin hopping

N = 13
TRI = np.array(list(combinations(range(N), 3)))
I, J, K = TRI[:, 0], TRI[:, 1], TRI[:, 2]
NT = len(TRI)


def signed_areas(X):
    x, y = X[:, 0], X[:, 1]
    return 0.5 * ((x[J] - x[I]) * (y[K] - y[I]) - (x[K] - x[I]) * (y[J] - y[I]))


def hull_info(X):
    h = ConvexHull(X).vertices  # counter-clockwise in 2-D
    x, y = X[h, 0], X[h, 1]
    H = 0.5 * abs(np.dot(x, np.roll(y, -1)) - np.dot(y, np.roll(x, -1)))
    return h, H


def score(X):
    try:
        _, H = hull_info(X)
    except Exception:  # noqa: BLE001
        return 0.0
    return float(np.min(np.abs(signed_areas(X))) / H)


def area_grad_terms(X, coef):
    """Gradient of sum_i coef_i * A_i (signed areas) w.r.t. X."""
    x, y = X[:, 0], X[:, 1]
    G = np.zeros((N, 2))
    np.add.at(G[:, 0], I, coef * 0.5 * (y[J] - y[K]))
    np.add.at(G[:, 0], J, coef * 0.5 * (y[K] - y[I]))
    np.add.at(G[:, 0], K, coef * 0.5 * (y[I] - y[J]))
    np.add.at(G[:, 1], I, coef * 0.5 * (x[K] - x[J]))
    np.add.at(G[:, 1], J, coef * 0.5 * (x[I] - x[K]))
    np.add.at(G[:, 1], K, coef * 0.5 * (x[J] - x[I]))
    return G


def hull_grad(X, h):
    G = np.zeros((N, 2))
    x, y = X[h, 0], X[h, 1]
    G[h, 0] = 0.5 * (np.roll(y, -1) - np.roll(y, 1))
    G[h, 1] = 0.5 * (np.roll(x, 1) - np.roll(x, -1))
    return G


def neg_softmin(z, beta, A):
    """-softmin_beta(|A_i|/H) and gradient, with X = (A @ z) when A given."""
    X = (A @ z if A is not None else z).reshape(N, 2)
    try:
        h, H = hull_info(X)
    except Exception:  # noqa: BLE001
        return 0.0, np.zeros_like(z)
    a = signed_areas(X)
    s = np.sign(a)
    r = np.abs(a) / H
    m = r.min()
    e = np.exp(-beta * (r - m))
    S = e.sum()
    f = m - np.log(S / NT) / beta
    w = e / S
    G = area_grad_terms(X, w * s / H) - (np.dot(w, r) / H) * hull_grad(X, h)
    g = G.ravel()
    if A is not None:
        g = A.T @ g
    return -f, -g


def sym_matrix(kind, rng):
    """Linear map from reduced coordinates to the 26 point coordinates."""
    if kind == "mirror":
        # 6 mirrored pairs + 1 point on the axis (x = 0)
        A = np.zeros((2 * N, 13))
        for p in range(6):
            A[2 * (2 * p), 2 * p] = 1.0
            A[2 * (2 * p) + 1, 2 * p + 1] = 1.0
            A[2 * (2 * p + 1), 2 * p] = -1.0
            A[2 * (2 * p + 1) + 1, 2 * p + 1] = 1.0
        A[2 * 12 + 1, 12] = 1.0
        z = rng.uniform(-1, 1, 13)
        return A, z
    if kind == "rot3":
        # 4 orbits of the 3-fold rotation + the centre
        A = np.zeros((2 * N, 8))
        for o in range(4):
            for q in range(3):
                c, s = np.cos(2 * np.pi * q / 3), np.sin(2 * np.pi * q / 3)
                p = 3 * o + q
                A[2 * p, 2 * o] = c
                A[2 * p, 2 * o + 1] = -s
                A[2 * p + 1, 2 * o] = s
                A[2 * p + 1, 2 * o + 1] = c
        z = rng.uniform(-1, 1, 8)
        return A, z
    if kind == "point":
        # 6 antipodal pairs + the centre (kept slightly off-centre by release)
        A = np.zeros((2 * N, 12))
        for p in range(6):
            for d in range(2):
                A[2 * (2 * p) + d, 2 * p + d] = 1.0
                A[2 * (2 * p + 1) + d, 2 * p + d] = -1.0
        z = rng.uniform(-1, 1, 12)
        return A, z
    raise ValueError(kind)


def anneal(z, A, betas, maxiter):
    for beta in betas:
        res = minimize(neg_softmin, z, args=(beta, A), jac=True,
                       method="L-BFGS-B", options={"maxiter": maxiter})
        z = res.x
    return z


def polish(X, rounds=6):
    """Exact epigraph polish: max t s.t. s_i A_i >= t H, signs/hull frozen."""
    best, bs = X.copy(), score(X)
    for _ in range(rounds):
        X0 = best
        try:
            h, H0 = hull_info(X0)
        except Exception:  # noqa: BLE001
            break
        X0 = X0 / np.sqrt(H0)
        a0 = signed_areas(X0)
        s = np.sign(a0)
        t0 = np.abs(a0).min()
        act = np.abs(a0) < 2.0 * t0 + 1e-9  # only near-active triangles
        ia, ja, ka, sa = I[act], J[act], K[act], s[act]
        na = int(act.sum())
        rows = np.arange(na)

        def hull_area(X):
            x, y = X[h, 0], X[h, 1]
            return 0.5 * (np.dot(x, np.roll(y, -1)) - np.dot(y, np.roll(x, -1)))

        def cons(v):
            X = v[:-1].reshape(N, 2)
            x, y = X[:, 0], X[:, 1]
            a = 0.5 * ((x[ja] - x[ia]) * (y[ka] - y[ia])
                       - (x[ka] - x[ia]) * (y[ja] - y[ia]))
            return np.concatenate([sa * a - v[-1], [1.0 - hull_area(X)]])

        def cons_jac(v):
            X = v[:-1].reshape(N, 2)
            x, y = X[:, 0], X[:, 1]
            Jm = np.zeros((na + 1, 2 * N + 1))
            Jm[rows, 2 * ia] = sa * 0.5 * (y[ja] - y[ka])
            Jm[rows, 2 * ja] = sa * 0.5 * (y[ka] - y[ia])
            Jm[rows, 2 * ka] = sa * 0.5 * (y[ia] - y[ja])
            Jm[rows, 2 * ia + 1] = sa * 0.5 * (x[ka] - x[ja])
            Jm[rows, 2 * ja + 1] = sa * 0.5 * (x[ia] - x[ka])
            Jm[rows, 2 * ka + 1] = sa * 0.5 * (x[ja] - x[ia])
            Jm[:na, -1] = -1.0
            Jm[na, :-1] = -hull_grad(X, h).ravel()
            return Jm

        obj_g = np.zeros(2 * N + 1)
        obj_g[-1] = -1.0
        v0 = np.concatenate([X0.ravel(), [t0]])
        try:
            res = minimize(lambda v: -v[-1], v0, jac=lambda v: obj_g,
                           constraints=[{"type": "ineq", "fun": cons, "jac": cons_jac}],
                           method="SLSQP", options={"maxiter": 150, "ftol": 1e-14})
        except Exception:  # noqa: BLE001
            break
        Xn = res.x[:-1].reshape(N, 2)
        sn = score(Xn)
        if sn > bs + 1e-12:
            best, bs = Xn, sn
        else:
            break
    return best, bs


def worker(args):
    seed, budget, start = args
    rng = np.random.default_rng(seed)
    t_end = time.time() + budget
    betas = [BETA0 * BETA_GROWTH ** k for k in range(6)]
    best, bs = (None, -1.0) if start is None else (start.copy(), score(start))
    while time.time() < t_end:
        u = rng.random()
        if start is not None:
            # exploitation: basin-hop around the shared (pool-wide) incumbent
            _, H = hull_info(best)
            X = best / np.sqrt(H)
            X = X + 10 ** rng.uniform(-2.5, -1.0) * rng.standard_normal(X.shape)
            z = anneal(X.ravel(), None, betas[3:], 200)
        elif u < SYM_FRAC:
            kind = rng.choice(["mirror", "rot3", "point"], p=[0.5, 0.35, 0.15])
            A, z = sym_matrix(kind, rng)
            z = anneal(z, A, betas[:4], 150)
            X = (A @ z).reshape(N, 2)
            X = X + 1e-3 * rng.standard_normal(X.shape)  # release symmetry
            z = anneal(X.ravel(), None, betas[2:], 200)
        else:
            ang = rng.uniform(0, 2 * np.pi, N)
            rad = np.sqrt(rng.uniform(0, 1, N))
            X = np.c_[rad * np.cos(ang), rad * np.sin(ang)]
            z = anneal(X.ravel(), None, betas, 200)
        X = z.reshape(N, 2)
        sc = score(X)
        if sc > 0.9 * bs:
            X, sc = polish(X)
        if sc > bs:
            best, bs = X.copy(), sc
    return bs, best


def main():
    seed = int(os.environ.get("HILLCLIMB_REPLICATE_SEED", "0"))
    nproc = min(8, os.cpu_count() or 1)
    t_explore = EXPLORE_FRAC * TIME_BUDGET
    with Pool(nproc) as pool:
        explored = pool.map(worker, [(seed * 1000 + 17 * w + 1, t_explore, None) for w in range(nproc)])
        # exploitation: every worker basin-hops around one of the pool's top-3
        # incumbents (half of them around the very best) instead of its own
        ranked = sorted([r for r in explored if r[1] is not None], key=lambda r: -r[0])
        starts = [ranked[0 if w % 2 == 0 else min(1 + (w // 2) % 2, len(ranked) - 1)][1]
                  for w in range(nproc)]
        results = pool.map(worker, [(seed * 1000 + 17 * w + 7919, TIME_BUDGET - t_explore, starts[w])
                                    for w in range(nproc)])
    print("explore per-worker:", sorted(round(r[0], 6) for r in explored))
    bs, best = max(results, key=lambda r: r[0])
    best, bs = polish(best, rounds=20)
    h, H = hull_info(best)
    best = (best - best.mean(axis=0)) / np.sqrt(H)
    with open("submission.csv", "w") as f:
        f.write("id,x,y\n")
        for i, (x, y) in enumerate(best):
            f.write(f"{i},{float(x)!r},{float(y)!r}\n")
    print(f"best normalized min triangle area: {score(best):.12f}")
    print("per-worker:", sorted(round(r[0], 6) for r in results))


if __name__ == "__main__":
    main()

Run hillclimb plot to draw the solution in your working directory. For the Heilbronn problem that is the 13 points, their convex hull, and the smallest triangle they span, the one the score is.

hillclimb plot
heilbronn-convex-13 — normalized-min-triangle-area (higher is better) · 24 candidates · best c022=0.03093721
solution
climb

Example problems#

hillclimb comes with example problems to try it on, each with a verifier and a best known score to climb towards.

See all example problems

More visualisations#

Besides running the search, hillclimb comes with views for following a run while it climbs and for digging into what it found afterwards. All of the views run directly inside your terminal.

01Explore the search tree

View one search as its search tree side-by-side to its hillclimb chart. Every candidate built out from the root node of the tree coloured by its score, marked by "expanded", "left" or "failed" and the best one's lineage drawn back to the start.

hillclimb treeclimb
tree
climb

press play to grow the search, or slide through it; hover a candidate for its score and operator

02See what the agents learned

The knowledge graph of problems, searches, techniques and claims acts as the memory that is continuously growing as searches finish. Agents in later searches can read from this memory, so they do not start from nothing.

hillclimb graph

drag rotate  ·  shift-drag pan  ·  scroll zoom  ·  click a type to hide  ·  slide to scrub time

03Candidate similarity

The candidate similarity plot helps understand how similar different candidates are to one another. Only very similar candidates can indicate that the search didn't explore the space of possible solutions well enough. There is no general distance metric between two different solution.py files, so multiple metrics exist (how it behaves, its code, how many edits away it is, etc) and the metrics are user customisable. Colour is score and the best one's lineage is traced from the corner, so you can see how its ideas developed.

hillclimb similarity

drag to turn the box  ·  shift-drag to pan  ·  scroll to zoom  ·  double-click to reset  ·  hover a candidate for where it sits  ·  play to watch the search grow

Form factor#

hillclimb is a CLI that drives coding agents in headless mode. You run it from the terminal, or let a top-level agent like Claude Code or Codex run it for you: hillclimb run starts the search in the background and hands the terminal straight back. Each step of the search is one headless subagent session (Claude Code, Codex or OpenRouter), scored through your verifier.

You or your agent call the hillclimb CLI, which runs the search, scores every candidate through the verifier, journals it and keeps the best. It executes each step by running one headless coding agent such as Claude Code or Codex. you, or your top-level agent hillclimb CLI runs the search, scores through the verifier, journals every candidate, keeps the best claude code codex … headless coding agents as operators

Modular interface#

There are two main modules in hillclimb: the problem you want solved, and the climber that decides how to search for a solution. Write your own of either, or start from a built-in one.

01Your problem

A folder with a verifier that runs a solution and writes its score. Start from a built-in one and edit it. How to create a problem

hillclimb problem get circle-packing
problems/circle-packing/
├── problem.yaml      # the metric, and whether higher is better
├── description.md    # what the coding agent is asked to do
└── verifier.sh       # runs a solution and writes its score

02Your climber

A climber is five modules, run in this order at every step of a search. Each one is a built-in or a Python file you can edit. How to create a climber

ModuleDecidesBuilt in
selector_policywhich candidate the next attempt builds onbest, map-elites
operator_policywhich operator to apply to itgreedy
operatorshow one attempt is made: the prompt the coding agent getsdraft, debug, improve, ensemble
tunerwhich parameter values to try on a candidaterandom, optuna
memorywhat carries over from one search to the nextfiles, none

Put one together in Python, your own modules next to built-in ones, and run it with the CLI:

climbers/my_climber.py

from hillclimb import Climber
from hillclimb.memory import FilesMemory
from hillclimb.operators import Debug, Draft, Ensemble, Improve
from hillclimb.sdk import OperatorPolicy, SelectorPolicy
from hillclimb.tuners import RandomSearch


class MySelectorPolicy(SelectorPolicy):
    ...                                 # yours: which candidate to build on


class MyOperatorPolicy(OperatorPolicy):
    ...                                 # yours: which operator to apply to it


climber = Climber(
    selector_policy=MySelectorPolicy(),
    operator_policy=MyOperatorPolicy(),
    operators=[Draft(), Debug(), Improve(), Ensemble()],
    tuner=RandomSearch(),
    memory=FilesMemory(),
)
hillclimb run circle-packing --climber climbers/my_climber.py

03Compare climbers

An experiment runs your climber next to a built-in one on the same problem, several times each, and tells you whether the difference is bigger than chance. How to run an experiment

experiments/mine-vs-greedy.yaml

problems: [circle-packing]
repeats: 3
budget: 30m
noise_floor: 0.01
experiments:
  greedy:                                 # the control
    climber: greedy
  mine:                                   # yours
    climber: climbers/my_climber.py
hillclimb experiment run mine-vs-greedy hillclimb experiment report mine-vs-greedy

Issues#

Found a bug or want to suggest an improvement? Create an issue on GitHub.

Licence#

hillclimb is free (free both as in freedom and "free beer") software under the MIT License.