hyperband.hpp File Reference

Hyperband (Li, Jamieson, DeSalvo, Rostamizadeh, Talwalkar, JMLR 2018): runs several Successive Halving "brackets" with different (num_configs, initial_budget) trade-offs – covering the tension between "few configs trained long" and "many configs trained short, halved down" that a single Successive Halving run commits to in advance – and returns the best result across every bracket. More...

#include <cmath>
#include <limits>
#include <stdexcept>
#include <vector>
#include "pulsatrix/successive_halving.hpp"

Include dependency graph for hyperband.hpp:

Go to the source code of this file.

Classes
struct	pulsatrix::HyperbandBracket
	One bracket's own (num_configs, initial_budget) trade-off point; s is the bracket index (s_max = most configs/smallest budget, down to s=0 = fewest configs/largest budget, matching the original paper's own naming). More...
struct	pulsatrix::HyperbandResult
	The best configuration/metric found across every bracket, and the total epoch budget spent summed across all of them. More...

Namespaces
namespace	pulsatrix

Functions
std::vector< HyperbandBracket >	pulsatrix::ComputeHyperbandBrackets (int max_resource, double eta)
	Computes the classic Hyperband bracket schedule: s_max = floor(log_eta(max_resource)), B = (s_max + 1) * max_resource; for s from s_max down to 0, num_configs = ceil((B / max_resource) * (eta^s / (s + 1))), initial_budget = round(max_resource / eta^s) (each floored at 1).
template<typename RNG >
HyperbandResult	pulsatrix::RunHyperband (const SearchSpace &space, const TrialFactory &make_trial, int max_resource, double eta, RNG &rng)
	Runs one Successive Halving bracket (successive_halving.hpp) per ComputeHyperbandBrackets(max_resource, eta), keeping the best result across all of them.

Detailed Description

Hyperband (Li, Jamieson, DeSalvo, Rostamizadeh, Talwalkar, JMLR 2018): runs several Successive Halving "brackets" with different (num_configs, initial_budget) trade-offs – covering the tension between "few configs trained long" and "many configs trained short, halved down" that a single Successive Halving run commits to in advance – and returns the best result across every bracket.

Note

Scope simplification, logged rather than silently assumed: the original paper's inner Successive Halving loop runs exactly (s+1) rounds per bracket, ending at precisely max_resource on its final rung (some brackets may finish with more than one surviving candidate at that point). This implementation instead runs each bracket via successive_halving.hpp's own RunSuccessiveHalving, which always halves down to exactly one survivor regardless of a resource cap – for larger brackets (small s), this can train the final survivor somewhat past max_resource rather than stopping exactly there. The core idea Hyperband is actually named for – explore the num_configs/initial_budget trade-off via multiple brackets, keep the best result across all of them – is preserved exactly; only the inner loop's precise stopping point differs from the original paper's.