successive_halving.hpp File Reference

Successive Halving (the rung-based promotion mechanism underlying Hyperband and ASHA, Li, Jamieson, DeSalvo, Rostamizadeh, Talwalkar, "Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization," JMLR 2018): train a population of configurations for a small budget, keep the best fraction, repeat with a larger budget, until one survives. More...

#include <algorithm>
#include <functional>
#include <limits>
#include <memory>
#include <stdexcept>
#include <vector>
#include "pulsatrix/hpo_sampling.hpp"
#include "pulsatrix/search_space.hpp"

Include dependency graph for successive_halving.hpp:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	pulsatrix::ResumableTrial
	A single hyperparameter configuration's live, resumable training state – own whatever network/optimizer/dataset a concrete trial needs, and train it incrementally across multiple calls rather than all at once. More...
struct	pulsatrix::SuccessiveHalvingResult
	The winning configuration, its final metric, and the total epoch-budget actually spent across every trial/rung (the exit-gate's own "reduces total training compute" measure). More...

Namespaces
namespace	pulsatrix

Typedefs
using	pulsatrix::TrialFactory = std::function< std::unique_ptr< ResumableTrial >(const Configuration &)>
	Builds a fresh ResumableTrial for a given configuration.

Functions
SuccessiveHalvingResult	pulsatrix::RunSuccessiveHalvingOnConfigs (std::vector< Configuration > configs, const TrialFactory &make_trial, int initial_epoch_budget, double eta)
	Runs Successive Halving over an explicit, caller-supplied list of configurations – the pure, deterministic core; RunSuccessiveHalving (below) is the thin SearchSpace/RNG-sampling wrapper around it.
template<typename RNG >
SuccessiveHalvingResult	pulsatrix::RunSuccessiveHalving (const SearchSpace &space, const TrialFactory &make_trial, size_t num_configs, int initial_epoch_budget, double eta, RNG &rng)
	RNG-driven wrapper: draws num_configs configurations from space via RandomSample, then runs RunSuccessiveHalvingOnConfigs.

Detailed Description

Successive Halving (the rung-based promotion mechanism underlying Hyperband and ASHA, Li, Jamieson, DeSalvo, Rostamizadeh, Talwalkar, "Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization," JMLR 2018): train a population of configurations for a small budget, keep the best fraction, repeat with a larger budget, until one survives.

Note

Decision Point 4 resolution (campaign_exai_dl_library_hyperparameter_optimization, Phase 3 activation, 2026-09-27): recon (Phase 1 Mission 0) already confirmed MetricsSink is observation-only (log_scalar/log_histogram) – no mid-training external stop-signal hook exists. Rather than adding one to MetricsSink itself (a core interface every module/training loop depends on), the resolution is a new, purely-additive HPO-side abstraction: ResumableTrial, below. Every training loop in this codebase (XorNetwork::train_step and friends) is already an ordinary, synchronous C++ function the caller loops over – "stopping early" needs no new capability from the network/optimizer/MetricsSink at all, just a caller that owns the trial's live state across multiple short training calls instead of one long one, and simply chooses not to call it again for pruned configurations. Zero changes to Module/Tensor/DeviceBackend/MetricsSink.

This interface is scoped to what Successive Halving/Hyperband/ASHA (Phase 3) need – train for N more epochs, report the current metric. The sibling campaign_exai_dl_library_evolutionary_deep_learning's Population Based Training mission is documented (this campaign's own Phase 3 exit gate section) as consuming this same "mid-training hook," but PBT also needs to read/replace a live trial's weights (its exploit/explore step) – not assumed here. Per this project's own precedent (RL's Agent interface was extended per-consumer, e.g. DQNAgent's epsilon-greedy methods, CategoricalPolicyAgent's log-probability accessor, rather than speculatively upfront), that campaign's own PBT mission must re-verify by recon whether this interface suffices or needs extending – not guessed here ahead of need.