Optuna

The hyperparameter-optimization framework that finds the right configuration in fewer trials.

Optuna defines the search space on the fly (define-by-run): each hyperparameter is sampled inside the objective function itself, which allows conditional and nested spaces. Its Bayesian TPE sampler proposes the next trials from previous results, its pruners (median, Hyperband) stop unpromising trials early, and studies run in parallel across machines through shared storage. Framework-agnostic, from boosting to PyTorch, it's our choice to replace grid search as soon as each training run costs more than a few minutes.