🔧 Optuna Flashcards
Automated Hyperparameter Optimization Framework
🚀 What is Optuna?
Optuna is an automatic hyperparameter optimization framework designed for performance, flexibility, and ease of use.
📦 Installation
Install via pip install optuna. Also supports LightGBM, XGBoost, PyTorch, TensorFlow, and more.
🔁 Define & Optimize
Use study.optimize() with an objective function to search for best hyperparameters automatically.
⚙️ Search Spaces
Define parameter ranges with trial.suggest_float, suggest_int, suggest_categorical, etc.
📊 Visualizations
Optuna provides built-in plots like optimization history, parameter importance, slice plots, and more.
🎯 Objective Function
The core function that returns a score (e.g., accuracy, RMSE) based on parameters sampled by Optuna.
⚡ Pruning
Stop unpromising trials early using built-in pruning logic to save resources.
🧠 Samplers
Choose from various samplers like TPE (Tree-structured Parzen Estimator), CMA-ES, GridSearch, etc.
🧪 Multi-Objective Optimization
Supports optimizing for multiple objectives simultaneously, using Pareto front analysis.
💾 Storage
Persist trials and studies to SQLite, PostgreSQL, or RDBs for sharing or continuation.
🔗 Integration
Works seamlessly with Scikit-learn, LightGBM, PyTorch Lightning, Ray Tune, Keras Tuner, and others.
📂 Study Management
Create, load, or resume studies with optuna.create_study() or optuna.load_study().