3. Optimization Algorithms: Differences and Use Cases

3.1. Why Learn About Different Algorithms

Parameter spaces and compute costs vary widely: grid search is reliable in small spaces but explodes combinatorially; heuristics or Bayesian methods trade accuracy for speed under a limited evaluation budget; algorithms also differ in support for continuous, discrete, or mixed parameters.

3.2. Comparison of Optimization Algorithms (Listed with Differences)

Algorithm	Brief Principle	Pros and Limitations	Typical Parameter Scale	Continuous / Discrete / Mixed	Typical Compute Cost
grid	Exhaustive search or traversal on grid points	Simple and reproducible; impractical in high dimensions	Small (2–3 dimensions)	Mostly discrete	Grows linearly with grid size
montecarlo	Random sampling of parameter combinations	Simple and easy to parallelize; needs many samples for stability	Medium to large	All supported	Determined by sample_count
GA	Selection, crossover, mutation	Suited to non-convex, multi-modal problems; hyperparameter sensitive	Medium to large	All supported	Determined by generations and population size
SA	Simulated annealing; probabilistically accepts worse solutions	Can escape local optima; slower convergence	Medium	All supported	Determined by iteration count
PSO	Particle swarm position and velocity updates	Strong in continuous spaces; discrete params need encoding	Medium to large	Mostly continuous	Determined by particle count and iterations
bayesian	Surrogate model + acquisition function	Efficient with few samples; costly in high dimensions	Small to medium	Mostly continuous	Determined by iterations and model fitting

3.3. Use Cases and Selection Advice

• Few discrete parameters: Prefer grid or small-scale montecarlo.

• Many parameters, few evaluation steps: Consider bayesian or GA/PSO.

• Continuous, multi-modal space: PSO, GA, or SA.

• opti_sample_count and parameter count: With too few samples or iterations, heuristics may stop early; increase opti_sample_count or reduce parameter dimensionality as needed.

3.4. Quick Comparison Table

See the table above. In practice, follow the opti_method values supported by the current qteasy release and its docs; note each algorithm’s typical settings (e.g., GA population size and mutation rate) and their configuration names and defaults.