Model Selection and Parameter Optimization

Why it matters

Model and parameter choices are measurements, not preferences. The optimizer turns them into a search over a declared space, scored by the same evaluators you already trust.

What to practise

Mark optimizable_params such as the model name and temperature on an LLM, give each a search_space, and add an optimizer section: reps_per_param_set, and eval_metrics that map names to evaluators with a direction and weight for accuracy, token efficiency and latency. Run nat optimize with sampler: grid on a chat workflow, then on the Alert Triage tool-calling agent, evaluating that agent with nat eval before and after. Finally turn on prompt optimization with ga_generations and ga_population_size to evolve the agent prompt. Read trials_dataframe_params.csv, the Pareto plots and optimized_config.yml, and watch how many workflow runs repetitions multiply into.

How it fits

This is 3.4.3 end to end: a search over models, numeric values and prompts, and a measured candidate to deploy. Use 3.4.1 to decide which sampling ranges are worth searching, and 3.4.2 to check that a winning model fits your latency and GPU constraints.