Profiling and Performance Monitoring of NVIDIA NeMo Agent Toolkit Workflows

Why it matters

Choosing a model or a cache strategy for an agent workflow is guesswork without per-call timings and token counts. The profiler turns an evaluation run into that evidence.

What to practise

Add a profiler block under eval.general with compute_llm_metrics, prompt_caching_prefixes, bottleneck_analysis.enable_nested_stack and concurrency_spike_analysis, then run nat eval. Open all_requests_profiler_traces.json and standardized_data_all.csv, and follow the walkthrough that compares several models on the email phishing analyzer. Read the nested-stack Gantt charts to see whether a model’s time goes to its first reasoning call or to later steps. Optionally, decorate your own code with @track_function.

How it fits

The profiler’s findings feed 7.3.3’s bounded workflow changes, such as enabling prompt caching or capping max_tokens. Swapping the model under comparison is 7.3.1’s point: change the llms entry, not the workflow.