Gate training data and retraining decisions

Learning objective

Govern fine-tuning data quality, filtering, evaluation, and retraining cadence.

Core idea

A fine-tuning dataset is a specification of behavior expressed as examples or feedback. If it contains contradictory labels, stale decisions, sensitive material, or duplicated cases, training can reinforce those defects. Governance therefore starts before upload and continues after a trained model is deployed.

For an architect, the task is to define which records are accepted, which evidence justifies promotion, and which changes justify another training run. Microsoft Foundry provides file processing, training jobs, metrics, and checkpoints. You still need an application-owned acceptance policy and independent task evaluation. Method selection belongs to 2.1.6; this lesson owns the lifecycle of the selected method’s data and results.

How it works

Accept records against a defined contract

Define the role, intended behavior, and coverage categories before collecting examples. A support classifier needs representative categories and ambiguous cases, rather than thousands of near-identical successful requests. Include cases where the correct behavior is to request clarification or decline an unsupported conclusion. Reviewers need one rubric so a record’s acceptance does not depend on who happens to label it.

Foundry’s fine-tuning overview calls for accurate, consistent, representative data and inspection of missing scenarios, duplicates, bias, and sensitive information before upload. Implement these checks in your dataset preparation pipeline. The cited training interfaces do not turn an arbitrary conversation archive into a vetted training set. Record a rejected example’s reason and retain enough provenance to explain why an accepted label is trustworthy.

Filtering has two jobs. First, exclude records whose source, permissions, or content make them unsuitable for training. Second, repair or reject records that teach the wrong behavior: incomplete targets, inconsistent classifications, accidental instruction overrides, or answers unsupported by the accompanying evidence. Removing names alone does not resolve a wrong label. Conversely, a correct answer does not authorize reuse of its source material.

Apply the selected method’s data contract. SFT JSONL holds a messages conversation with the desired assistant response. DPO uses input.messages, preferred_output, and non_preferred_output; both outputs answer the same input. RFT records contain a messages array ending with a user message and the reference fields needed by the grader. The RFT guide’s reference_answer is a sample reference field, not a requirement that every business task use that name.

These contracts govern syntax and training signal. They do not determine whether an SFT answer is correct, a DPO preference is consistent, or an RFT reference is trustworthy. Keep those semantic checks explicit and avoid silently converting one method’s dataset into another’s.

Split before uploading

Maintain separate training, validation, and final test datasets. Training examples drive the update; validation supports iteration and checkpoint comparison; the final test checks a selected candidate independently. Microsoft explicitly warns against evaluation leakage and judging improvement from training loss alone.

As an application design rule, split related cases together. A support ticket copied into several paraphrases should not appear in training and the final test through different wording. Track the case or source-group identifier before transformations, then deduplicate within and across the splits. Otherwise the final test measures recall of a near-duplicate rather than transfer to an unseen task. Preserve important coverage categories in each appropriate split without manufacturing identical examples in all three.

For the documented Python SFT workflow, client.files.create() uploads training and validation files with purpose="fine-tune". client.files.wait_for_processing() waits for each file to be processed. client.fine_tuning.jobs.create() receives their IDs through training_file and validation_file. Treat file processing as a service preparation check, not the semantic acceptance gate. Keep the final test out of both job inputs. Only the training and validation files enter the job; the final test checks the selected candidate independently.

Retain a dataset version, preprocessing version, rubric, base-model identifier, job configuration, and job ID in your experiment record. These are application-owned audit fields. A comparison is difficult to interpret if a prompt change, dataset correction, and base-model change all occur without a record of which candidate contains which change.

Evaluate the signal the method actually learns

For SFT, the documented metric identifiers are train_loss, full_valid_loss, train_mean_token_accuracy, and full_valid_mean_token_accuracy, when reported by the selected model. Falling training loss alongside rising full validation loss is a reason to inspect failing examples and overfitting. Token accuracy measures agreement with target tokens; it does not establish that the agent selects a safe tool or resolves a claim correctly.

DPO evaluation checks whether candidate responses satisfy the preference rubric on held-out examples. The DPO guide explicitly distinguishes that task evaluation from token-prediction accuracy. Examine factual correctness as well as preference adherence: a concise response can win a superficial preference while omitting a required exception.

RFT measures grader rewards. Its metric events expose train_reward_mean and valid_reward_mean under data.scores; per-grader scores are under data.scores.graders. Grader errors appear under data.errors.graders. Inspect parse failures, absent reference fields, and execution errors before interpreting a reward trend. For Python, validate the grader with client.fine_tuning.alpha.graders.validate(grader=grader), then use client.fine_tuning.alpha.graders.run with the grader, sample response, and dataset record to test correct, incorrect, malformed, and adversarial responses. This checks whether an exploitable scoring rule could become the behavior you train.

Select checkpoints through measured task results. client.fine_tuning.jobs.retrieve() exposes job status, errors, fine_tuned_model, and result_files; list_events() exposes progress; checkpoints.list() exposes available checkpoints. Creation and retention depend on the model and workflow. Do not assume a fixed checkpoint count or select the final checkpoint solely because it was produced last. When an RFT job provides linked automatic evaluations, use them to compare scores and failed responses across runs and checkpoints.

Key distinctions and trade-offs

Filtering versus safety evaluation: Dataset filtering controls what you train on. Foundry’s overview also calls for reviewing managed safety evaluation and retaining application-level safety controls. Neither replaces evaluation of the deployed serving runtime.

Validation versus final test: Repeatedly choosing settings against validation data makes that set part of the development process. Preserve an independent final test and inspect failures before promoting the selected model. After deployment, run that held-out comparison through Run evaluations from the Foundry portal, or use Cloud evaluation with the Microsoft Foundry SDK for an SDK-driven evaluation. These are the evaluation paths named by the SFT and RFT guides; you still own the test split and acceptance rubric.

Training success versus release readiness: A completed job proves that training completed. Job metrics and independent task results decide whether its behavior is acceptable. Use the promotion discipline from 2.1.4 for the candidate’s surrounding prompt and release configuration.

Better coverage can cost more labeling and training work. More epochs can increase overfitting. The SFT API names that setting n_epochs; the Azure Developer CLI configuration uses epochs. Keep model-supported defaults for the first run and change settings based on validation evidence, rather than increasing passes to compensate for poor labels.

Architecture / decision scenario

A claims-review role learns a stable categorization and explanation contract. The team filters its source cases for reuse permission, sensitive content, label consistency, and missing categories. It splits by claim identifier before creating paraphrases, then versions the accepted SFT dataset. Training and validation IDs enter the job; the final test remains separate.

Training loss improves, but full_valid_loss worsens on ambiguous claims. The team inspects those cases and compares available checkpoints instead of deploying the last one. For a DPO explanation candidate, it applies the same held-out preference rubric used by labelers. For an RFT candidate, it checks valid_reward_mean, grader errors, and independent claim correctness, rejecting an apparent improvement caused by a grader that rewards a label without testing the evidence.

The team sets a recurring review of production failures and dataset coverage using the monitoring and evaluation signals in 1.2.8, with retraining triggered by sustained category errors, a changed task contract, or an approaching base-model retirement. Microsoft documents reassessment and possible retraining, not a universal calendar interval. A policy-document update alone goes through grounding rather than automatically entering training. Each retraining run repeats acceptance, split, and release gates; an unsuccessful candidate leaves the accepted deployment in place.

Common exam traps

• Assuming processed JSONL proves label correctness or permission to train.
• Splitting after paraphrasing and leaking the same source case into evaluation.
• Treating validation data repeatedly used for selection as an untouched final test.
• Using token accuracy to measure DPO preference alignment or RFT reward quality.
• Promoting a checkpoint from reward alone without inspecting grader failures.
• Retraining on a fixed calendar without evidence that the learned behavior needs changing.

What to remember

Filter for suitability and correctness before upload, preserve independent evaluation splits, and inspect method-specific metrics alongside held-out task outcomes. Version datasets and experiment configuration, compare available checkpoints, and review retraining triggers against task changes and production evidence. A training job is one step in a governed release process.

Knowledge check

References

• Microsoft, Fine-tuning in Microsoft Foundry
• Microsoft, Fine-tune a model with supervised fine-tuning
• Microsoft, Fine-tune a model with direct preference optimization
• Microsoft, Reinforcement fine-tuning