AWSAWS Certified Generative AI Developer - Professional2026-101.6 Implement prompt engineering strategies and governance for FM interactions

Refine prompt structure from measured failures

Completed by the knowledge check

Learning objective

Refine prompts with structured inputs, output specifications, chain-of-thought instruction patterns, and measured feedback.

Core idea

Prompt refinement changes a hypothesis about why the model fails. If the model confuses evidence with instructions, improve input separation. If it returns the wrong shape, specify the output contract. If it skips a necessary comparison, test a reasoning instruction. Retain the change only when measured outcomes improve under the application’s acceptance criteria.

For an architect, the useful unit is a controlled comparison between prompt configurations, not a collection of persuasive wording. Keep the model and test inputs fixed when measuring an instruction change. If you also change the model, treat that as a separate experiment or an explicit model-migration comparison.

AWS describes chain-of-thought reasoning as step-by-step reasoning prompted toward an answer. That is an instruction pattern, not an independent validator. A detailed explanation can accompany an incorrect answer. Score the answer and its supporting evidence rather than assuming that more reasoning text means better reasoning.

How it works

Make input ownership explicit

AWS prompt templates illustrate task instructions, reference text, explicit output formats, and examples. Structure those parts so that the reader and the model can distinguish them. A parameterized template might name the request, approved evidence, product version, and required decision separately. Keep instruction ownership and Converse message structure aligned with 1.3.6.

Use consistent labels and delimiters for supplied data, but do not describe delimiters as an authorization boundary. Text enclosed in a document section can still contain instructions that compete with the task. Retain the independently enforced controls from 1.6.1 and the adversarial tests from 1.6.6.

Specify how missing or conflicting input changes the answer. If the product version is absent, require clarification. If the approved evidence does not support the claim, require an explicit unsupported outcome. A template that only describes the happy path invites the model to fill gaps with plausible content.

Examples can clarify the intended mapping from input to response. Choose examples that demonstrate the difficult distinctions: evidence versus inference, supported versus unsupported answers, and clarification versus refusal. Keep examples consistent with the written rule. Conflicting examples make it unclear which behavior the test suite should reward.

Specify output semantics and shape

An output specification should describe fields, permitted categories, evidence requirements, and fallback behavior. “Return JSON” describes serialization but leaves the meaning of the values undefined. Define whether an absent fact becomes null, a clarification request, or an unsupported result, and make those choices part of evaluation.

The Bedrock structured outputs feature provides a stronger shape mechanism for supported models and interfaces. With Converse, outputConfig.textFormat selects the JSON Schema output format. Verify model support and the supported schema subset before choosing this mechanism. This is separate from telling the model in prose to produce a particular format; see 1.6.1 for the enforcement boundary.

Do not assume every business constraint belongs in the schema. The documented subset excludes numerical constraints such as minimum and maximum, and string constraints such as minLength and maxLength. Validate those requirements in application logic. A schema can require an evidence identifier without establishing that the identifier points to a real supporting source.

Test reasoning patterns as candidates

For a task that requires several dependent decisions, test an instruction that separates the subproblems: identify relevant facts, compare them with the rule, and produce the final decision. AWS prompt guidance includes step-by-step instruction examples. Treat the pattern as a candidate whose value depends on the selected model and task.

Separate internal problem solving from the user-facing output contract. The application might need a concise decision and a short evidence-based justification rather than a long reasoning transcript. Explicitly specify that final format and test whether the instruction causes extra prose or exposes information the interface should not return.

Compare a direct-answer baseline, a structured checklist instruction, and a step-by-step candidate against the same cases. Measure final correctness, unsupported assertions, required-format compliance, response length, and end-to-end latency. A candidate that improves complex decisions but adds cost to simple classifications might belong only on the complex path.

Close the loop with measured feedback

Amazon Bedrock distinguishes simple prompt optimization from Advanced Prompt Optimization. Simple optimization rewrites a single prompt without evaluation. Advanced optimization uses an evaluation feedback loop and returns optimized templates with evaluation scores, cost estimates, and latency. Do not call a rewritten prompt validated merely because the optimizer produced it.

Advanced optimization accepts up to 10 prompt templates and 100 evaluation samples per template, with up to 5 models compared in a job. Its input JSONL uses promptTemplate and evaluationSamples; samples supply inputVariables and can include referenceResponse. These limits bound the optimization dataset, not the size of your independent regression suite.

Select one evaluation method per template: steeringCriteria, customLLMJConfig, or evaluationMetricLambdaArn, or omit the optional evaluation fields for the default. Use natural-language criteria for understandable goals, a custom judge rubric for semantic comparisons, and a Lambda evaluator for computable requirements. When using customLLMJConfig or evaluationMetricLambdaArn, supply the required customEvaluationMetricLabel. A missing label, or combining steeringCriteria with either custom evaluator, returns ValidationException. The Lambda score must follow a higher-is-better scale. Verify it penalizes malformed and clearly incorrect outputs before using it to steer optimization.

Review failures by category, change one relevant instruction, and rerun the comparison. Keep held-out cases that the optimizer has not seen, as AWS recommends. Feed recurring failures back into the regression suite in 1.6.6; promote an accepted candidate through 1.6.3 rather than editing the production reference during experimentation.

Key distinctions and trade-offs

Structured inputs versus trusted inputs: Labels and delimiters improve interpretation. They do not establish that supplied content is accurate, authorized, or safe.

Output instruction versus schema enforcement: Prompt text expresses the intended response. The structured outputs feature constrains supported response shapes. Application validation still checks business meaning.

Reasoning instruction versus correctness evidence: A step-by-step response is generated text. Correctness requires comparison with trusted facts or an independent computation.

Heuristic rewrite versus evaluation-driven refinement: Simple optimization supplies candidate wording. Advanced optimization steers changes with scores. Both require independent acceptance evidence before release.

Architecture / decision scenario

A procurement assistant compares contract terms with an approved purchasing policy. Its baseline mixes the request and contract excerpts, returns inconsistent categories, and overlooks exceptions. The team separates labeled inputs and specifies a decision, supporting clause identifiers, and a missing-evidence outcome.

The team compares a direct instruction with a checklist and a step-by-step candidate. The structured outputs feature constrains the supported model’s response shape; application checks reject nonexistent clause identifiers. A Lambda evaluator scores the decision and required fields, while a reviewed judge rubric scores whether the explanation follows the evidence.

Advanced optimization improves the development cases, but the held-out set exposes an exception the candidate misapplies. The team revises that rule and retests all candidates. It retains the configuration that improves verified decisions within the latency budget, without rewarding extra reasoning text. The chosen template then enters the existing release gate.

Common exam traps

Treating input delimiters as a security control.
Assuming “return JSON” enforces a machine-readable contract.
Equating a longer chain-of-thought response with a correct decision.
Changing the model and prompt while attributing all gains to wording.
Treating simple prompt optimization as a scored validation process.
Using optimization examples as the only acceptance dataset.

What to remember

Refine the part of the prompt implicated by measured failures.
Define input roles, output semantics, and missing-evidence behavior.
Test reasoning instructions against final correctness and operating cost.
Choose a concrete evaluation mechanism that matches the requirement.
Confirm gains on held-out cases before promoting the candidate.