Generate CI tests from standards, fixtures, and coverage

Learning objective

Provide project standards, available fixtures, and existing test files so CI test generation avoids duplicates.

Core idea

Test generation needs both a quality standard and evidence of existing coverage. Project instructions define how the team writes tests and which cases deserve protection. Existing test files show which behaviors already have that protection. Fixture definitions show how to exercise missing behavior within the team’s test environment.

A prompt that says “add useful tests without duplicates” supplies none of those facts. Give Claude Code the concrete inputs, then ask it to explain the coverage gap before proposing a case. The unit of duplication is the behavior and assertion, not the test function’s name.

This lesson concerns CI-generated tests for a developer-productivity workflow. See 3.6.3 for making instructions available in ordinary or bare print runs, and 3.6.5 for independently reviewing generated changes.

How it works

Put standing test criteria in project instructions

Anthropic documents CLAUDE.md as the place for project-specific commands, conventions, and broad instructions. A project file can live at ./CLAUDE.md or ./.claude/CLAUDE.md. For test generation, name the framework, test locations, commands, and conventions the model cannot safely infer.

Useful entries are concrete: the focused test command, how the repository isolates database state, where fixtures live, and which behaviors justify a test. Avoid “write thorough tests” when the team means “assert the externally visible result of the changed authorization branch using the existing authenticated-client fixture.”

Define valuable-test criteria as repository guidance, not as universal Claude behavior. For example, a team might require a regression case that distinguishes the changed implementation from the previous defect, reject assertions that merely mirror implementation details, and prefer an existing fixture over a second setup path. Claude Code does not supply those project-specific criteria automatically.

Keep the instruction file concise. Anthropic recommends targeting fewer than 200 lines per CLAUDE.md file because longer instruction files consume context and reduce adherence. Put broadly applicable standards there; provide PR-specific code and coverage evidence with the assignment. See 3.1.2 and 3.3.1 for organizing larger instruction sets without repeating their loading mechanisms here.

Provide fixtures as usable contracts

Listing a fixture’s name helps, but the generator also needs its relevant behavior. Give the path to its definition and explain what state it creates, what it cleans up, and whether it supplies a privileged or ordinary user. If those facts are available in the repository, let the assignment read the actual definition rather than invent setup from its name.

For example, a Python service might have tests/conftest.py defining db_session and customer_client. Those are example repository identifiers, not Anthropic-provided fixtures. A generator writing a checkout regression should inspect them and reuse the one that represents the required actor and transaction lifecycle.

The mechanism is ordinary Claude Code file inspection through Read, with the job’s permitted tools and project instructions supplying scope. No fixture-discovery flag replaces this evidence. If the generator cannot read a fixture definition because the pipeline did not check it out or allow access, it lacks the setup contract. Repair the input rather than asking for more confident output.

Supply existing tests for the changed behavior

Anthropic’s testing workflow says Claude examines existing test files to follow their style, framework, and assertion patterns. Direct the assignment to the relevant tests and source explicitly. This improves coverage selection as well as stylistic consistency: it gives the generator evidence of which conditions and outcomes already have assertions.

A filename alone is not the contents of a test. The job can let Claude read named paths with Read, or include selected file contents through print-mode standard input. Piped stdin is capped at 10 MB; above that cap, Claude Code exits with an error and non-zero status. For larger inputs, the documentation recommends writing content to files and referencing their paths rather than piping it.

Do not put the entire suite into every run merely because more context is available. Name tests relevant to the changed behavior and its dependencies. If fixture use or a wrapper makes coverage unclear, ask Claude to inspect that specific evidence before recommending another case. A scoped assignment should say which areas to investigate and what counts as a missing case.

Ask for a coverage comparison before edits

The request should separate three questions: what changed, what the suite already asserts, and what meaningful scenario is missing. A compact assignment could say:

Inspect src/orders.py, tests/test_orders.py, and tests/conftest.py.
Apply the project's testing standards. For each proposed test, identify
an uncovered behavior and the existing fixture it uses. Do not add a
case already asserted by the relevant suite. If coverage is sufficient,
report that conclusion instead of adding tests to meet a quota.

These paths describe an example Python repository. They are evidence locations the pipeline must actually provide. The instruction requests a comparison, rather than prescribing a fabricated “deduplicate tests” CLI option.

After selecting a meaningful case, run the repository’s documented focused command through an allowed Bash rule. For example, if the team documents pytest tests/test_orders.py, pre-approve that operation under the policy in 3.6.1. Use the actual project command, not a remembered runner from another repository.

A passing generated test confirms execution of that case; it does not prove the test adds coverage. Review the proposed condition and assertion against the existing suite. Anthropic’s best-practices guidance recommends giving Claude verification criteria and ways to check its work. Here those criteria are a real coverage gap, correct fixture use, and the documented test command.

Key distinctions and trade-offs

The inputs have different roles:

Copying complete standing standards into every workflow can create divergent instructions. Keep them in the repository and use the selected loading contract. Conversely, keeping only standards without supplying the changed source and existing tests leaves the generator unable to compare coverage.

Duplication is not always a textual match. Two tests can have different names and identical behavioral assertions. Two tests with similar setup can protect distinct boundaries. Make the comparison about the behavior the team values, using source and test evidence instead of name similarity.

Architecture / decision scenario

A checkout PR changes how cancelled orders are handled. CI generation repeatedly suggests an empty-basket test that already exists. The team checks the prompt and finds that it contains the diff but neither tests/test_orders.py nor the fixture definitions.

The repaired job loads project standards and names the relevant existing test files and tests/conftest.py in the assignment. Claude compares the existing assertions with the changed cancellation path, reuses the documented customer fixture, and proposes only an uncovered cancellation outcome. The job runs the project’s focused test command through its explicit Bash grant. If that outcome is already asserted, the correct result is no additional test, not a renamed duplicate.

Common exam traps

• Asking for no duplicates without providing existing tests.
• Treating test names as proof of distinct coverage.
• Naming fixtures without supplying their definitions or contracts.
• Replacing precise standards with “write high-quality tests.”
• Assuming a passed test necessarily adds meaningful coverage.
• Copying standards into every prompt while omitting PR-specific evidence.

What to remember

Give test generation standing criteria, usable fixture definitions, existing tests, and the changed behavior. CLAUDE.md carries broad conventions; Read or supplied file contents carry coverage evidence. Ask for a specific missing assertion and verify it with the documented runner. A completed comparison can correctly conclude that no additional case is needed.

Knowledge check

References

• Anthropic, How Claude remembers your project
• Anthropic, Common workflows
• Anthropic, Best practices for Claude Code
• Anthropic, Run Claude Code programmatically