AI Bug Fixing: From Finding to Fix Pull Request
Give QA a path from finding to repair. Give engineering a concrete fix to review.
A temporary read failure should not erase an agent’s saved state. In a Pydantic AI persistence path, that was possible.
Early identified the bug, then generated a draft Fix PR with a regression test after user initiation. The result was a repair proposal engineers could inspect alongside the original finding.
A detailed bug report still leaves someone to investigate the implementation, prepare a patch, add a test, and get the change reviewed. Engineers may already use coding agents for that work. The question is who has to drive the repair from the beginning, and what arrives ready for review.
AI bug fixing uses AI to implement a change that addresses a defect. In a repository workflow, automated bug fixing carries a finding into a code change and tests for engineers to review in a pull request. For QA leaders, that means a way to move an issue beyond the report. For engineering leaders, it means less repair preparation competing with feature work.
A Read Failure Became a Destructive Write
Pydantic AI is an open-source framework for building AI agents. Its CodeMode can retain a conversation’s Python variables, imports, and functions between runs.
Early found a failure in that persistence path. If loading the saved state failed, the agent continued with a fresh session. If saving then succeeded, that fresh session could overwrite the conversation’s previously valid state. A temporary read problem could therefore erase information that was still safely stored.
The distinction matters. Continuing after a read failure can be a deliberate fallback. Writing over the state that could not be read is a separate consequence. A run can finish while leaving the next run without the variables it expects.
We reported this finding to Pydantic AI. Early’s finding, Failed load still overwrites saved REPL, gives the team a specific failure and a reason to act on it.
Real-world Case File: Pydantic AI
See the actual finding, repair initiation, and Early-generated draft Fix PR.
QA Can Move the Finding Forward
QA brings the behavioral question into focus: should a temporary read failure be allowed to destroy the last valid saved state? That is a useful starting point for repair because it describes what needs to remain true, not just where the code looks suspicious.
In the recorded product workflow, the user selects the finding, confirms the destination repository and target branch, and initiates the Fix PR. Early then prepares the change and test on the fork. Engineering receives a concrete proposal rather than another description of the problem.
Actual product capture. The user confirms the repair destination before Early creates the proposed PR. Engineering controls the merge.
That is the Empowered QA opportunity: carry a finding toward repair without having to write the patch and tests. Teams use the repair permissions configured for their workflow. Engineers still own review and merge.
What the Fix PR Contained
The Early-generated draft Fix PR #3 proposes a targeted change. It records when loading failed and skips the ordinary save afterward, preserving what the store already held. An explicit restart remains an exception because clearing the session is then deliberate.
The PR also adds a test for the affected behavior. Its scenario saves a variable, introduces a load failure, checks that the stored state was not replaced, and then checks that a later run can recover the variable. It updates the persistence documentation alongside the implementation.
The proposal changes four files: the implementation, test, and two documentation files. The value for engineering is not the line count. It is receiving the failure, proposed correction, and test together, with enough context to review their relationship.
Tests added during repair can help catch the same failure if it returns. That depends on whether they assert the right behavior and are subsequently run and maintained. A generated test is a useful part of the proposal, not a result by itself.
Where Engineering Judgment Matters
Engineering reviews whether the proposed repair expresses the right policy. In this case, preserving the old state avoids one kind of data loss, but means the fallback run’s new work is not persisted. An explicit restart needs its own clear behavior and coverage.
Those are review questions, not reasons to start the entire repair from scratch. Engineers can examine the proposed guard, amend it, expand the tests, and decide whether the change belongs in the project.
Pydantic AI’s triage bot makes the policy choice visible: preserve the previous state after a failed load, or keep saving the fallback session. It recommends preservation but asks for maintainer discussion. An AI-generated patch does not remove that ownership decision.
For a QA leader, this is a more useful handoff than a ticket that loses context between teams. For an engineering manager, it directs attention toward correctness, compatibility, and acceptance rather than requiring an engineer to prepare every part of the repair.
What Was Actually Verified
The fork PR reports a passing Python compilation check. Its behavioral test was added but not executed in the available record. Pydantic AI’s triage bot separately reports reproducing the original state-loss failure, not validating the proposed repair.
The next steps are to execute the proposed repair’s behavioral checks, add coverage for the explicit-restart exception, and complete engineering review. PR #3 remains a draft on Early’s fork, with those steps still open.
That leaves a useful artifact for review while making its current state clear. Finding the bug, preparing the patch, running the tests, and accepting the change are different pieces of work. The AI code verification guide explains how to evaluate the execution evidence in more detail.
Evaluate the Repair Handoff
For QA and engineering leaders evaluating AI bug fixing, the question is not simply how many findings a tool produces. Ask what the team receives and what work remains:
- A credible finding: Can the team understand the failure, its consequence, and the expected behavior?
- A usable proposal: Does the patch address the cause without mixing in unrelated changes?
- Relevant tests: Do they check the failure and important surrounding behavior?
- Visible results: Which checks actually ran, against which revision, and what happened?
- Engineering control: Can reviewers amend, reject, or merge the proposal through their normal process?
Follow a small set of findings through that whole sequence. Track accepted, reworked, rejected, and still-open proposals, along with the engineering effort needed to reach a useful outcome. Finding more bugs can increase the number of repairs. The aim is to reduce the effort people spend driving each repair, not to promise fewer bugs will be reported.
More Capacity for Building
The opportunity is to shorten the path from a finding to a useful repair. Measure the time spent preparing, reviewing, and reworking proposals, not just how quickly a tool opens a PR.
The practical change is still important. QA has a path from a meaningful finding into an authorized repair workflow. Engineering gets a proposed fix and tests to review, while keeping control of what changes and what ships.
That is the role of Early Release Guard: find bugs, create proposed Fix PRs after authorized initiation, and make the available verification evidence visible. The workflow supports engineering judgment rather than removing it.
For the visual product walkthrough, explore the Pydantic AI finding-to-fix Case File. For the review itself, use these AI code review practices.
Questions About AI Bug Fixing
What is AI bug fixing?
AI bug fixing uses AI to implement a code change that addresses a defect. A useful workflow carries the finding into a proposed fix, tests, and engineering review.
How can QA drive AI bug fixing?
QA establishes the failure and expected behavior, then moves an eligible finding into an authorized repair workflow. Early prepares the proposed fix and tests. Engineers review the change and control the merge.
How do you verify an AI-generated fix?
Reproduce the failure, run the relevant checks against the proposed revision, and confirm the important surrounding behavior still works. Review the patch and test results before deciding whether to merge.
