Why Fixing One Bug in Your AI App Breaks Something Else
Use regression testing to check an AI code repair across pricing, reports and stored records. Preserve the original failure and verify the whole rule.
Nov 9, 2026 · 4 min read
Field Notes from Thunkle, a studio that takes AI-built apps from prototype to secure, production-ready software.
During a marketplace review, I found that checkout displayed a higher total than the payment provider charged. A payout calculation elsewhere used another interpretation of the booking amount. A repair needs regression testing across those callers, not just a new total on one screen.
Pricing decisions existed in several places. Asking an agent to “fix the total” would leave an important question unanswered: which definition should everything else follow?
This is where I start when a founder brings Thunkle an app that keeps breaking after apparently small changes. I trace the rule before recommending another patch or a rebuild.
A local fix can change a shared meaning
Here is a made-up example, not the client's figures. A booking costs $100, the platform charges a $10 fee and there is no tax. The intended customer charge is $110; the provider receives $100.
Checkout adds the fee and displays $110. The payment function charges a stored base amount of $100. An export reports $110. Payout subtracts a fee from the amount actually charged and sends $90.
Nothing needs to crash for all four results to disagree.
Changing the stored amount to $110 might fix the charge. If checkout still adds its fee, it now displays $120. The local repair has changed what that field means to another caller.
The original review found disagreements between paths. This example explains a possible regression, not a claim that this particular repair was deployed.
Trace the decision, not just the screen
Follow the value from creation through persistence, payment, jobs, emails, exports and reporting. For each caller, record what it expects the field to contain.
Does total include a fee? Is it a current quote, a completed charge or an estimate? Does the code read today's pricing rules or the values agreed when the customer bought?
A map of these few dependencies is more useful than an agent listing every file in the repository. If the explanation has not found the payout code, the investigation is incomplete.
Ask for the proposed repair before authorizing edits. Keep unrelated upgrades, renames and redesigns out of the same change so the result can be reviewed and reversed.
Some duplication is correct
A current product price and the price on last month's invoice may have started as the same number. They should not necessarily change together.
The invoice records a completed transaction. A current quote can be recalculated. A cache may need invalidation. Treating them all as duplicates to eliminate can damage historical records.
For the booking example, distinguish the base price, fee, amount charged and payout. Establish them from trusted inputs and retain the values needed to explain the actual purchase.
Centralizing new calculations helps only after the meanings are clear. A shared function containing the wrong rule just makes the error consistent.
Use expectations you can calculate independently
Keep test inputs and expected results together: $100 base, $10 fee, $110 customer charge and $100 payout.
Check the complete affected paths against those values. Do not copy the implementation's formula into the test and call agreement proof. Both copies could contain the same mistake.
Add cases the product supports: a discount, refund, cancelled booking or changed fee. Include an old completed purchase so the repair cannot silently reprice history.
Run the same regression test against the broken version and the repaired version in an isolated environment. It should fail because of the original defect and pass after the repair. A missing dependency or broken fixture is not the failure you are trying to prove.
The fix-verification guide covers that evidence in more detail.
Separate code repair from data correction
Correct code does not tell you which old records are wrong. Some may have been charged correctly but reported incorrectly. Others may have produced an incorrect payout.
Identify the affected versions and transaction period. Compare records with payment evidence before proposing changes.
Keep originals, define a recovery route and get approval for production data changes. A broad script that forces every old record to fit the new formula can turn confusing history into false history.
The screen repair, shared-rule change and historical correction may need separate estimates. That separation is useful, not administrative overhead.
Know when to investigate more broadly
One contained defect does not justify rebuilding a product. Repeated disagreements across credits, subscriptions and financial totals do justify asking how many parts of the app independently make the same decision.
The investigation should leave a concrete output: the rule, its callers, a reproducible failure, options for repair and acceptance tests. “The architecture is bad” is not enough to quote meaningful work.
A repair might stay local, replace one component or become part of a staged migration. The right option depends on the dependencies and the business running behind them.
Through Thunkle's application rescue work, I start with the recurring failure and a way to reproduce it. The aim is to make the next change understandable and testable, not pay for another confident explanation after the screen turns green.
Find why the same workflow keeps breaking.
I can trace the business rule, identify the callers that disagree and scope a repair with regression checks, rather than another patch to one screen.
