What came out

Across 300 cycles, nearly everything caught was a defect in the automation itself. Given the premise, that’s correct.

One thing wasn’t explained by that premise.

The first thing that wasn’t an automation defect

In 147 of the 300 cycles — nearly half — the same warning appeared in the log. A graphics option was written plainly in the config file and wasn’t being applied. The cause: that value was never the kind of variable a config file can set in the first place.

There’s something to be careful about here.

This is not confirmed as a product defect. It may well be intended. That call belongs to the people who built it. Only one thing is certain: it isn’t an automation defect.

The distinction matters because of the premise above. I had been fixing automation on the assumption that the product passes everything — and this was the first signal that premise couldn’t explain. But jumping straight to “I found a product bug” would mean applying a standard of evidence to my automation that I refuse to apply to the product.

And this isn’t a brag

Honestly: the automation wasn’t clever enough to catch this.

The warning was there the whole time, all 300 times. Nobody was looking at it. What unattended repetition did was not discovery — it was accumulation. One run makes it a single log line you scroll past. Three hundred runs make it 49%, and 49% is hard to scroll past.

Frequency is a kind of evidence people can’t count by hand. Run it ten times by hand and see the same warning in five of them and it stays at “that shows up sometimes.” 147 out of 300 is a different kind of fact.

What this loop is genuinely good at isn’t finding new things. It’s making already-visible things too large to ignore.

So what changed

Since that finding, there’s one more thing in the verdict set: counting repeated warnings in the log.

It used to look at pass and fail per case, and nothing else. Now the end of a run also emits a frequency table by warning type — which warning appeared in what percentage of cycles.

This earns its keep because it puts in front of a human exactly what the automation can’t judge. A machine can’t decide “is this warning a problem.” It can decide “this warning appears in 49% of runs,” and a person looking at that number can.

The table prints even on an all-PASS run. It’s the only window into what was accumulating behind the green lights.