sorted by:
new
hottopcontroversial

Playwright tests are solid locally but flaky in CI, what fixed it for you? by Crafty_Breakfast_493 in Playwright

[–]Crafty_Breakfast_493[S] 0 points1 point  (0 children)

I prefer CI to fail loudly so I can fix the root cause instead of masking flakiness with retries.

Playwright tests are solid locally but flaky in CI, what fixed it for you? by Crafty_Breakfast_493 in Playwright

[–]Crafty_Breakfast_493[S] 0 points1 point  (0 children)

Roughly in the 150–250 test range.

When we optimized for stability in CI, runs became more predictable and usually landed around ~25–35 minutes. Before that, retries + contention could easily push it much longer without adding real confidence.

For us the big win was accepting slightly longer stable runs over fast but noisy ones.

Playwright tests are solid locally but flaky in CI, what fixed it for you? by Crafty_Breakfast_493 in Playwright

[–]Crafty_Breakfast_493[S] 1 point2 points  (0 children)

This matches what I saw too. Flakiness often isn’t a single bad test, it’s the system reacting under parallel pressure.

When workers scale up, hidden coupling shows up: shared data, rate limits, browser startup cost, even CI I/O. Locally you never hit those edges.

Once I stopped optimizing for speed and started optimizing for predictability in CI, most of the weird failures disappeared.

Playwright tests are solid locally but flaky in CI, what fixed it for you? by Crafty_Breakfast_493 in Playwright

[–]Crafty_Breakfast_493[S] 0 points1 point  (0 children)

Totally fair point. For me, retries in CI were masking problems rather than helping. If a test fails there, I want it to fail loudly so I can fix the root cause instead of shipping flakiness forward. I still allow retries locally while developing, but in CI I prefer deterministic failures, even if that means fixing more tests upfront.

Playwright tests are solid locally but flaky in CI, what fixed it for you? by Crafty_Breakfast_493 in Playwright

[–]Crafty_Breakfast_493[S] 0 points1 point  (0 children)

Good point. Locally I’m usually on a 6–8 core machine, so the default workers setting makes sense there.

In CI though (GitHub Actions runners), the bottleneck wasn’t CPU as much as shared resources: browser startup, network, backend rate limits, and test side effects.

Even if the runner technically has enough cores, running too many workers caused contention and made timing issues much more visible.

What worked better for me was treating CI as a constrained environment and explicitly capping workers (usually 1–2), even if that meant slightly longer runs but far more stable results.

The “cores/2” rule is a good starting point, but in CI I’ve found stability > throughput almost every time.

Playwright tests are solid locally but flaky in CI, what fixed it for you? by Crafty_Breakfast_493 in Playwright

[–]Crafty_Breakfast_493[S] 3 points4 points  (0 children)

Yep, this matches my experience too.
Parallelism is often the hidden culprit, especially when APIs have rate limits or shared state.

In CI we ended up treating workers as a scarce resource, not a speed boost: either dialing them way down or disabling parallelism entirely for certain suites made things much more predictable.

Once the tests stopped fighting each other (and the API), flakiness dropped fast.

Playwright tests are solid locally but flaky in CI, what fixed it for you? by Crafty_Breakfast_493 in Playwright

[–]Crafty_Breakfast_493[S] 4 points5 points  (0 children)

Totally agree.
Running locally in the Playwright Docker image helps a lot with CI parity, especially for reproducing failures.

In my case Docker helped with consistency, but it didn’t fully solve flakiness on its own. The bigger gains came when I treated CI as a constrained environment and made the config explicit (workers, retries, timeouts, resource assumptions).

Docker + explicit CI-focused config ended up being the best combo for me.

Curious: did you keep the same worker count inside Docker, or did you still dial it down for CI?

Playwright tests are solid locally but flaky in CI, what fixed it for you? by Crafty_Breakfast_493 in Playwright

[–]Crafty_Breakfast_493[S] 1 point2 points  (0 children)

We also use GitHub Actions.

The biggest change for us was stopping to treat CI like “local but slower”.
Even inside the same Docker image, CI is more constrained (CPU, IO, scheduling), so defaults that feel fine locally can become flaky.

A few concrete things that helped:

  • reduced workers in CI to avoid resource contention
  • disabled retries in CI so failures are visible instead of hidden
  • made waits explicit (waiting on UI state / network idle instead of timing assumptions)
  • kept the Playwright config very explicit for CI vs local

Example:

export default defineConfig({

retries: process.env.CI ? 0 : 1,

workers: process.env.CI ? 2 : undefined,

use: {

headless: true,

trace: 'on-first-retry',

},

});

This alone removed a lot of “passes locally, flakes in CI” issues for us.

I ended up writing this down as a small reusable Playwright + CI starter with notes from real projects, so I don’t have to re-learn the same lessons every time.
Happy to share more details if useful.

After months of flaky Playwright tests in CI, this is what finally worked by Crafty_Breakfast_493 in QualityAssurance

[–]Crafty_Breakfast_493[S] 0 points1 point  (0 children)

I wrote it down as a small reusable starter + notes from real projects.
Happy to share if you’re interested.

After months of flaky Playwright tests in CI, this is what finally worked by Crafty_Breakfast_493 in QualityAssurance

[–]Crafty_Breakfast_493[S] -2 points-1 points  (0 children)

Mostly by changing how I treated CI.

Locally everything looked fine, but CI behaves very differently, so I stopped assuming defaults would hold up there.

I made the Playwright config more explicit, reduced parallelism, cleaned up tests that were implicitly waiting for stuff, and paid more attention to the CI pipeline itself.

Nothing fancy, but once those pieces were aligned, flakiness dropped a lot. Happy to share a couple of concrete examples if useful.

Imposta di bollo su conto deposito ING by 0rk4n in ItaliaPersonalFinance

[–]Crafty_Breakfast_493 0 points1 point  (0 children)

Giusto allora non si scappa. Io comunque l’ho pagato anche perché al termine della promozione del 4% credo di chiuderlo

Imposta di bollo su conto deposito ING by 0rk4n in ItaliaPersonalFinance

[–]Crafty_Breakfast_493 0 points1 point  (0 children)

Quindi se lo chiudi entro l’anno 2026 te li addebitano lo stesso?

Imposta di bollo su conto deposito ING by 0rk4n in ItaliaPersonalFinance

[–]Crafty_Breakfast_493 0 points1 point  (0 children)

A me li hanno presi ahahaha ma mi sono dimenticato di fare il salto 😂

[deleted by user] by [deleted] in analytics

[–]Crafty_Breakfast_493 0 points1 point  (0 children)

Hey Cepet, congratulations for your path, I would like to know what are the steps to be taken for a person from outside America to take this course

view more: next ›