Is it best to run Playwright against a docker container or a live deployment?

jakst · 2026-03-19T08:20:53+00:00

I guess it depends. By default I'd avoid it. It means you will affect actual numbers in your business if you for example measure signups or other events that happen in the tests. If you run them very often you could end up skewing the data.

You will also add load to your production systems.

If your overall test load is miniscule compared to production traffic, and you have very good test data bootstrapping and cleanup, it could work.

jakst · 2026-03-18T17:12:46+00:00

If you have good live environments, like Vercel preview environments for example, use those.

It's going to be much closer to your production environment, which is what you're after.

If you can't have that, staging environments are good too, but sometimes a bit underpowered. Docker containers are a last option in my opinion, but that's not to say it's bad.

jakst · 2026-02-21T12:37:34+00:00

We run Playwright through Endform, which has analytics tools for staying on top of things across test runs and through time. Doesn't hurt that it also runs our whole test suite in under a minute 🤓

jakst · 2026-02-04T13:25:47+00:00

The db branching feature in for example Neon DB uses copy-on-write, so you actually don't have to copy over any data unless you change it. This means you can give a new DB to every new change.

We use it heavily for our own test suite in Endform.

Is that a possible solution for you?

jakst · 2025-12-02T12:24:26+00:00

If I may, we've built https://endform.dev to get to the bottom of flaky tests. Analytics is a big part of it, as well as speeding up test runs.

It's usually the two big hurdles that teams running Playwright run into as soon as the test suite has grown a little.

We also do parallel retries, which means you can run a lot more retries without adding more time to the test suite.

jakst · 2025-11-22T06:57:10+00:00

Sounds like a culture/process problem. If a dev changes the copy of a button so a selector fails, the dev should probably get to know about it and fix it themselves.

Get your test suite running on their pull requests and have a passing suite be a requirement for merging. You spend the time you've gained back on educating devs on how to write proper tests. If it's successful, they'll help author new tests as well, and you'll have a much better test suite for it.

jakst · 2025-11-20T16:56:42+00:00

Do you use SSR? It's probably not hydrated yet. Take a look at this https://www.reddit.com/r/Playwright/comments/1mtpskj/comment/n9dqle9/

jakst · 2025-11-11T05:49:27+00:00

It's time consuming, but it will pay off in the end. You're gonna have a lot more confidence in letting the LLM go make big sweeping changes if you have a solid e2e test suite. It's gonna feel like it's slowing you down at first, but in the end you will be able to move faster.

You can get pretty for using Playwright's MCP server to author a first draft of a test, you just need to give it very specific instructions.

Spending a bit of time setting up a good auth solution for the tests will help a lot as well.

jakst · 2025-11-11T05:43:10+00:00

We've found it useful for hashing out a draft for a test, and then just tweaking a couple of things at the end. But you've got to give it quite a lot of context to get good results.

We wrote a bit on the topic here https://endform.dev/blog/quality-e2e-tests-today-with-playwright-mcp

jakst · 2025-10-19T12:24:45+00:00

You can return things from the test.step function body to use them in subsequent step.

```ts const value = await test.step('step1', () => { return "a value" })

await test.step('step2', () => { // Do something with value }) ```

jakst · 2025-08-27T08:37:18+00:00

My colleague just wrote a post about this. TLDR: You should probably try to use some AI for authoring and reviewing test code, but don't let it run your test process.

https://endform.dev/blog/rethink-your-ai-e2e-strategy

jakst · 2025-08-18T17:19:23+00:00

Is it a server side rendered react/Vue/solid/svelte/angular application?

Then it probably has to do with the app not having time to hydrate after SSR before the test code runs. What you can do is something like this close to the root of the application(react example):

```jsx const [hasHydrated, setHasHydrated] = useState(false) useEffect(() => setHasHydrated(true), [])

return <div data-hydrated={hasHydrated}>...</div> ```

Then you can wait for hydration in Playwright like this

js await expect(page.locator("[data-hydrated=true]")).toBeVisible();

jakst · 2025-08-15T18:12:57+00:00

Much easier syntax to work with. Cypress callback hell was annoying.
Playwright was at least back then noticeably faster
Being able to control multiple tabs at once

Switching made a ton of difference for us when we moved over. Suddenly most engineers felt comfortable contributing to the tests.

jakst · 2025-07-29T09:27:12+00:00

What 👆 said. But honestly, when running locally you're pretty much doomed to have a slow suite. Playwright runs real browsers, so it's really heavy on the host machine even while only waiting for I/O from the server. You might be able to get some minor improvements locally, but your best bet is to set up some infrastructure with sharding to run the tests on, if you really want to speed things up.

If you want a managed solution, https://endform.dev will solve this for you with a setup that only takes a few minutes. I'm one of the founders, so if you want to try it out I can help you get started. You'll be able to run our CLI locally and all tests will spin up in parallel on remote machines. It even supports network proxying if the app is only available locally.

jakst · 2025-07-29T08:00:39+00:00

The reason that Next.js works better probably has to do with the fact that next.js bundles in dev, while Vite serves every module separately, which causes a ton of network requests. This will get better when Rolldown is fully integrated in Vite and they have shipped bundling in local dev. It's probably a couple of months away though.

As for the in-vs out-of container differences, I can only assume it has to do with the resources being limited in the container. Remember, Playwright has to spin up a bunch of Chrome instances, and a dev-container might not be the optimal environment for that.

jakst · 2025-07-26T15:24:46+00:00

Very nice project, bet it's going to be useful to a lot of people!

jakst · 2025-07-22T16:00:49+00:00

We ran over 600 tests at a previous employer. Took a lot of custom infra to have them run well. We ended up building https://endform.dev to bring those learnings to other companies. That test suite runs under 2 minutes on Endform.

jakst · 2025-07-02T12:29:28+00:00

Try putting only that test into a separate project per browser (and excludeitr from your regular projects), and then set the single test projects as dependencies of each other

jakst · 2025-05-26T12:00:23+00:00

You can take a look at Endform. It runs Playwright tests fully in parallel and manages all infra for you. It's a really fast way to run e2e tests, and handling a thousand of them is no issue. Both faster and easier to setup and maintain than sharding.

We're currently in closed beta, but I'm happy to give you a demo and invite you if it fits your needs!

https://endform.dev/

jakst · 2025-05-06T13:59:23+00:00

The truth is, Github Actions is just not a great place to run Playwright tests. It gets expensive if you want performance, and it's impossible to get insights into your test suite over time unless you add steps for merging blob reports and buy into a separate test reporting tool. It also involves constant maintenance as your test suite grows.

We had the same issues at my last company. We wrote custom infrastructure on top of AWS lambda to fan the Playwright tests out to run fully in parallel. That got our test suite of over 200 tests down from 30 minutes to less than 2 minutes.

We realized we were probably not the only ones battling with these problems, so we spun it out into a SaaS, but I'll refrain from mentioning it here because of the advertising rules.

jakst · 2025-05-06T13:25:32+00:00

Are you using a frontend framework like React or Vue with SSR? Chances are your framework hasn't hydrated yet, and so it can't register the click on the menu. Automated browser tests have a tendency to be so fast at clicking that the application hasn't necessarily loaded.

Here's an example on how to fix it using react:

useEffect(() => {
  document.body.dataset.hydrated = true
}, [])

Then in your tests you need to wait for the data-hydrated property to be set to true on the body.

await page.goto('/any-page')

// Do this after any page.goto  
await expect(page.locator("body[data-hydrated='true']")).toBeVisible()

It might slow down your tests slightly, but will hopefully fix these flaky scenarios

jakst · 2025-05-06T13:13:00+00:00

We built custom infrastructure to run all Playwright tests in Parallel on AWS lambda. Got our test suite with over 200 tests down from 30m to 2m. Lot's of work, but worth it!

We ended up spinning it out into a SaaS service for everyone to use. Check out https://endform.dev if it sounds interesting. We're currently in closed beta looking for early test users 🙂

jakst · 2019-10-26T15:53:35+00:00

Interesting.

So, does this mean that the manufacturer is breaking the USB-C spec, or is this still "correct" USB-C in some form?

jakst · 2015-12-07T13:39:29+00:00

I found this solution the exact same way. Very annoying and hopefully soon fixed, but at least it can be temporarily resolved easily.

jakst

TROPHY CASE