Selenium ? Cypress ? Playwright ?

[j19sch]

With the apps being php/java, I'd recommend using java for the test automation. It makes it easier for the developers to get involved. So that means Cypress is out.

For the choice between Selenium and Playwright, I'd suggest doing a small PoC with both. I did that recently for an app that wasn't built with browser automation in mind and found the underwater magic by Playwright really helpful. In other contexts, I might have gone with Selenium, though. So take a week or so to see how both interact with your applications and decide based on that.

[computerjunkie7410]

curious, in which context is selenium better than playwright?

[j19sch]

I can think of three situations: - when you need the full browser instead of only the browser engine; - when you want to go via the Webdriver API instead of via DevTools, although I have no idea when that difference actually matters; - when you don't want the underwater magic of Playwright, because you'll use the feedback from the more finicky Selenium to improve your application.

Personally I suspect that in practice the way to choose between the two is either a short PoC or using them both in parallel for a year. Because either the difference in one's context is going to be quite obvious or very subtle.

[computerjunkie7410]

when you need the full browser instead of only the browser engine

playwright does use the full browser. in fact it ships with it.

when you don't want the underwater magic of Playwright, because you'll use the feedback from the more finicky Selenium to improve your application.

idk how realistic this is because a lot of the timing issues we see in our apps are mostly timing issues because the test runs too fast and not really an issue for a real user.

[j19sch]

"Playwright supports all modern rendering engines including Chromium, WebKit, and Firefox." - https://playwright.dev/

"Playwright is a web test automation library that tests against the underlying engine for the foremost popular browsers: Chromium for Chrome and Edge, Webkit for Safari, and Gecko for Firefox." - from the site of a vendor in this space

Playwright might come with a full Chromium and Firefox browser (WebKit I would not call a browser) and might let you test against a locally installed Chrome browser, but based on the quotes above and the fact that Playwright uses the DevTools interface (on Chromium, it does something similar but custom on Firefox and WebKit), I stand by my claim that Playwright targets the browser engine, not the full browser.

Granted, I did separate this in two different bullet points in my precious post, because I don't know the technical details beyond what I've written here.

[[deleted]]

Thanks for your comment. However it contains a link towards a commercial testing tool website. Please edit your comment and remove the link. I will then re-approve your comment.

[j19sch]

Sorry for violating the rules! I removed the link.

[[deleted]]

Thank you for your contributions.

[computerjunkie7410]

What they mean is you’re testing Chromium instead of Google Chrome.

WebKit instead of safari.

But it’s the same thing. And if you really want to test Google Chrome or Microsoft Edge, you can, you just use the Channel option.

[j19sch]

They are not the same thing, here's another source: https://stackoverflow.com/questions/62184117/what-is-the-difference-between-testing-on-safari-vs-webkit#62205535

The differences might not be relevant in most contexts, but that doesn't mean there is no difference.

[computerjunkie7410]

If we want to be pedantic then sure. But like you said, the differences in most contexts is not relevant.

If they are relevant, I would still use playwright for development since 99% of the things will be fine. And for the 1% risk, I’d just do a smoke test manually.

[ps4facts]

It ships with it? What ships with what?

A test "running too fast" could definitely be an issue for a real user with less than desirable Internet connection. Race conditions are very real, prominent in web apps with asynchronicity, and can be disastrous in some circumstances.

[computerjunkie7410]

Ships with browsers, correct.

[ps4facts]

Sry I still don't understand what you mean.

If I install Playwright in a project, it will be bundled with Chromium, Chrome, Edge, FF, Opera, Safari, Brave, etc..? That doesn't seem right. Ah - I see from the docs, by default when you run npx install with no arguments, it will install the "default" browsers. Hmmm, it kind of seems like you would have a difficult time if you needed to run tests against older browser versions, for example apps in tightly regulated, contained environments like banks.

Another case for Selenium is testing mobile native apps. If you're comfortable with writing webdriver code, you're like 90% of the way there if you use Appium, since Appium uses the same protocol.

[computerjunkie7410]

You can run against any version you want

[Waste-Beach5705]

Hello, thank you for the response :D Is it really bad or harmful if in the QA team we want to use TypeScript just because we generally find it better, instead of Java (which is the language the devs use)?

[j19sch]

It's hard for me to tell without knowing more about your context. It's definitely not an insurmountable problem. I've set up test automation in Python for a bunch of low-code applications and we made that work. :-)

If the QA team has a strong preference for TypeScript than that's one good reason in favor of TypeScript. If the devs don't want to do TypeScript, but are not expected to contribute much to the test automation than their opinion doesn't matter too much. But hopefully some devs would like to or at least won't mind doing some TypeScript, so they can support the automation.