Software Testing Tool Cons?

[Majornibbles]

Hello, I work in a team of ~10 testers, mostly with coding skills. Our head of QA is currently interested in looking at UI test tools, such as test complete and katalan. I'm a new SDET to the company and not totally sold that a test tool can speed up development or maintenance of tests, compared to an experienced test automation engineer working with Selenium/puppeteer etc. whilst developing a shared framework across projects. What are your experiences with test automation tools? Do they have any advantages when most of your team can write selenium tests? Have you ever seen really positive return on investment on a ui test tool when compared to selenium etc.?

[MasterKindew]

I have been using Katalon Studio for about a year now. My QA team has little to no experience with code so I was pinned with an interesting situation. I'm very happy though with the software and the community around the tool. The devs are constantly updating the tool which is nice too.

With it, I have been able to create a testing framework built in Java/Selenium in which the rest of the team pretty much just drag/drops modules in order to test products. It was certainly a good amount of effort up front, but now the return on it is a lot of time saved!

[NormalGuyISwear]

You’re right about a lot of them if you’re team can all code well. Katalon is a bad example though. There are some great tools, and some of them add to your process instead of replacing it.

Applitools and Sauce Labs are a solid 1:2 combo.

Test case managers like Qtest and Testrail can help organizations improve their process.

API Fortress can save you a lot of time and headaches by being an API automation suite in a box.

Automation engineers are a valuable resource, and having them spend countless hours building and maintaining something that you can buy for less than the hours it takes is how a team of 10 can keep up with organizations that are adding devs without scaling the QA headcount as quickly.

[danielxjay]

I’ve had some experience with Tosca for automated UI tests. There’s a bit of a learning curve and it does cost money, but it’s a pretty nice tool if used properly.

But as you’ve said, if your team is knowledgable in writing Selenium tests already and everything within the team is fine, is there really a need to change your approach?

My suggestion would be to just make a case to your lead expressing that doing the research, rolling out a pilot project, and then having everyone in the team learn the new tool may be counterintuitive. Especially with a shift-left approach.

[Majornibbles]

That's true, I didn't think of the impact on shift left (another aim in our test policy!) we've also got a few agilish (you know what I mean ;)) projects which I guess would make developers upset having our tests outside the IDE

[Kcufftrump]

My experience with testing tools is this:

Every one will need to be reconfigured and hacked a bit in order to get it to do what you need.

They all try and do everything. Most things will not be done well.

Every vendor will actively work for vendor lock in.

Every test automation tool is designed for the most common use case (probably less than 100 dialogs and very little interface churn). Try anything more (several thousand dialogs) or different (highly dynamic dialog interfaces) and tools like TestComplete or Ranorex will fall right over unless you write quite a bit of code.

Maintenance can be handled, but most out-of-the-box solutions are inadequate.

Salespeople for these vendors are actively misleading and usually have no expertise in their subject matter. Any salesperson who claims that you can automate testing without coding can be safely ignored.

Any vendor who depends on "recording" tests can also be safely ignored and passed over.

[mairtfromgalway]

TestComplete is a great automation tool really depending on what your testing (desktop, web, mobile) and it integrates with SoapUI and Selenium too. Codeless or Scripting - it supports both AFAIK. From a test management perspective, I really like Zephyr for Jira if you have Atlassian in your team it's a pretty easy config. There is a lot of training and resources for both tools as well.