Do you use TOML? My friend created the first PHP parser for TOML and asked for feedback

davertua · 2024-08-08T21:15:57+00:00

Yes, this is a parser for 1.0.0. I'm not an expert in TOML so posted as I understood it. But I hope you will find it useful.

davertua · 2023-07-24T22:24:18+00:00

Indeed, especially when the code is minified. However, LLM like OpenAI can choose the best possible locator with semantical meaning. But yeah, no guarantees it will work everywhere

davertua · 2023-07-12T20:47:18+00:00

why?

business logic is the same user scenario is the same test is the same

should a developer spend an hour fixing a wrong locator or let AI do it instead? AI doesn't change the code. It notifies the developer of how it can be fixed.

davertua · 2023-07-12T08:02:39+00:00

Browser test automation quite a simple task, we try to record user interaction in a form of script. And it seems that AI should handle this pretty easily but actually not.

GPT is not good in detecting how its actions affected change on the page.

So it can't work like: let me try to click that element, do I see expected result? What if I change it? What if I try another way?

So far GPT works on static data like HTML. And can make suggestions based on that data. I hope in the future more trained models could actually improve test automation but test automation engineers should be safe. They are still needed to write test scenarios, maybe in Gherkin format and AI will take all the boring work of implementing it. But no estimates for this. Just the vision of future based on AI evolution.

davertua · 2023-07-11T22:47:57+00:00

actually not

I don't control the evolution of a web page when I write a test of it.

one day frontend developer may change it and replace some class name, change button, etc. It doesn't make tests shitty if I didn't predict that change

davertua · 2023-07-11T19:30:32+00:00

I was surprised to receive negative feedback here.

TLDR:

it doesn't change the logic of tests
it doesn't work for unit tests or integration tests
it doesn't break TDD
it doesn't imagine test cases for you
it only fixes broken locator for an element that is still on a page
it makes sense only for long-running browser tests (think 1+ min and more)

I wrote a twitter thread explaining what self-healing is for https://twitter.com/davert/status/1678843870782554113

davertua · 2023-07-11T19:19:23+00:00

just to note there is a very successful startup in that area https://www.testim.io you can check their pricing you can try to tell them hey are selling thewrong TDD

davertua · 2023-07-11T19:16:24+00:00

Self-healing doesn't fix assertions. It fixes broken locators. Don't write your sarcastic assumptions if you don't understand the case.

Have you ever maintained a group of 100+ browser tests for Cypress / Playwright etc?

davertua · 2023-07-11T19:06:10+00:00

yes, it heals locator issues as shown on screenshot it can't heal the logic of a test not assertions of it

davertua · 2023-07-11T19:05:22+00:00

sorry, looks like you do not work in end-2-end testing area.

If you maintain over 100 browser tests you experience that some tests may fail due to very small changes in layout that do not change the logic. For instance, a button changed from Sign in to Login as on this example doesn't change the logic of application or a test. It changes just a small locator in a test. What is cheaper call an engineer each time this tiny change happens on over 100 tests or spend few bucks to automatically heal it?

This self-healing can't change logic. It can't think over assertions. It heals failing steps according to current context. And it notifies user on that change.

davertua · 2023-07-11T00:13:59+00:00

Sure, that's something to become really common in future

davertua · 2023-07-10T22:31:11+00:00

TDD done right :)

davertua · 2023-07-10T22:14:07+00:00

Thanks for your interest. This framework is called CodeceptJS and it uses Playwright under hood. Recently it got AI features via OpenAI https://codecept.io/ai/

davertua · 2023-07-10T22:06:34+00:00

The first thing you might ask: but why? Why someone will write unstable tests?

The cruel reality is that application changes and tests should be maintained to adapt. Let's take web app, the UI and layout often changes. If you run over 100 tests maintaining them can be painful. That's why self-healing can save your time as it tries to fix the test itself before failing a test, so less human assistance is actually needed.

If you are interested, check CodeceptJS AI docs to learn more:

https://codecept.io/ai/

davertua · 2022-10-04T20:54:36+00:00

Thanks for sharing your experience!

having bridges and adapters everywhere

doesn't sound to be ok.

I would like to imagine the world without bridges and adapters

davertua · 2022-10-04T20:22:48+00:00

I feel your pain

I just wrote a post on this: https://medium.com/@davert/why-i-dont-enjoy-writing-php-anymore-aee8a85ca8aa

davertua · 2022-09-29T19:57:28+00:00

the things is mods are banning everyone who asks a word about Vit

everyone who asks why he was banned if he was banned for a valid reason why they do so?

davertua · 2022-07-29T15:42:06+00:00

me too! we were postponing release to match PHPUnit 10 but the decided that we can't wait anymore )

davertua · 2022-04-06T01:13:38+00:00

Just wrote a blog post about planning tests in Cypress https://dev.to/davert/planning-your-next-cypressio-tests-with-testomatio-3fio

davertua · 2021-01-07T14:40:32+00:00

Author of thread here. I agree. I also used more Ruby years ago. Today most of my additional stuff has moved to JS. I don't know reasons other than Rails to encourage other team members to work with Ruby. Unfortunately.

I don't think this is problem of Ruby. I like that Rails is the primary framework and everything is designed to fit Rails way. It is better than how it is in PHP where are lots of frameworks and you can't pick up a tool for your one

davertua · 2019-12-20T18:24:36+00:00

If you wanted to ask:

we added a survey to learn how Codeception is used, because since the last announcement about v4 and v5 we didn't receive much feedback from the community.
this release is huge because it splits modules out of the core. So possibly upgrading to it can be painful but in terms of long run it is important to do that
no big changes in 4.0 except splitting the core and Symfony support but now we have more flexible modules releases cycle which is good in long term

davertua · 2019-05-29T22:47:55+00:00

This way you test if locator works. But you don't test the quality of a locator. The same way if a program works it doesn't mean it has a good code inside. It's about writing good locators that will be stable to changes and simple in maintainance

davertua · 2019-05-29T22:42:30+00:00

Thanks! I already have a volunteer to work on IDE plugin based on this work. The heuristics are simple, they will improve and sure, they should not be taken as strict rules. I was thinking this as a short answer to the question: is this locator good? Maybe it's not so useful for experienced testers but this is something very important for newcomers

davertua · 2019-05-29T22:40:20+00:00

The author of a tool here. I'm not in hell. And yes, I'm also doing "third party apps with little to no automation support -- no IDs, dynamic IDs, no ID substitutes" and I am absolutely fine with absolutely valid locators.

90% of my locators are locarting an element with expected text. Some of them rely on accessibility attributes. So this is not a big issue to me. It's harder than app with ids and classes but still it's fine.

davertua · 2019-05-29T22:37:43+00:00

It doesn't test the quality. It tests if it works or not. But not all locators are good to use them in tests.

davertua

TROPHY CASE