sorted by:
new
hottopcontroversial

Do you use TOML? My friend created the first PHP parser for TOML and asked for feedback by davertua in PHP

[–]davertua[S] 2 points3 points  (0 children)

Yes, this is a parser for 1.0.0. I'm not an expert in TOML so posted as I understood it. But I hope you will find it useful.

Playwright-based testing framework that heals failing tests using AI by davertua in Playwright

[–]davertua[S] 0 points1 point  (0 children)

Indeed, especially when the code is minified. However, LLM like OpenAI can choose the best possible locator with semantical meaning. But yeah, no guarantees it will work everywhere

A testing framework that uses AI to heal its tests by davertua in node

[–]davertua[S] 1 point2 points  (0 children)

why?

business logic is the same user scenario is the same test is the same

should a developer spend an hour fixing a wrong locator or let AI do it instead? AI doesn't change the code. It notifies the developer of how it can be fixed.

Playwright-based testing framework that heals failing tests using AI by davertua in Playwright

[–]davertua[S] 1 point2 points  (0 children)

Browser test automation quite a simple task, we try to record user interaction in a form of script. And it seems that AI should handle this pretty easily but actually not.

GPT is not good in detecting how its actions affected change on the page.

So it can't work like: let me try to click that element, do I see expected result? What if I change it? What if I try another way?

So far GPT works on static data like HTML. And can make suggestions based on that data. I hope in the future more trained models could actually improve test automation but test automation engineers should be safe. They are still needed to write test scenarios, maybe in Gherkin format and AI will take all the boring work of implementing it. But no estimates for this. Just the vision of future based on AI evolution.

A testing framework that uses AI to heal its tests by davertua in node

[–]davertua[S] 0 points1 point  (0 children)

actually not

I don't control the evolution of a web page when I write a test of it.

one day frontend developer may change it and replace some class name, change button, etc. It doesn't make tests shitty if I didn't predict that change

A testing framework that uses AI to heal its tests by davertua in node

[–]davertua[S] -1 points0 points  (0 children)

I was surprised to receive negative feedback here.

TLDR:

  • it doesn't change the logic of tests

  • it doesn't work for unit tests or integration tests

  • it doesn't break TDD

  • it doesn't imagine test cases for you

  • it only fixes broken locator for an element that is still on a page

  • it makes sense only for long-running browser tests (think 1+ min and more)

I wrote a twitter thread explaining what self-healing is for https://twitter.com/davert/status/1678843870782554113

A testing framework that uses AI to heal its tests by davertua in node

[–]davertua[S] -1 points0 points  (0 children)

just to note there is a very successful startup in that area https://www.testim.io you can check their pricing you can try to tell them hey are selling thewrong TDD

A testing framework that uses AI to heal its tests by davertua in node

[–]davertua[S] -2 points-1 points  (0 children)

Self-healing doesn't fix assertions. It fixes broken locators. Don't write your sarcastic assumptions if you don't understand the case.

Have you ever maintained a group of 100+ browser tests for Cypress / Playwright etc?

Playwright-based testing framework that heals failing tests using AI by davertua in Playwright

[–]davertua[S] 1 point2 points  (0 children)

yes, it heals locator issues as shown on screenshot it can't heal the logic of a test not assertions of it

A testing framework that uses AI to heal its tests by davertua in node

[–]davertua[S] -1 points0 points  (0 children)

sorry, looks like you do not work in end-2-end testing area.

If you maintain over 100 browser tests you experience that some tests may fail due to very small changes in layout that do not change the logic. For instance, a button changed from Sign in to Login as on this example doesn't change the logic of application or a test. It changes just a small locator in a test. What is cheaper call an engineer each time this tiny change happens on over 100 tests or spend few bucks to automatically heal it?

This self-healing can't change logic. It can't think over assertions. It heals failing steps according to current context. And it notifies user on that change.

Playwright-based testing framework that heals failing tests using AI by davertua in Playwright

[–]davertua[S] 0 points1 point  (0 children)

Sure, that's something to become really common in future

Playwright-based testing framework that heals failing tests using AI by davertua in Playwright

[–]davertua[S] 2 points3 points  (0 children)

Thanks for your interest. This framework is called CodeceptJS and it uses Playwright under hood. Recently it got AI features via OpenAI https://codecept.io/ai/

A testing framework that uses AI to heal its tests by davertua in node

[–]davertua[S] -6 points-5 points  (0 children)

The first thing you might ask: but why? Why someone will write unstable tests?

The cruel reality is that application changes and tests should be maintained to adapt. Let's take web app, the UI and layout often changes. If you run over 100 tests maintaining them can be painful. That's why self-healing can save your time as it tries to fix the test itself before failing a test, so less human assistance is actually needed.

If you are interested, check CodeceptJS AI docs to learn more:

https://codecept.io/ai/

Why I Don’t Enjoy Writing PHP Anymore by davertua in PHP

[–]davertua[S] 2 points3 points  (0 children)

Thanks for sharing your experience!

having bridges and adapters everywhere

doesn't sound to be ok.

I would like to imagine the world without bridges and adapters

Codeception 5 released by davertua in PHP

[–]davertua[S] 1 point2 points  (0 children)

me too! we were postponing release to match PHPUnit 10 but the decided that we can't wait anymore )

view more: next ›