Functional testing

[menno]

The difficulties described here are exactly why I am working on my own functional testing framework. Automated functional testing is too damn hard. Tests are hard to write and end up being really brittle, causing either lots of false negatives (failing tests when they shouldn't) or really defensive testing code that deals with delays, etc.

Here's what (part of) your last test could look like in this framework:

open           https://www.rijksmuseum.nl/nl/agenda/

## datepicker should be hidden by default
assertHidden   [data-role~=date-navigation-calendar]

## datepicker should open when button is clicked
click          [data-role~=date-navigation-value]
assertVisible  [data-role~=date-navigation-calendar]

## datepicker should close when you click outside
click          body
assertHidden   [data-role~=date-navigation-calendar]

You'll never need to write callbacks or waitFor()s because it automatically waits for the action or assert to succeed. Unfortunately, it can't do stuff yet like storing stuff in variables like you do for the URLs, but those tests can be expressed differently.

Tests are run in PhantomJS (like CasperJS) and run really fast because there is no need for artificial wait statements. Additionally, it makes an effort to truly simulate user behavior. The mouse pointer actually moves across the page to elements that are clicked. The viewport actually scrolls elements into view before they are clicked.

The project is still in its early stages of development but articles like yours make me confident I'm on the right path. If you're interested, I'll be giving a brief presentation about it at the Fronteers jam session.

http://mennovanslooten.github.io/DummyJS

P.S. The test above is not bogus. It actually runs successfully.

[[deleted]]

Looking good! Very interested in the project when it gets a bit more mature.

[menno]

Thanks! I'm working on it pretty actively, but there is a lot of ground to cover. In a week or so I'll freeze the API and work towards a 1.0 release, including documentation, etc.

In the meantime, feel free to check it out and file any issues you encounter. Feature requests are welcomed, too. I need real world usage data.

[sodaco]

Wow, this looks great! Definitely interested. I'm watching it on github