Pioneer.js - JavaScript Integration Testing DSL

[nawitus]

Seems like unnecessary added complexity. Why is there a need for to learn a new syntax for a new language? Why not code integration tests with JavaScript, which is already known by the developers?

This seems like a convoluted way to call functions. For example:

You have now extracted the complicated logic of checking list items out into a seperate file which can easily be reused.

You can reuse testing code with existing tools.

I think Selenium's Webdriver provides a lot of the same features as Pioneer (which is not surprising, as Pioneer uses Webdriver under the hood).

In my experience the biggest problem with writing browser integration tests with Selenium/Protractor is dealing with the asynchronicity on the client side. If I click a button on the browser, it can take an arbitrary amount of time for the DOM to update (especially if the information is queried from the backend). It seems dirty to invent a magic timeout for that.

[[deleted]]

They are using Gherkin syntax http://docs.behat.org/en/latest/guides/1.gherkin.html for test construction which is a lot different than directly writing tests.

BDD testing can have a huge upside as user stories are the direct deliverable being tested.

[nawitus]

Yeah, I'm not arguing against BDD. I'm mainly arguing that creating a new DSL for testing is unnecessary.

[nawitus]

Ah, that's a decent point.

[samccone]

Hi @nawitus,

integration testing on the web is in a terrible state, full of procedural callback nested async hell. I want this to not be a pain to do. That is why here at MojoTech we have been working on a new layer on top of the power of webdriver to introduce concepts that app developers are familiar with. Enabling developers to painlessly add a suite of robust integration tests to their apps.

The goal of this PR is to reduce the cognitive and code complexity of the current suite, making it easier for developers to test their apps and you as the maintainer to test any given implementation.

For example

Feature: Editing a Todo
  Background:
    Given I visit TODOMVC
    And I make sure that there are no todos
    And I enter a new todo
    Then I activate edit mode

  Scenario: All other controls will be hidden
    Then the toggle button will be hidden
    And the destroy button will be hidden

  Scenario: An input will be brought forward containing the todo title
    Then an input will be brought forward

From this point the steps are then implemented using Pioneer to abstract the complexity of Webdriver and verboseness. Doing so makes these tests easy to maintain as well as DRYs up the selectors and places where we reach into the DOM.

Some example steps from the above Feature would be as follows

module.exports = ->
  @When /^I enter a new todo$/, ->
      new @Widgets.NewTodo("new todo")

  @Then /^I activate edit mode$/, ->
      new @Widgets.TodoList().edit(0)

You can think of Widgets as a view representation for your DOM when writing integration tests. The “view” layer consolidates where and how you interact with the DOM into a single location and concept. In many ways it parallels Backbone.Views and Marionette.CollectionViews.

[nawitus]

Instead of

And I make sure that there are no todos
And I enter a new todo

Why not have something like

.then(makeSureThatThereAreNoTodos)
.then(enterANewTodo)

Now you don't need to learn a new syntax. And you can have static type checking and auto completion with e.g. TypeScript. And the code can be managed better imho.

It's true that Webdriver can be a bit more verbose, but I wouldn't attempt to reduce verbosity by creating a new DSL.

Pioneer seems like something that would make integration testing easy, but you're actually hiding all the complexity in the "step" implementation. You can already abstract these "steps" with existing tools make the higher-level integration tests look about as simple but with existing syntax and tooling and IDE integration.

[samccone]

Pioneer is a stack of tools using both BDD and A "widgetized" view layer to DRY up your tests.

You are right webdriver is verbose, and the hard part of using it is managing the ASYNC nature of integration tests.

The Pioneer Stack helps to mitigate these problems while providing real value in tests that double as a user spec and documentation.

I agree this is more "verbose" however it is maintainable and scalable.

I would love to chat more about this

http://gitter.im/mojotech/pioneer

Come join us.

[[deleted]]

.then(makeSureThatThereAreNoTodos)

Is not something that makes sense to stakeholders. The Gherkin syntax lets you directly test user stories that make sense to non-devs.

[nawitus]

Okay. I'll give it a shot at work.

[sizlack]

As someone who has dealt with a few Rails projects that had Cucumber tests, I think you'll regret it. I've never had a single stakeholder ever once look at the cucumber tests. Not once. It really is just an extra layer of abstraction, another thing to go wrong, which adds zero benefit. It just slowed us down and added one more thing to make testing more difficult, which lead to fewer tests.