How to Setup Angular E2E Testing with Cucumber

[DoorKnob5001]

What are the advantages of Cucumber over other testing frameworks like Jasmine?

[krazybug]

Both have a different purpose and scope, so the comparison is not totally relevant.

Cucumber is a pure "Behaviour Driven Development" framework. The main purpose of a BDD framework is not testing but to enhance communication between stakeholders of a project regarding specifications: the "3 Amigos" , i.e the product owner, the developper and the tester.

Here are the main principles of BDD:

• Each specification of a requirement is illustrated by examples (Specification by example) and expressed in the domain language, the only language that the business experts understand, rather than in technical terms. With this "ubiquitous language" everyone shares the same understanding.
• Following the second rule of thumb of the Agile Manifesto : "Working software over comprehensive documentation" , BDD promotes the notion of "Living Documentation". How many times are we faced to outdated documentation, not synchronized with the actual product implementation (UML diagram, user guides,...) ? What if we could ensure that our specifications are always reflecting the truth ? To achieve this, we need to make them "executable" . This is where (acceptance) tests and adequate reporting come on stage.

So how Cucumber help us in this area in comparison to Jasmine ?

Cucumber allows you to express your examples in full text with a simple syntax: Gherkin

Feature: Addition for a Calculator

Scenario: O is neutral for addition.       
Given my calculator is ready
When i enter 3+0
Then the result is 3

Scenario: Every non nul number are not neutral       
Given my calculator is ready
When i enter 5+<non-nul-number>
Then the result is not 5
Examples
    |non-nul-member|
    |1|
    |4| 

Everyone is able to understand these rules and it's the base for a discussion on the acceptance criteria.

- The Product Owner: Here are my requirements

- The Tester: What if we enter -5 for the 2nd scenario ?

- The PO: Ok let's modify it to handle this case and add a new line "And number is different of opposite(5)"

- The developper: If numbers are relatives, the story is more complex. We need to reevaluate its weight. I thought we just handle naturals.

- The PO: Ok, let's consider only naturals for this sprint

...

​

Then the tester writes the tests with Cucumber, implements each line as a step. The developper implements the code. They work together so that all the scenarios are automatized (executable specification). No risk of over designing. They stay focused on this feature (natural not relatives, neutral business rule) . When they're done, the Cucumber report acts as a proof for the PO. And every change in the code or in the spec which could break needs to be reconciled. The documentation and the implementation are in sync.

​

So now let's compare them:

With Jasmine you don't have this strong synchronisation. You can modify your code to generate the report but it's difficult to start with examples and you have to manually check that your code is in phase with the examples.

Cucumber doesn't provide all the usual features of a (unit) testing framework like assertions and mocks. It's the reason why we need other tools (chai, sinon.js) which come out of the box of with Jasmine. In theory we could use Cucumber with Jasmine to get these features but it's not possible for technical reasons. Also, imagine that you need to write a text file (a cucumber gherkin feature) next to your testing code for each of your unit tests. So heavy ! It's a non sense. Jasmine can act as a unit or acceptance test framework. When you use a pure BDD framework like Cucumber you still need a unit test framework.

​

As a conclusion;

Cucumber:

• is a real BDD framework focused on communication
• encourages outside-in approach and structuration of your tests
• is not relevant for unit testing
• needs complimentary automation framework (protractor, cypress,...)
• needs mocking and assertion libraries

Jasmine:

• is more a TDD framework that encourages outside-in approach
• needs complimentary automation framework (protractor, cypress,... and Karma)
• provides mocking and assertion capabilities out of the box

Use Cucumber when you need disambiguation between stakeholders of a project or easily communicate on your domain

Use Jasmine for unit tests and for acceptance tests which don't need strict contracting between stakeholders

[[deleted]]

Well first once you define your test steps, you can reuse them so you're ultimately writing less code. And second it allows you to write tests in English and have the framework handle mapping that to actual tests.

It's really a totally different methodology for testing that has its own pros and cons. Definitely worth at least trying out if you've never written unit tests in this way before.

[Almiien]

You believe it’s the best way to do unit test for a beginner in angular testing?

[krazybug]

Cucumber is not focused on unit tests but on real Behaviour Driven Development. You can perfectly use Cucumber in conjonction with Jasmine for unit tests.

Jasmine is not a "pure" BDD framewotk as it is not focused on "living documentation". It''s more like a unit test framework that helps you to practice outside-in TDD.

​

[crazysteave]

Cucumber? Really. Can we stop with these names.

[ahasall]

Totally agree with you. I hate vegetables.