which bdd framework to mark your features “done” ?

Hi everyone,

It’s been a while I haven’t written a word. I’ve had a new job and had to catch up with concepts/techniques I wasn’t familiar with. Among them was the use of an acceptance framework on a regular (daily) basis.

I won’t talk about the one we use at work (Cucumber [2]) because it’s written in ruby and even after 6 months working with it I really don’t feel at ease with ruby. I can feel the power of it but for that very reason I know It will take me some time to reach even half the level I have java. After all I didn’t acquire my java skills in a only few months. I was positively surprised to find a really capable language, specially in the web area with it’s shinning Rails framework.

It’s certainly the main reason I started looking for a java alternative. And guess what ? I did not find. Why ? Because java is not suited for simple thing. You heard me ! Already getting upset ? Let me re-phrase. After many year of over-engineering you end up thinking like me : you want simple thing that work just fine.
The guys of a past job of mine really did a great job at opening my eyes. Guys (or even ex-guys) from Vidal if you read me I shall thank you. I had a fair programming level, and now I can say that I have a good programming level because I feel curious again of so many thing. A feeling I thought I had lost.

Well I did not find in java but I found in groovy. Groovy is really really fun because you can hardly get stuck while learning it because of its very high compatibility with java. This is a very important aspect to me. I don’t have that much time to learn new languages outside my work mainly because of personnal life (kids you know). So when you want to get somewhere “fast” (may seem fast but you’re a lot faster with groovy when used to it), you write it in java. But you loose all the benefit of groovy.

Anyway my research pointed out 2 potential winners : Spock [3] and Easyb [1].
But before losing sight (for an old javaist like me all that changes really strain my focus. I’m curious with so many thing but so little available time … never mind) of my initial requirements I wrote them down.

I was looking for a framework which :
– integrates well with maven : important because maven is, with jenkins, our software factory’s backbone. Automation is a key feature for the team.
– has a simple yet structured syntax : simple to allow a non IT Product Owner to write down its requirements just as he would with a post-it. Structured to have an homogeneous way of expressing requirements. It helps emerging an ubiquitous language
– leverages the classic given/when/then BDD paradigm
– cleanly separates behaviours definitions from their implementation : this allows for loose coupling. Moreover, business can feed the backlog regardless of the team velocity.
– allows advanced test techniques like parameterized tests, setup/teardown hooks, tags.
– has a fairly low memory footprint
– runs fairly fast

Let’s be honest : no tool meets those requirements. Even the most advanced (cucumber) one has its technical issues.

Cucumber is written in ruby. The framework’s author, Aslak Hellesoy, wrote a module (cuke4duke) which allows us, java users, to run it with maven ! But the cost is arguable performance and not so low memory footprint due to the whole stack : ruby + jruby + maven mojo.

I didn’t use spock but It appeared to me more like a bdd framework for groovy, not an acceptance tests tool.
What the difference ? The ability for a non IT to express a requirement in plain text. It really helps to focus on the business. I must admit I didn’t go too far with the tool. Its main strength is it advanced testing techniques. But I find them too close to programmer. It’s more like a simplified junit in groovy.
Any upgrade on my opinion on spock is more than welcome.

This leaves us with easyb. Let’s review the requirements above :

– integrates well with maven : check.
Below a maven plugin configured to run after jetty and output html reports

<plugin>
<groupId>org.easyb</groupId>
<artifactId>maven-easyb-plugin</artifactId>
<version>${easyb.version}</version>
<executions>
<execution>
<id>run-stories</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
<configuration>
<easybTestDirectory>${basedir}/src/test/stories</easybTestDirectory>
          <storyType>html</storyType>
          <storyReport>${project.build.directory}/easyb/stories.html</storyReport>
</configuration>
</plugin>

– has a simple yet structured syntax : check but incomplete (spock and cucumber do a better job) .
Below an example of requirement before the implementation details.

scenario "finding interest points around me should succeed"
    given "I am a valid system user"
    and "I provide the location '10 bd haussmann'"
    and "I send 'application/x-www-form-urlencoded'"
    and "I receive 'application/json'"
    when "I ask for interest points around that location"
    then "I should get a successfull response"
    and "The response should include 21 interest points"
    and "The response should include 5 restaurants"
    and "The response should include 9 pubs"
    and "The response should include 2 subway stations"

– leverages the classic given/when/then BDD paradigm : check . Cf. the above code.

– cleanly separates behaviours definitions from their implementation : this its the big drawback. The stories get written like plain text but the implementations are written in a closure, just after the definition. This isn’t ideal (cucumber does the separation).
Below, a requirement definition completed with its implementation

    and "I receive ${responseContentType}", {
        headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON))
        headers.setAcceptCharset(Arrays.asList(MappingJacksonHttpMessageConverter.DEFAULT_CHARSET))
    }

– allows advanced test techniques like parameterized tests, setup/teardown hooks, tags : check.
Below an example of parameterized test

examples "Number examples with #text and #number", [text:["12", "8", "20", "199"], number:numberArray]

scenario "Text #text should equal #number", {
  when "we parse #text", {
    parsedValue = Integer.parseInt(text)
    whenCount ++
  }
  then "it should equal #number", {
    parsedValue.shouldBe number
    thenCount ++
    numberTotal += parsedValue
  }
}

– has a fairly low memory footprint : check. after all everything gets compiled to java. Not the case with cucumber.
– runs fairly fast : check

Well you all guessed that I’m kind of found of easyb because I like simplicity.
Maybe I could find another reason : I’m new in the area and I feel less intimidated with a simple tool that I can understand. Maybe with experience and time I’ll feel less shy and I’ll need a more featured tool. For now this tool suits me.

[1] – For further reading on easyb I found those resources rather helpful :
– http://www.easyb.org/index.html
– https://github.com/easyb/easyb-core
– learning examples/where syntax
– http://groovy.dzone.com/news/easyb-introducing-conversation
– injecting test data with dbunit

[2] – For further reading on cucumber I found those resources rather helpful :
– To understand the philosophy
– to integrate with java
– to contribute or look for examples

[3] – For further reading on spock I found those resources rather helpful :
– google code former centralized resources provider
– github
– Mathilde Lemée’s articles (in french)