Cucumber: keeping the build passing
One benefit we have when using BDD techniques properly is that we always have a failing acceptance test to guide us. The idea is that we write a failing feature, then make it pass by iterating using the standard red-green-refactor procedure. However, this is a challenge as well, as our acceptance test is going to be failing for most of the time that we’re working on the feature, and we have to decide how to handle that.
This is a problem because it’s good to make small commits into source control, sometimes only representing a few minutes work. This helps future source code readers to see the train of thought I went through when working on the feature, and allows us to use tools such as git bisect
to discover which commit might have broken something.
However, if the tests don’t pass for every commit, then our ability to use git bisect and other similar tools is more limited. If we don’t have a clean state from which to test, we can’t tell whether or not a build is broken for a reason that’s expected or unexpected.
What do we do at this point? Do we continue to make small commits to our codebase, knowing that we are going to be checking in code that breaks the build each time? Is it more important to have a detailed history of the change, or a clean history that always passes?
There are a few options here:
Check in the feature file at the end
I like this option the least, as it’s unclear what’s being worked on, and our source control history doesn’t show the code being driven out by the acceptance tests.
Go back and edit our commits later
Assuming we have a source control system that allows us to do this, we can always rewrite our history and insert the feature at the beginning of the change. Some have success with this, but it strikes me as dangerous and risky: there’s always a danger we might rewrite pushed history.
git squash
The squash command allows us to combine a number of commits into one when we’ve finished working on a feature. This will ensure that the one large commit passes, but then we’ve lost our “working”: our history doesn’t reflect our thought process.
Cucumber WIP
Cucumber can be run in a special mode called wip
mode. When we run cucumber --wip
, the tool will fail if any scenarios pass (you might want to read that bit again.)
Normally we run cucumber in this way in conjunction with the --tags
option, which ensures that only a set of scenarios run. Therefore, this command:
…will ensure that we only run the scenarios tagged @wip
, failing if any pass, and succeeding if they fail.
We can also run cucumber excluding certain tags, too:
This command will ensure that cucumber skips all of the @wip
scenarios for a normal run, which means that all the features should pass.
Armed with these tools, we can come up with a workflow which will allow us to keep our source code history, but also to ensure that our changes don’t break the build.
How to use Cucumber WIP in practice
Here’s an example of how this process might work:
-
Pair A start work on a new feature. They check out the code, switch to a new feature branch, and write a failing feature, which they tag
@wip
. They work on this feature, checking in liberally whenever their unit tests pass. -
Pair A push their code to their branch. They can push the code any time they like, as they’re pushing to a branch. Should anyone check out and start working on their branch, all the tests will continue to pass: the tests are run with a similar command to the following:
-
Pair B start another feature. They use the same process as above.
-
When Pair A’s feature is finished, they remove the @wip tag. Eventually the feature starts passing, and therefore the build fails:
@wip
scenarios aren’t supposed to pass. Removing the@wip
tag from the feature ensures the build passes again. -
Pair A push their code to master. This can be a straight merge, or if no-one else is working off the feature branch, (and we’re using something like git to do source control) we can rebase it and push it to master as a fast-forward.
-
Later, Pair C debugs an old commit. Pair C have found a bug, and they check out one of Pair A’s interim commits to find out if the issue presents itself in that commit. They run the build and it passes - the
@wip
feature fails, but build is set up to handle that. Pair C aren’t distracted by meaningless ‘work in progress’ failures and can get on with solving the problem they’re trying to fix.
A couple of important notes on this process:
-
We don’t check in failing unit tests. The build as run by a similar script to the one above will pass, even with a failing
@wip
scenario, but will not pass if unit tests are failing. Passing unit tests are a sign of basic code integrity: if any of them fail it’s very hard to trust the code at all. -
We don’t push @wip scenarios to the master branch. If pairs A and B in the example above were both working in the master branch, they would get the other pair’s failing scenarios when they attempt check their own
@wip
scenario. This introduces unnecessary noise into their work flow.
With a little build setup, and some effort and discipline on behalf of the team not to check @wip
features to master, this workflow can be used to ensure that the build passes at each stage and yet the history of the development of features is preserved.
Do you have a preferred way of solving this problem? How would you do it?
Share
More articles
Extreme YAGNI: How BDD nails your prototyping stage
Sometimes people don’t see the value in the BDD process. They contend that the BDD ceremonies are a waste of time, and get in the way of delivering real features to customers. Others cannot see how to apply BDD to their project, as no-one knowns exactly what the project will look like yet. As they’re only in the prototyping stage, by the time a feature file is written and made executable, it’s already out of date.
I don’t agree with this. If our process is set up right, we can prototype using just as effectively and retain the collaboration benefits that BDD gives us.
You Ain’t Gonna Need It
One of the biggest wins that Test-driven Development (TDD) gives us is the principle of YAGNI - “You Ain’t Gonna Need It”. It’s very tempting when writing code to go off on a tangent and produce a beautiful structured work of art that has zero practical use. TDD stops us doing this by forcing us only to write code that a test requires. Even some experts who don’t practice or encourage TDD often espouse the power of writing the calling code first in order to achieve much the same effect.
BDD gives us the same YAGNI win: but at a level higher than TDD. With the BDD cycle we’re adding thin slices of customer observable behaviour to our systems. If we can only write the code thats directly used by the business, then in theory we should be cutting down on wasteful development time.
However, there’s a snag here. if we’re prototyping, we don’t know whether this feature will make it into the final product. We still need to give feedback to our product team, so we need to build something. If the feature is complex, it might take a while to build it, and the feature might never get used. Why bother going through the process of specifying the feature using BDD and Cucumber features?
Happily, we can take YAGNI a level further to help us out.
Extreme YAGNI
Often in TDD, and especially when teaching it, I will encourage people to take shortcuts that might seem silly in their production code. For example, when writing a simple supermarket checkout class in Javascript, we might start with a test like this:
Our test defines our supermarket checkout to have a total of zero on creation. One simple way to make this work would be to define the following class:
You might think that’s cheating, and many people define a member variable for total
, set it to 0 in the constructor, and miss this step out entirely. There is however an important principle at stake here. The code we have does exactly what the test requires it too. We may not need a local variable to store total
at all.
Here’s the secret: we can practice this “extreme YAGNI” at the level of our features, too. If there’s a quick way to make our feature files work, then there’s nothing to stop us taking as many shortcuts as we can to get things working quickly.
For example, if we’re testing the user interface of our system via Cucumber features, one fast way to ensure things are working is to hard code the values in the user interface and not implement the back end behaviour too early. Why not make key pages static in your application, or hard code a few cases so your business gets the rough idea?
Again, you might think that’s cheating, but your features pass, so you’ve delivered what’s been asked for. You’ve spent the time thrashing out the story in a 3 amigos meeting, so you gain the benefits of deliberately discovering your software. You’re giving your colleagues real insight to guide the next set of stories, rather than vague guessing up front. Our UX and design colleagues now have important feedback through a working deployed system very quickly, and quick feedback through working software is a core component of the agile manifesto.
By putting off implementing the whole feature until later, we can use BDD to help us navigate the “chaotic” Cynefin space rather than just the “complicated” space. This in theory makes BDD twice as useful to our business.
Fast, fluid BDD
This all assumes that we have a fast, fluid BDD process, with close collaboration built in. If it takes a week to coordinate everyone for the next feature file, then the temptation is to have a long meeting and go through too many features, without a chance to pause, prototype, deliver and learn from working software. Maybe it’s time to re-organise those desks and sit all the members of your team together, or clean up your remote working practices, or block out time each day for 3 amigo sessions. You’ll be suprised how much small changes speed your team up.
Read moreWhy BDD works solo, and why that matters for everyone
“The monotony and solitude of a quiet life stimulates the creative mind.”
– Albert Einstein
It’s possible to play chess and many other games completely solo, without anyone else to play against.
Incredibly, our minds are able to take on the role of both players in a competitive match. Our brains allow us to place ourselves in the opposing player’s shoes. Even more impressively, we are able to hide certain information from our own decision making process – such as what we’re planning to do next turn. We can analyse this incomplete set of information and play a move based solely on that, even if it’s sub-optimal.
BDD can be done solo
Using Behaviour-driven Development (BDD) on our own is much the same as playing a game against ourselves.
Instead of playing the opposing role, we take on the role of the stakeholder. We put ourselves in the shoes of the person paying for or commissioning the project. Because the BDD practice of writing gherkin-style features forces us to think in plain English, we jettison the role of programmer for a moment, and drift in the non-technical solution thought-space.
Why is this important? It’s a valuable exercise for a couple of reasons:
-
We gain a fresh perspective on what we are doing. By taking our heads out of the code and putting ourselves into the role of the stakeholder, we are able to see the wood for the trees and get perspective on the code we’re writing right now. It might be interesting code to write, but does it matter?
-
We gain a fresh perspective on the stresses of others. Every developer should take the opportunity to be a stakeholder once in a while. It’s very helpful to see what life it like on the other side of the estimation table. If all the members on a team are able empathise with the constraints and pressures our colleagues from different disciplines have, our team will run much more smoothly.
BDD solo is why BDD matters in the first place
Sometimes we confuse the concepts behind BDD with the tools used to practice BDD, like 3 amigo meetings, story breakdown and Cucumber.
BDD isn’t about the tools we use, it’s about the communication in the team. It’s about having the right conversations at the right times, hammering things out together, airing and refining thoughts and ideas, beating out ambiguity and forging common goals. This is why it matters.
If it was just about the tools, BDD would have been lost in the noise years ago, and BDD solo would be an expensive overhead. Why add another layer of tool complexity if it’s just us?
However, because BDD is about the concepts, not the tools, BDD works just fine solo: the process of deliberately stepping outside our developer role and thinking through our project from a product perspective gains us insight and understanding. This works no matter what tools, languages, frameworks or practices we may prefer.
When we’re solo, we dispense with a lot of the tools we don’t need. We don’t need to schedule a 3 amigo meeting with ourselves, for example. We’re left with a very pure form of BDD, set free from the needed constraints of multi-person teams.
Then we can begin to understand which of the common practices of BDD are only there to work around problems and can be discarded as needed, and which are truly foundational.
Try it out
Try using BDD on a personal project. Play the product owner, and write some Gherkin-style features. Pretend you can’t code. Really think about your problem without thinking about exactly how you’re going to solve it.
Whilst doing this, reflect on what practices you really need, and which you don’t. Which tools are less useful on your own? Which are more useful?
If you feel we’re just wasting time, I’d challenge us to think about what that’s telling us about our own ‘standard’ agile practice. How much of it is truly making us more agile, and how much of it is simply getting in the way?
Read moreHow to layer context into your features using Real Narrative
Take a look at this Cucumber feature and see what you think:
Preambles don’t have to be boring
This preamble uses real fictional narrative to make its point.
The whole point of the preamble is to provide this context for your feature. This allows the business and the customer to get the development of the feature right. Why shouldn’t we include some actual fiction to give us a flavour of the feature that’s required?
This might seem easier with a space shooter game, but narrative on any project, even one that is more business focused, will help to get across a sense of how the feature should feel.
If we employ our reader’s imaginations, it’ll help them connect to the intangible reasons for the feature that we can’t quite fit in a concise agile story description.
Proper context drives implementation
The narrative above gives me plenty of context to write stories such as these:
It also raises plenty of questions. What happens if I push full power to two seperate systems? How should that feel to the player? Should that cause overheating? Should it even be possible? The narrative has driven a conversation with more depth and richness that I might not have had otherwise.
Behaviour-driven development is for everyone
Who says that BDD is just for business web apps? It can work for games, too, any any other kind of software project. We’ve shown that in any domain where technical and non-technical people have to communicate about a software project, the core tenets of BDD are useful.
Whether it’s software for a space shooter or a sugar factory, real people are always involved. There’s always a translation process from great concepts to carefully crafted code. We should make that process as painless as possible.
Let’s choose to be interesting
The things we often write down in our stories are often the least interesting part of the conversation. Sometimes, in a drive to be pithy, we miss the point. How about we take a chance and include some actual narrative in our feature preambles and our story writing?
How are you leveraging the power of preambles to add context to your features?
Read moreMake Cucumber features more readable with this one weird trick
“Sorry I’ve written such a long letter; I didn’t have time to write a short one.”
If you’ve got an issue with your customers not reading your feature files, then try this: it’ll help ensure their eyes stop glazing over.
As I travel around coaching teams doing BDD, a common problem I see with Gherkin features is that they are nigh on unreadable except by the people that wrote them… and even then, the writers often have trouble explaining what they mean.
This often happens when non-technical stakeholders have little to no involvement in writing features, leaving technical people free to be as obscure as they like. It’s easy to write an obscure feature: it takes a lot less effort to ramble and choose the first words that come to mind, rather than crafting carefully considered prose. The pain comes later: an obscure feature is imprecise, error-prone and unmaintainable, as the effort required to understand it will stop others from maintaining it.
How I help people simplify their features
When asked to review the language of a preamble, I read the feature out loud, and then ask the writer of the feature what it means.
A deep breath often follows. “Ok, this is the one where…” The writer will explain the feature clearly and concisely using completely different words to what is actually written in the feature. They use great contextual information that’s missing from the written preamble, and work to ensure I really understand what’s going on.
Once I’ve got a good response that I can understand, I ask the writer to replace the preamble with exactly what they just said.
It’s never exactly what they said (natural language is different to spoken language, after all) but it’s always better than what they had before.
Sometimes, when asked what the feature means, the writer will give a short hesitant explanation, and then proceed to read out the preamble again. My response is normally: “yes, but what does it mean?” Occasionally I have to ask five or six times until I get a considered, clear response.
This also works for scenarios, scenario descriptions, tag names, etc.
Why this works
Sometimes it’s easy for us to get so wrapped up in the detail of the features that we’re writing that we’re unable to see the wood for the trees. Our features read much more like computer code than they do plain natural language.
Features are communication tools first and foremost, and leveraging different ways of communicating while writing them will help to ensure they’re as carefully considered as possible.
(This works for natural prose, too. if you write a blog, documentation or even just emails to colleagues, taking a moment to read your text out loud will tell you in an instant whether it makes sense.)
A challenge
Pick a feature on your project, and read it out loud. If it makes no sense, explain what the feature does to a colleague and ask them to write it down for you. If it reads awkwardly and could be improved, take ten minutes to do so. Your future readers will thank you for it.
Read moreBDD Kickstart is dead. Long live...
We are coming up to the one year anniversary of the first public BDD Kickstart, and we’ve got some news…
Matt and I have enjoyed running BDD Kickstart over the last year or so. We’ve had hundreds of people go through the course in all, with courses run for the public and privately in-house.
Over the year we’ve grown a team of real expert practitioners who also happen to be fantastic at teaching and mentoring. Seb, Rob, Julien and Steve have been partnering with us to help us run many more courses than we could have done on our own. With Suzan publicising and marketing the courses, we have built up some great momentum: enough to expand our course lineup.
BDD Kickstart’s fundamental philosophy has always been to put you in a room with experts. With the addition of new partners, we’re growing the range of subjects we can offer at that very high quality level. We can offer courses in more than just BDD. So we needed a new name.
Introducing…
Quality matters to us enormously, so we’re growing slowly. We’re focusing on running our existing Continuous Delivery course, and we’re already running in-house Ruby courses. We’re writing a “Hexagonal” course to teach pure OO design principles for modern web applications.
We’d like your feedback please!
We’re actively seeking feedback to help guide us as we grow and expand. Your emails are always welcome, but we’d especially like to speak to you on the phone if:
- You have been on a course with us and you have some feedback or a testimonial
- You’ve been thinking about coming along but haven’t done so yet for specific reasons (location, can’t spare the time, financial, etc)
- You have an idea of a subject you’d like to learn from us that we don’t offer yet
Send us an email with your phone number or Skype handle and we’ll set up a call.
What’s next
We have two public courses available at the moment: we’re teaching our popular BDD Kickstart course in London in December and our Continuous Delivery course in Paris in March. If you’re interested in either of these courses, you can sign up for London now, or email us to express your interest in the Paris course.
It’s an exciting time for Kickstart Academy: we look forward to seeing you at one of our workshops soon!
Read more