R.I.P. Web 1.0

Layers of abstraction: writing great cucumber code

I blogged about Gojko’s thoughts on layers of abstraction a week or so ago, discussing three different ways we can think about the behaviour of any system. These way are: the business rules, the workflow of that system, and the specific activity the user is undertaking.

Today I want to think about how we can leverage these insights to help us write really good cucumber features and step code.

I used to write terrible features

Do these look familiar?

Feature: Paul pays employees
  In order to retain staff
  As Paul the Payroll Manager I want to pay my staff

Scenario:
  Given an admin user called "Paul" with a password "password"
  And a user called "Bob"
  And "Bob" is an employee
  And "Bob" gets paid "$2,000" a month
  When I visit the homepage
  And I click on "Log in"
  And I fill in "username" with "Paul"
  And I fill in "password" with "password"
  And I click "Log in"
  And I click "Payroll"
  And I click "Bob"
  And I click "Pay"
  And I enter "$2,000"
  And I click "Pay"
  Then I should see "Bob has been paid"

We’ve all written features like this in the past: there’s plenty of guidance out there these days to help you write better features than this. However, rather than just accept “best practices” at face value, let’s take a look under the hood and work out why this is better.

Variance revisited

Last time I discussed this topic, I mentioned the key differentiator was variance. Business rules are unlikely to change significantly unless the company decides to pivot: this is more likely in a startup but still less likely overall. The workflow is normally fairly static, but the activity the user follows changes regularly.

Co-incidentally, there are three levels of behaviour implementation that we write when we work with Cucumber:

• The feature files. Ideally there are written in collaboration with the customer and are written out before coding begins.

• The step definitions. We implement each step of our feature with ruby code as we are writing the feature, sometimes reworking existing steps to be more powerful (often at our peril).

• Support code. Cucumber executes our steps inside a ‘world object’, which we can easily extend through the adding of modules and methods.

Each of these implementation levels is also differentiated in terms of variance:

• The feature files are the most difficult to change, as this ideally requires a conversation with the customer, and any wording changes have a knock on effect on step definitions.

• Step definitions can be tricky to change, especially if they are used by multiple feature files. Their implementation is closely tied to the regular expression they match, which can make them difficult to understand if highly reused: one case where Don’t Repeat Yourself can fall down quickly.

• Support code is easy to change as it’s plain ruby and as such very malleable: we can easily refactor and be careful with our naming to tease out duplication.

Where to put the code?

If it’s easy to change support code, then it follows that we should put our higest varying code there: namely the code which describes specific activity. Normally only programmers are interested in this code and it’s easy enough to find and understand if the support methods are well-named.

The workflow code lives best in individual steps which aren’t often reused and which have simple regular expressions. The people who are interested in this area are normally designers and User Experience people, who should be able to read well-named ruby code at a pinch and therefore can understand what’s going on.

The code that’s least likely to change (the business rules) can safely live in the feature files with impunity, where it can be discussed with product owners. The product owner is most interested in the rules of their system: they’re only moderately interested in the workflow and usually aren’t too opiniated about the specific activities. That’s partly why we struggle to write features with our clients: if we’re trying to discuss activity specifics like in the example feature above, we’re probably nailing down details too early and bore our product owner to tears. It’s hard enough for a programmer to read these sorts of features: how can we expect anyone else to understand them?

An example

Given this, how would I refactor the feature above to improve things? After deleting web_steps.rb, I would rewrite the feature with my customer citing the business rule, rather than any specific workflow:

Feature: Paul pays employees
  In order to retain staff
  As Paul the Payroll Manager I want to pay my staff

@paul_signed_in
Scenario:
  Given an employee called "Bob"
  Then I should be able to pay "Bob" his salary

My step definitions would look something like this:

Given /^an employee called "([^"]+)"$/ do |employee_name|
  create_employee(employee_name)
end

Then /^I should be able to pay "([^"]+)" his salary$/ do |employee_name|
  pay_salary_to(employee_)
end

And the support code might look roughly like this:

Before("paul_signed_in") do
  paul = create_employee("Paul")
  paul.assign_role("payroll")
  sign_in_as(paul)
end

def create_employee(name)
  Employee.create!(:name => name, :username => name,
    :password => "password")
end

def pay_salary_to(name)
  payee = Employee.find_by_name(name)
  visit payroll_employee_path(payee)
  fill_ :salary, :with => payee.salary
  click_button "Pay"
end

You’re also free not to test the UI if you’d prefer not to in your support code. However, as we’ve given ourselves the ability to remove duplication, it’s easy to change the code when the UI changes. So far I’ve not found UI brittleness to be too much of an issue.

In conclusion

These are rules of thumb, but they can be very helpful in keeping the rate of development up as our codebase expands. One change that I’ve made recently to my own practice is to be more aggressive at pushing activity code down into support code, and it’s really helped to keep feature code flexible and easy to change.

Many people have given up on Cucumber, citing long build times and the brittleness of the test code as primary reasons. Obie Fernandez recently blogged about finding “high-ceremony” development too much work in a startup. I think that’s a real shame: it’s a fantastic way to drill down to specific behaviour and ensure you’re only building what you need. If you think about the behaviour of your system correctly, aggressively remove duplication in all your code (including test code), and only test code you own then you shouldn’t be burdening yourself with too much of an overhead.

Have you given up using Cucumber? Or if you use it, is this the way you do it or do you have a better method?

Layers of abstraction: combining BDD and UX

I first came across Gokjo Adzic’s thoughts on the different levels of UI test automation some time ago: it’s a really nice way to think not just about test automation, but about the different levels of behaviour in any software project.

I’ve been considering the similarities between these levels of behaviour and the user experience discipline, and how we might leverage that thinking to iterate towards a better way of combining agile methods with UX.

Three ways to think of behaviour

Applying these rules to any software system gives us three ways to think about the behaviour of that system.

The business rules. These are high level and abstract. To take the example of a Payroll system: the business rules represent such things as “Staff members always get paid on the last Friday of the month”, or “Temporary staff workers must submit a timesheet before being paid.”

The workflow. The workflow represents the logical steps a user might go through to fulfil a business rule. For example: “As a HR person, I want to see a list of temporary workers and pay those who are shown to have submitted a timesheet.”

The specific activity. The detailed steps a user goes through to achieve the workflow: “I click on the ‘show temp workers’ link; I see an icon next to those who have submitted timesheets, along with the last date they submitted; I click the ‘Pay’ button…”

The key differentiator here is variance. The business rules of a system are the least likely to change: changing these might represent a pivot and will incur significant development cost.

The specific activities change most often: perhaps at a designer’s whim, or through localisation or other text changes. We should therefore strive to ensure that changing the activities incurs as small a development cost as possible.

The agile / user experience process is similar

If you think about it, the three layers represent the different and progressive stages of thinking that we go through when designing the user experience of new applications.

Business rules of the system are laid down by the product owner at the start of a project. When considering the user experience of the application, we are careful to first understand a high level overview of the purpose of the software, getting as much useful information as possible from the product owner at an inception.

Workflow design is led by the UX team and happens when a new story is created. When creating a new slice of functionality, we carefully think through the workflow of that particular feature, using wireframing, personas or whatever works combined with lots of discussion. The net result is a basic step by step workflow of the new feature, without too much detail added.

Specific activities are created by developers and graphic designers. Developers and designers make a thousand little decisions about the user experience of the application as they build the feature, hopefully discussing their thoughts with the UX experts on their team if they feel out of their depth.

From general to specific

As we create features, we are iterating from the general to the specific; from the high level to the detail. To determine all the granular behaviour up front (and all the precise graphical designs) is inefficient: we are likely to change our minds about the detail. Yet this is what many user experience practioners and designers try to do: if not for the whole project, then for whole sections of the project.

For example, if you’re doing more than a dozen wireframes or so in advance, are you doing too much thinking ahead of time? Why not resist, have more agility, and let the completion of some of the features guide your future thinking? Likewise, I have often been presented with dozens of perfect photoshop mockups to code up, often without any clear direction on the behaviour represented within them. It is more agile to keep things as high level as possible for as long as possible. How about producing a guidance mockup and a style guide, and then sitting down and guiding the developers on the design when they come to build the feature?

This doesn’t mean you can avoid the detail. You need both ends of the behaviour spectrum: neglect the detail of the experience and you settle for mediocrity. Conversly if we neglect the high level our application becomes a ship without a rudder and the user experience will become confused.

In the future I plan to discuss how the use of Cucumber fits in to this, and how we can progressively iterate our cucumber features as we get more and more specific about a particular feature.

Cucumbers with personality

"Personality is everything in art and poetry."

– Johann Wolfgang von Goethe

One element of Cucumber feature-writing that is often neglected is the role. This is the section that sits between the well known In order that and I want to.

It’s very easy to concentrate on what we wish to accomplish, and why we want to do it. The role that we’re in, however, affects the way that we do it, and how the action is perceived as it is carried out. It is in fact, the key to user experience of the feature.

How often have we lazily written features like this?

In order to know how much money I am making
As a user
I want to see a report of widgets sold this month

What we want is clear, as is why we want it. What’s not clear is who is doing the asking, and therefore there’s no clue as to what the user experience should be like.

A sprinkling of personality

Next time we find ourselves writing “as a user”, let’s take a minute to stop and think whether we can be more specific.

Consider this feature:

In order to know how much money I am making
As the head of the company
I want to see a report of widgets sold this month

This guy is time poor, and just wants the facts, right now. Sure, he might want it to look good, but doesn’t care much beyond the numbers.

Now how about this one?

In order to know how much money I am making
As a design assistant responsible for collating reports
I want to see a report of widgets sold this month

This person possibly has a little more time on their hands, and their overriding concern might be to impress their boss. Therefore the aesthetics of the report layout might be very important, and it might not matter so much if the page is slower to load.

Personas driving stories

On a recent project for a coaching company we took this a step further. We produced some great personas during the inception (Bob Coachee, Jean Coach), and then went so far as to use them in our features:

In order to know who I am coaching next
As Jean the coach
I want to see a calendar showing today's appointments

Because everyone on the project knew the background behind these characters, the resulting features communicated a lot of knowledge. We all knew exactly how they were using the system, and what they needed out of it. We had about six personas in total, and they proved very helping in communicating the user experience of the feature to the team.

Do you create specific personas for your projects, and have you ever used them when writing your features? Did you see a benefit?

User Experience Is Everything

User Experience Design is a field which has been around as long at computer science itself, but has only recently come to the fore in the world of software development. While it’s been an important component of the work we do for some time, I’ve recently realised that it’s actually much more than that. It isn’t a component of what we do, it is the entirety of what we do.

How is a user to see the hard work we put in to fashioning a particular system, if it’s not through the use of that system? Their experience using the system is key to what they think of it, and the benefit that they get out of it.

When they encounter a bug with our software, it’s a failure of user experience. The “user” should not have “experienced” a bug in the system. When we build software, we try our hardest through all sorts of techniques and practices to create a system that is bug free. We are unconsciously thinking of user experience, even if we’re not aware of it.

Obie Fernandez recently gave a talk in which he made the statement “Perception is Reality”. He’s right. What a user perceives to be the case with the software they are using is what is real to them.

What use is it if our software is beautifully designed internally, but pig ugly to look at and difficult to use? The user’s perception of the software will be that it’s bad software. We might disagree with them all we like, but their perception is impossible to change, and we shouldn’t try to change it. Let’s instead change our software so that it’s perceived to be good software.

User experience goes so far beyond the common notion of “web design”. It’s everything about the site that the user uses and interacts with. It’s the feeling the user gets when they “open the box” - the login screen, the page load time, and the blank state. It’s the presence or absence of features. It’s the ease of accomplishing difficult tasks.

An important corollary to this is that it’s not desirable to build a piece of software that looks and feels beautiful, but internally is poorly implemented. It’ll become obvious, especially over time. The system might work after a fashion now, but inevitably most software will need to change over time. If the cost of change is high the user experience will eventually diminish, as the essential features will be difficult if not impossible to finish.

What does all this mean for web developers? We must pay the same attention to how something feels to use as we currently do to how it works. We should ask ourselves: does it move well? Do the pages and clicks that make up the feature flow just so? Is it juicy?

Is the user thinking about the task they want to perform, or how to make the system we’ve written do the task they want to perform? It’s a subtle yet crucial difference.

The last 5% we need to put into a feature will probably take us 50% of the total effort, but it’s worth it. It’s the last 5% that helps our web app rise above the sea of mediocre sites out there and be something truly great. It’s doing the last 5% that separates great developers from merely good developers. Who wants to be merely good?

User experience is everything. What can we do to put in that extra 5%, so our users can experience something wonderful?

How to Build a Robust LLM Application

Last month at Cherrypick we launched a brand new meal generator that uses LLMs to create personalized meal plans.

It has been a great success and we are pleased with the results. Customers are changing their plans 30% less and using their plans in their baskets 14% more.

However, getting to this point was not straightforward, and we learned many things that can go wrong when building these types of systems.

Here is what we learned about building an LLM-based product that actually works, and ends up in production rather than languishing in an investor deck as a cool tech demo.