The power of feedback

How to Build a Robust LLM Application

Last month at Cherrypick we launched a brand new meal generator that uses LLMs to create personalized meal plans.

It has been a great success and we are pleased with the results. Customers are changing their plans 30% less and using their plans in their baskets 14% more.

However, getting to this point was not straightforward, and we learned many things that can go wrong when building these types of systems.

Here is what we learned about building an LLM-based product that actually works, and ends up in production rather than languishing in an investor deck as a cool tech demo.

The Job Is Not To Build

Startup CTOs or founding developers are the first technical people in the business. It is natural to think your job is to write code and build software. This is backwards.

Your first job is not to build software. Your role is to use your technical expertise to help the startup figure out fast if you have a valid solution to a compelling problem, and then a valid product for a big enough market.

You might do this through building software, but you might not need to.

Here is a story of how I did this wrong, and how you can do it right.

Your Code Is A Liability

Every chunk of code you commit is more for someone else to read, digest and understand.

Every complex “clever” expression requires another few minutes of effort for each of your team. They must now interpret what you wrote and why you wrote it.

Every line you add limits your project’s responsiveness to change.

Your code is a liability. Never forget this.

The First Thing A Startup CTO Must Do

Perhaps you are a technical co-founder who has managed to raise funding and you have been catapulted into the startup life. Perhaps you have just been brought in to handle the startup’s tech after the first round came in.

As the CTO, or the most senior technology person in the company, there are so many calls on your attention at this stage.

There is plenty of interesting new tech to build. There are potential customers to speak to (hopefully). There are investors to keep updated, who will want to know when the company is going to grow. There are people to hire. It can feel like you are drowning in possibility.

In the midst of all of that, we neglect this one thing at our peril.

Extreme YAGNI: How BDD nails your prototyping stage

Sometimes people don’t see the value in the BDD process. They contend that the BDD ceremonies are a waste of time, and get in the way of delivering real features to customers. Others cannot see how to apply BDD to their project, as no-one knowns exactly what the project will look like yet. As they’re only in the prototyping stage, by the time a feature file is written and made executable, it’s already out of date.

I don’t agree with this. If our process is set up right, we can prototype using just as effectively and retain the collaboration benefits that BDD gives us.

You Ain’t Gonna Need It

One of the biggest wins that Test-driven Development (TDD) gives us is the principle of YAGNI - “You Ain’t Gonna Need It”. It’s very tempting when writing code to go off on a tangent and produce a beautiful structured work of art that has zero practical use. TDD stops us doing this by forcing us only to write code that a test requires. Even some experts who don’t practice or encourage TDD often espouse the power of writing the calling code first in order to achieve much the same effect.

BDD gives us the same YAGNI win: but at a level higher than TDD. With the BDD cycle we’re adding thin slices of customer observable behaviour to our systems. If we can only write the code thats directly used by the business, then in theory we should be cutting down on wasteful development time.

However, there’s a snag here. if we’re prototyping, we don’t know whether this feature will make it into the final product. We still need to give feedback to our product team, so we need to build something. If the feature is complex, it might take a while to build it, and the feature might never get used. Why bother going through the process of specifying the feature using BDD and Cucumber features?

Happily, we can take YAGNI a level further to help us out.

Extreme YAGNI

Often in TDD, and especially when teaching it, I will encourage people to take shortcuts that might seem silly in their production code. For example, when writing a simple supermarket checkout class in Javascript, we might start with a test like this:

    var checkout = new Checkout();
    expect(checkout.total()).toEqual(0);

Our test defines our supermarket checkout to have a total of zero on creation. One simple way to make this work would be to define the following class:

    var Checkout = function() {
      this.total = function() {
        return 0;
      };
    }

You might think that’s cheating, and many people define a member variable for total, set it to 0 in the constructor, and miss this step out entirely. There is however an important principle at stake here. The code we have does exactly what the test requires it too. We may not need a local variable to store total at all.

Here’s the secret: we can practice this “extreme YAGNI” at the level of our features, too. If there’s a quick way to make our feature files work, then there’s nothing to stop us taking as many shortcuts as we can to get things working quickly.

For example, if we’re testing the user interface of our system via Cucumber features, one fast way to ensure things are working is to hard code the values in the user interface and not implement the back end behaviour too early. Why not make key pages static in your application, or hard code a few cases so your business gets the rough idea?

Again, you might think that’s cheating, but your features pass, so you’ve delivered what’s been asked for. You’ve spent the time thrashing out the story in a 3 amigos meeting, so you gain the benefits of deliberately discovering your software. You’re giving your colleagues real insight to guide the next set of stories, rather than vague guessing up front. Our UX and design colleagues now have important feedback through a working deployed system very quickly, and quick feedback through working software is a core component of the agile manifesto.

By putting off implementing the whole feature until later, we can use BDD to help us navigate the “chaotic” Cynefin space rather than just the “complicated” space. This in theory makes BDD twice as useful to our business.

Fast, fluid BDD

This all assumes that we have a fast, fluid BDD process, with close collaboration built in. If it takes a week to coordinate everyone for the next feature file, then the temptation is to have a long meeting and go through too many features, without a chance to pause, prototype, deliver and learn from working software. Maybe it’s time to re-organise those desks and sit all the members of your team together, or clean up your remote working practices, or block out time each day for 3 amigo sessions. You’ll be suprised how much small changes speed your team up.