The ticket queue grew every day. Never by much on any single morning. It simply never went down. The people working it were the hardest-working team in the building. That is where I started looking and it was the wrong place.
The professional services team was losing money. Everyone in it was staying late. Every visible sign pointed at them because every bit of the pain was landing on them. Under-resourced, I thought. Possibly not organised tightly enough. Perhaps the wrong shape for the client base we had won.
None of that was true.
The tickets were coming from our own releases. The engineering team was under real pressure to keep producing new capability, which is what a software business is supposed to do. So they produced it. The new feature worked. It also broke something else further down the product and that breakage arrived three or four days later as a fault on a desk in professional services.
Nobody in that story was failing. Engineering was hitting its measure. Professional services was hitting its measure. Both teams were performing well against the thing they had been asked to perform against. The business was still losing money doing it.
A business that measures activity cannot see rework. Every hour looks like an hour. The hour spent building something new and the hour spent repairing what last week’s release broke sit in the same column of the same report, staffed by the same people, described in the same language. One of those hours a client will pay for. The other one you absorb.
That is the whole mechanism. It is not complicated. It is simply invisible and it stays invisible for as long as the team absorbing it keeps absorbing it well. Hard work makes it worse rather than better. A team that copes buys you time to carry on getting it wrong.
The fix was ordinary. Nothing goes to production until it has been through the sandbox and passed. No exceptions, including for the things everyone was certain were safe.
Our chief executive wanted new features, which was the correct instinct for his role. Roadmap pace is what gets discussed with investors. It is what sales leads with. It is the evidence that a product is still alive in a competitive market. So I did not argue that we were shipping too fast. That argument is unwinnable and rightly so. Caution has no numbers behind it.
I argued the arithmetic instead. Feature velocity that generates rework does not add capability. It moves cost out of engineering and into professional services, where it cannot be billed and does not show up as a failure until the margin has already gone. We were not building faster than our competitors. We were building twice and charging once.
Then I made the point that actually carried it. If this continued, people would leave. Good people, the ones who had spent months fielding the consequences of somebody else’s deadline. We could not recruit fast enough to replace them at the rate we would lose them. By that stage it was not a quality problem or a margin problem. It was a question of whether the business survived the year.
The gate went in.
Two weeks later the ticket volume was down seventy per cent. Client confidence recovered faster than I expected. Morale recovered faster than that. The people doing the repairing had known where the damage was coming from long before anyone holding a P&L worked it out and now they could finally see the end of a day.
What stayed with me is not the number. The number is the easy part of the story. It is how long I spent studying the team that was suffering. Everything about that queue told me to look there. The overtime was there. The client complaints were there. The unbillable hours were there. The cause was three doors away, doing exactly what the business had asked of it and doing it well.
In a small company there is nobody arriving from outside with a fresh view of your numbers. You are engineering and you are professional services. The pressure that produces the rework and the effort that absorbs it are both yours, which makes the distance between them almost impossible to see from where you are standing.
The instinct will be to work harder. It is the instinct that got you here and it has never let you down before.
But you cannot outwork a problem that begins upstream of you. You can only find it, and finding it takes a different kind of attention than working does.
Field Note
Effort is only an answer when the problem is a volume problem. Most of the time it is not.
Try This
Take the busiest function in your business this week. Not the most important one. The busiest.
Work out roughly what proportion of its hours are spent doing something for the second time.
Then find out what created the need for the second time and follow it back until you reach a decision somebody made on purpose.