How to evaluate developer work without reading code

Four measurements any non-technical founder can run, the startup I lost by writing deadlines in a notebook, and the cases where this ruler breaks down.

On a job site nobody asks if it is done: someone counts what is installed and signs off

You can evaluate developer work without reading a single line of code by measuring four things that live outside the sprint report: what is in production that you can open on your own phone right now, how long it takes between someone saying "it's done" and that thing being live, what broke afterwards and how long it stayed broken, and what the team warned you about before it went wrong.

None of the four needs technical vocabulary. All four need you to drop the question "is it done?", because that question has exactly one answer, and the answer is always yes.

• What is live: ask for the URL and open it yourself, on your phone, with nobody from the team sitting next to you. If it takes a developer to make the screen work, it was not delivered, it was demoed.
• The gap between "it's done" and "it's live": if finished code waits ten days for someone to ship it, your problem is not coding speed, it is the pipeline nobody built.
• What broke and for how long: one production bug tells you nothing. Four hours before anyone noticed tells you everything about the monitoring you are paying for.
• The warning that came first: who wrote it, in which channel, on what date, that this would slip. Risk that only shows up after the damage was never recorded, it was invented in the retro.

The question that cost me a company

The first measurement is opening it on your own phone, with nobody from the team next to you

Before Revin I ran a startup renting out construction equipment through an app. I was the owner, the salesperson and the guy driving to job sites. I did not write code back then. I had sworn off programming after a Java course at twelve.

So I hired developers and did the only thing I knew how to do: I asked for the deadline, wrote it in a notebook, and went back to running the business.

The sprint would end and the thing would not be there. Every single time.

For about three years I was sure the team was the problem. I swapped people, tightened deadlines, asked for more detailed reports, moved to two check-ins a week. The notebook filled with dates and the product stayed where it was. I went back to coding so I could understand what people were telling me, and that is when the uncomfortable part landed: nobody had lied to me. I just could not hear the answer. "About a week" reached my notebook as a promise. On the other side of the table it was an estimate with three dependencies hanging off it, none of them controlled by the person answering me. The company died. The ruler stayed, and I use the same one today from the supplier side, when it is my team that has to commit to a date.

On the job site I measured. In software I asked

The irony is that I came out of industrial mechanical assembly. I signed off as the responsible engineer for the steel structures of a racetrack and I helped build cement plants. On a job site nobody asks whether it is done. There is a progress measurement sheet: someone walks around with the drawings, counts how many tonnes of structure are standing, how many metres of piping have passed pressure testing, and signs off. Progress there is not an opinion, it is what you can point your finger at.

In software I switched that reflex off completely. I accepted conversation instead of measurement because I assumed measuring software meant reading code. It does not. It means picking the few things that only exist if the work actually happened.

And your vendor has a reason to prefer the conversation. Fixed-price contracts with no continuity attached reward whoever ships something that looks finished and then walks away. A vendor selling monitoring who hands you a health endpoint returning 200 and a colourful dashboard sitting on empty data is meeting the contract word for word. Test coverage at 92% with half the tests asserting nothing also meets it. From the outside it looks complete, and under the hood there is nothing. That only gets through because the buyer does not know what to measure, and the full bill arrives the day the vendor leaves.

The four questions I ask now, from both sides of the table

The notebook filled up with dates while the product stayed exactly where it was

They fit in a forty-minute meeting. The hesitation before the answer matters more than the answer.

• "Send me the link to what's live and I'll open it here." If you get a screenshot, or an offer to schedule a demo, you have your measurement already.
• "Last time something broke in production, how did you find out?" A good answer names an alert and a timestamp. A bad one says the customer called.
• "What in this plan depends on someone who is not in this room?" This is the question I did not know how to ask in 2017. Every blown estimate I have seen since had its answer hidden in there.
• "If I switched vendors in January, what could the next team not do without you?" That is where you find out how much of the system lives in one person's head.

None of them is a trap. With a serious team they speed the conversation up, because the team wants somewhere to record risk and usually has none.

"I hired people precisely so I wouldn't have to understand this"

That objection comes up in almost every sales call, and it is fair. You did not hire engineering to become an engineer. You hired it so you would not have to.

But there is a difference between understanding how the code gets written and knowing whether the work happened. You do not need to know how to weld to check the progress sheet on a construction site, and I have never met a developer of buildings who signs off on measurement without looking. In software, most buyers sign.

The cost of not measuring is not the cost of the mistake. It is the cost of the time it takes you to find the mistake. For me that was roughly three years. At US agency or staff augmentation rates, three years of measuring the wrong thing costs more than the product was ever worth. That is why I treat technical debt as a business risk line, not an engineering topic: it shows up in your dates and your revenue long before it shows up in the code.

AI does not fix this part, and it makes the disguise better. With an agent writing, screens stand up fast, and the feeling of progress arrives weeks before the progress. Measurement is still the only defence available to someone who does not read code.

Where this ruler fails

It does not work for the first two or three weeks of a product that is still being discovered. At that stage there is no "what is live", there are people testing a hypothesis, and demanding measurement gets in the way. It also does not work for a two-person team shipping a first version: the pipeline I ask for in the second item costs a few days of work, and on a six-week build those days might be the whole build.

There is a risk on the other side too. Measurement that turns into a daily accountability meeting is micromanagement with a nicer name, and I did that as well, back when I believed a more detailed report would save me. You want four measurements the team checks on its own, not four questions you repeat every morning.

I cannot tell you exactly where that line sits for your company. I know it exists, because I crossed it from both directions. There is more of how we work with that in what Revin does.

What to do this week

Open your last sprint notes and look for one sentence where somebody recorded a risk, with a date, before the risk happened. If three months of history contain none, your team's speed is not the issue. Nobody in your contract has any incentive to write down what is about to go wrong.

Which leaves one question to take back to your own company: when the next deadline slips, will you find out from a dashboard, from a customer, or from a notebook full of dates that nobody ever measured?