The great thing about measuring developer productivity is that you can quickly identify the bad programmers. I want to tell you about the worst programmer I know, and why I fought to keep him in the team.
A few years ago I wrote a Twitter/X thread about the best programmer I know, which I should write up as a blog post. It seems only fair to tell you about the worst one too. His name is Tim Mackinnon and I want you to know how measurably unproductive he is.
We were working for a well-known software consultancy at a Big Bank that decided to introduce individual performance metrics, “for appraisal and personal development purposes”. This was cascaded through the organisation, and landed in our team in terms of story points delivered. This was after some considered discussion from the department manager, who knew you shouldn’t measure things like lines of code or bugs found, because people can easily game these.
Instead we would measure stories delivered, or it may have been story points (it turns out it doesn’t matter), because these represented business value. We were using something like Jira, and people would put their name against stories, which made it super easy to generate these productivity metrics.
Which brings me to Tim. Tim’s score was consistently zero. Zero! Not just low, or trending downwards, but literally zero. Week after week, iteration after i
42 Comments
franktankbank
Sounds like Tim was the manager. What exactly was the paid for manager doing?
sehyun
Discussion 2 years ago: https://news.ycombinator.com/item?id=37361947
croes
So the real metric is the manager who knows their team and what each member brings in.
JohnMakin
Used to be bitten by stuff like this until I figured out something Tim and this author apparently didn’t – the problem is trivially fixed by management by attaching tim’s name to any tickets he may have helped out on. he can ask his teammates to do this and they gladly will, or, nice teammates will usually throw a “figured this out with the help of @Tim” in the ticket. goes a long way to keep “tim” on your team against obtuse velocity metrics like this.
productivity metrics aren’t entirely worthless. if I come into a team, for instance, and I see they have 1 PR mapped to roughly every 1 jira ticket, and I see a guy on a 3 person team that’s got 70% of the PR’s in a year, that isn’t a clueless piece of info. he’s probably the lead. not always, and people game stuff like this, but it is a data point you can make note of.
AGorilla
[dead]
eckesicle
These sort of stories seem to be dime a dozen and weirdly celebrated around HN and the software engineering community.
We’re told of the hero, who goes against their managers and executives and doesn’t deliver any stories as agreed in sprints.
We’re told of the engineer who isn’t hired by Google because he can’t invert a binary tree. Everyone else piles on and decree that, yes indeed, you cannot measure developer efficiency with a Leetcode or whiteboard problem. We’re too good for that. Another engineer chimes in: “I don’t test my candidates. The best people I worked with were hired over a beer and a chat at the local pub”
We’re told of the MBAs who destroy the organisation, by introducing evil metrics, and how that the work we do are immeasurable and that the PHBs don’t understand how great we are. 10x engineers aren’t a real thing, everyone is equally productive in our digital utopia.
Meanwhile in the real world, hordes of awful engineers deliver no story points, because they in fact, do nothing and only waste time and lowers morale.
Meanwhile in the real world, each job opportunity has thousands of applicants who can barely write a for loop. Leetcode and whiteboards filter these people out effectively every day.
Meanwhile in the real world, metrics on delivery, features and bugs drive company growth and success for those companies that employ them.
To me, all these heroes, and above process people, just strike me as difficult to work with narcissists who are poor at communication. We are not special, and we do not sit above every other department in our organisation.
bsindcatr
Some things that don’t measure whether a developer is “good”:
– # LoC added, changed, removed
– number of points earned in a sprint, when those points aren’t quantitatively indicative of business value, and they never are
– number of on or off-the-clock hours schmoozing with others to solidify relationships with the business and “play the game”
– number of times they “sound like good developers / intelligent people” in meetings or presentations
– number of weeks they spent on really complex problems they worked to solve when they could have provided more incremental value earlier much more quickly
– number of solutions they provided the company quickly while leaving many times more LoC to maintain
– number of hours they spent honing code, formatting, updating to the latest versions but doing so for their own edification and preferences rather than focusing on the team and the business and what would help them
and so many more…
mjburgess
I've had some thoughts on programming practice somewhat related to Tim's role, and some on language design this morning.
I write code for others to read a lot, to explicitly teach them how to do something. This code has always been overly verbose, "decompressed" and teaches people the thought process behind a solution above being a "neat" solution. I was dragged a little reluctantly into this style, by being forced to anticipate all the ways the code may be misunderstood ahead of time — by all the ways it was misunderstood previously. Its much less work for me to be verbose up-front, than fix misunderstandings later.
After watching and looking at some of the best systems programming code/coders — I've come to think this is just the best way to program in general.
The latest program I wrote this way, and the quality of code is vastly higher. No abbreviations, any unusual coding practice (eg., conditional imports, using classes as namespaces rather than object templates, etc.) I noted briefly in comments to highlight it's unexpected. Any functions which are domain/app-specific have execessively long names which fully describe that step/asepct of the solution. Extensive design comments where my thinking is noted. etc.
Programs which teach their readers how to think about the solution, rather than how to understand the code — and teach it to a "smart junior" rather than to an expert. I think this is different than literate programming, and of the maxim "code is read more than written" — it's a specific "educational ethic": code isnt clean, or literate, or readable — its educational.
If this is the best way for "programming in the large", then a property of programming languages follows: languages which enable the programmer to "decompress" their thoughts over many lines are preferable to ones which hinder this. This I think then explains something about the uptake of functional languages — which resist this decompression of thoughts over lines of code. They are often sold on their conciseness — as if a 200 line C++ ought be reduced to a 10 line F# program. This trades the wrong sort of productivity: long-lived large programs require programmers to build mental models of them far more often than they require them to add to them at-scale. A programmer self-teaches the code base more often than they write it: this isnt about reading.
This goes somewhat to the role of Tim in OP. Perhaps the right software engineering programming practice is to write code as-if you are pairing with a new programmer in the future. To verbalise everything Tim would say, already, in the code.
failrate
The worst programmer I know literally could not go a day without either checking in code that did not compile or saying something creepy and sexual in a large open plan office, and we all worked together to get them fired.
decGetAc
How did the manager not know Tim didn't have tickets slated for him ? How did Tim not even pick up some (at a lower capacity) and then still helped with the remaining time ?
I guess as someone who does a lot of the same that Tim is doing,and I bet others can resonate, I still "have to" pick up tickets and I think that's always the expectation in any job I've had as an IC. Is Tim managing his time well ?
gravy
It never occurred to me that a rebuttal to "not using lines of code or bugs solved because it can be gamed" is just to point out productivity is literally always gamed
NalNezumi
Glad to hear that Tim stayed and author managed to steer the entire process towards the right direction. Which requires a listening manager.
I experienced the "bad ending" of this productivity metric trickle down: OKR. This startup wanted not just team-based 3-month review Objective Key Result but also individual one, and on top of it, tied stock option to OKR. It was a robotics startup so very cross-domain teams (Software, Hardware, Embedded, DevOps, HW design, HW testing, HW maintenance etc etc).
The result? Developers became lonely islands. There were no "Tim" anymore. When I (Software Integrator) encountered an issue, that I just couldn't figure out but had a hunch it must be a deep-rooted issue, went to the expert (control/kinematics) for feedback. The answer I got was "I'm sorry I really want to help you but my OKR deadline is very close, and I simply don't have time". He could've probably fixed it in a day or less, but it ended up taking 2 weeks.
The problem turned out to be quite deep: inside layer upon layer of C++ mega monorepo, I found that boost library and a custom kinematics library had implicit struct copy and the different libraries (more than two) used different order for representing translation & rotation (xyz, rpy, Euler, Quaternion) and all of the ordering of each components were different. Somehow over 2 years of operation nobody got troubled by this until our new team had to use it.
Afaik I reported it to the Software team, but again, because OKR, nothing was done about it.
lapcat
Was Tim hired as a teacher or trainer? What I find strange is that it sounds like Tim was not doing any individual work.
As a very experienced programmer, I'm sure that I could increase the productivity of other programmers by pairing with them all day. However, is that actually the best and most productive use of my time? In my estimation, I think that overall team productivity would be maximized if I spent approximately 20% of my time pairing with others and 80% on individual contributions. (That's a rough estimate; it could be 25/75 or 30/70.) The article said that Tim was "patient", but perhaps he was too patient, wasting a lot of his time that could be spent accomplishing other things.
readthenotes1
Kinda concerned the author thinks story points correlates with biz value….
RKFADU_UOFCCLEL
Very good article, managers just want short summaries of what's going on and sometimes these summaries come in the form of bad metrics.
_fat_santa
Honest Question: Are there any metrics around developer productivity that actually work? Reading stories like this and others over the years I've come to the conclusion that you simply can't measure developers productivity on a granular level, it's just about the final product. However, I would love to be proven wrong.
dosinga
Tim would do well in these times of LLMs. They can write code faster than anybody but also need more guiding and coaching than anybody. They don't learn anything no matter how much we call it machine learning. These days what we most need is a Tim of Tims – somebody who coaches software engineers into becoming Tim.
darthrupert
Damn. Now that I read this I realize why my current team feels like nothing gets done and the reason is the team has no Tim.
easyThrowaway
I mean, I've worked on some companies where management would've seen what Tim was actually doing and nonetheless reply with "We're not paying him to hang out with other developers, we pay him to sit at his desk and code!"
Vox Jira, Vox Dei. Make sure you're not working in a company with such kindergarden-like approach at managing people before attempting anything like that.
Beijinger
This reminds me of the chicken problem:
https://medium.com/heroicpresence/the-super-chicken-study-pa…
Waterluvian
When you look at damage per second graphs and conclude that all your healers need to be kicked from the raid.
It’s so difficult to quantify the value of “support” but they’re indispensable. I have yet to really find a sensible way to quantify it. It’s ultimately just something that you need to trust leaders to get right.
pipes
I've never worked on a team where story points aren't translated into time. I've found story points to be a completely useless level of indirection. Genuine question, has anyone here ever found them useful?
cornel_io
There are a couple important things to also keep in mind:
First: just like there can be individuals who lift up an entire team but are not ticking off tasks themselves, there can be apparently individually productive team members who slow the entire team down for any of a number of reasons. In my experience it's usually either that they are a) fast but terrible, b) have really weird coding styles that might not be bad but are very difficult for others to work with (architecture astronauts and people who barely speak the language often fall here), or c) are process bullies who try to set up the entire review system to enforce their preferences in a hardline way that suits them but delays everyone but them. Each needs to be dealt with very differently, to varying degrees of success, but my honest opinion at this stage is that no matter how productive these people seem by themselves it's mostly harmful to have them on a team. Behavioral issues in senior people tend to be really tough to adjust, and take a lot of energy from a manager that is better spent helping your top performers excel; that said, if you can get them to adjust sometimes it's worth the effort.
Second: pair programming works great for some people, but it is terrible for others. I've measured this on teams by trial and error over fairly long periods, and unfortunately it's also the case that people don't segment neatly based on their preferences, so the obvious "let them choose" method isn't ideal. There are pairs of people who really love pair programming and desperately want to do it all the time who are almost fully 2x as productive when split apart instead (yes, including downstream bugs and follow-ons, meaning that they really are just having two people do the job of one), and there are people who hate pairing who see similar multiples by being forced into it even though they hate it. My rough intuition is that there are two independent variables here, a social factor that measures how much you enjoy pairing, and a style factor that determines how much benefit you derive from it, with the two not correlating much at all. There might even be a slight anticorrelation, because the more social people who love it have already naturally done as much knowledge sharing as is helpful, and the people who hate it are underinvested there and could use some more focus on the fact that they're part of a team.
karaterobot
> You see, the reason that Tim’s productivity score was zero, was that he never signed up for any stories. Instead he would spend his day pairing with different teammates.
I'm very glad this essay turned out to take this position, because when I read the first couple paragraphs, I was ramping up to write a mean internet comment about how stupid measuring developer productivity through story points is.
A friend of mine is exactly like this Tim guy he talks abou: he spends time helping other people fix their problems, or takes extra time doing things the right way. His personal velocity suffers, but the team and the code base is greatly improved on net. He was put on a Performance Improvement Plan at one company, and told his job was at stake at another as a result. No question it's kept him at a Senior level when he should be higher at this point in his career. I sort of take it personally, because I worked with him at the company that gave him the PIP, and both he and I quit as a result.
When I see companies trying to measure developer productivity (and tying it to employment and advancement) I think about Seeing Like a State by James Scott, and the concept of legibility: imposing by force a set of rules on a complex, messy system to make it easier to steer from some office far away. Makes it very convenient to managers, and often leads to disaster.
noqc
Goodhart's law is not a joke. There is no standin for value.
dmoy
Yea the best teams I've ever been on have always had someone like Tim.
The best best teams I've been on, Tim's helpfulness osmoses onto other people, and instead of Tim always being the one, the process goes like this:
1. Dev gets stuck
2. Dev attempts to unstick self for appropriate amount of time given problem and probability of self-unsticking given resources (e.g. some stuff is easier to search for internally, or dismantle and go piece by piece)
3. Dev announces "hey I'm blocked on XYZ thing"
4. Whoever knows the most about that topic (not always Tim, but often Tim) puts that as like their highest priority thing, and almost always jumps in right away to help (unless they're like ignoring chat cus they're in the zone)
Works great especially if you do #4 at some break time (like say lunch or standup), and everyone has enough things to work on that they can do their own internal CPU pipelining and work on other stuff until someone has time to help unstick them
morsecodist
The idea of measuring individual developer productivity is kind of absurd to me. I'm not saying that what we do is magic, there are just so many variables.
Measuring story points or lines of code is kind of the opposite of productivity. This encourages developers to do as much meaningless work as possible. I'd want a developer to make a task simpler or use an existing tool which means less time writing code. The value of them saving that work from knowing another way is high but hard to measure.
What you want is measuring business outcomes but those are hard to attribute to a particular developer.
I think unfortunately we're left with our subjective judgement here. I think we'd do better admitting to ourselves that we can't measure this than to pretend we have some sort of science here.
markus_zhang
A much easier solution is just assign a team improvement ticket to him and call it a day.
I doubt he never worked on any tasks though, so maybe something for him once for a while.
ChrisMarshallNY
I think this was posted here a year or so ago.
I love this story.
ewalk153
I love a good pairing ladder. While there is no absolute good measure of productivity, a suite of thoughts observation can provide at least tripwires.
One major problem with tools that ask crafters to do data entry to show their value is that best people are most likely to refuse. You really need to focus on tooling to help capture what’s going on without toil.
For example, if your folks are remote and they use software to aid in pairing (eg Tuple), script the system to log the tuple sessions and perhaps even capture the pairing in the commits made together.
This can be used as an input to bring visible to be best leaders in your org.
tasn
I get the point that they are trying to make, but Tim should try to find a way to ship things. It sounds like he could ship a lot and make the codebase better for everyone if he did.
wwarner
Hm. In the agile world, non-coders don't typically sign up for stories. So maybe this person shouldn't have been expected to land stories, or possibly there wasn't room in the budget for someone to be just a peer coder. I personally like the story paradigm as a way of working out (and then sticking to) priorities, and I love it when managers and principals work on stories like everyone else. Also, in the remote work context, everyone has to work harder on figuring out the right thing to work on, and stories are a decent way to achieve that.
ibash
Maybe.
There’s also folks that pair because they’re a crutch for one or two other engineers. The other engineers never improve or are let go, but softly slow down the team.
There’s also the folks that pair because their code doesn’t make sense on its own. Or they have some config files they’ve refused to check into the repo, etc.
Pairing makes them feel valuable.
hansmayer
I think a lot of this idiotic thinking about measuring productivity through "stories" is to pin on the "Agile" industry overselling their low value "methodologies" to the MBA-type managers, which are now unfortunately just about everywhere. Hence you have some clueless person deciding who has to go, based on how many tickets get moved on the board. As Joel Spolsky once said, no software company can succeed unless a programmer is at the helm.
andix
The title should be: Why measuring developer productivity by story points is bad.
I was once part of a team that had a zero-storypoints developer too, let's call him Zero. He always refused to get any stories assigned during planning, his goal was always zero. He felt responsible for delivering all the planned stories and helped the people who struggled. He often came to me and told me to help someone on a specific task, because it was too much for him to help everyone.
This was well understood by management though. Zero took a 4-6 week vacation every year, and management always pushed important releases back if Zero was not there. The team could've done it without him, and it might've been better for the team not to have Zero around for all important releases. But management was scared something could go terribly wrong without him.
aaws11
previously on hacker news: https://news.ycombinator.com/item?id=37361947
m463
Reading this is similar…
https://www.folklore.org/Negative_2000_Lines_Of_Code.html
casey2
This just seems like you performed a social engineering attack on your boss to keep someone on the team who was objectively performing poorly but happened to be doing something that is valued in western culture.
In short you replaced one well defined metric with an adhoc one following your bias.
You are a peon. For all you know your local improvement has a negative effect on the whole company. Listen to your boss and let the company fail fast if it has an unfit system.
slowtrek
Moral of the story:
Anyone that looks for the metric to find the weakest link, is the weakest one at heart.
iamleppert
Counterexample: Not everything needs to be a group activity or social. If a developer of average skill can't manage to deliver a feature without the constant help from a "Tim", it says something about your code, processes, team. I've worked at places where they used pairing as a signal the team was working well together, when the reality was the code was so brittle and full of hacks it took the combined help (usually moral support) of a Tim to help support through every little change.
Every team is different, but software development needs the time and space for individual work just as much as group work. A small team who needs a dedicated person for pairing, who also doesn't (or won't) do any IC work, is a huge red flag.
tzs
I had a sort of similar situation. I was one of two senior developers at a company making utility software for Windows and Mac in the late '90s and early 2000's. The other senior developer was the owner and CEO of the company and had to spend most of his time doing owner/CEO things so was only able to spend maybe one day a week developing.
The way we organized most of our Windows programs is that I'd write a non-GUI core that implemented all the underlying system level stuff we needed, and then the junior developers would write a nice Windows GUI that used that core. If the product needed anything like a VxD, a filesystem hook, an LSP, or any other Windows kernel extension I'd write that too.
We were profitable and there would be profit sharing bonuses quarterly (I think…might have been monthly but that doesn't significantly change what follows). The owner was always tinkering with how to decide how much profit sharing each person got.
One scheme, which he was sure was going to be great, was to take the total amount available for profit sharing for a quarter and divide that by the number of employees. Call that amount "1 share". So if there were N employees there were N shares available.
He wanted those shares to be allocated so that 25% of the employees got 2 shares, 50% got 1 share, and 25% got 0 shares. Who is in what group would be determined by a company wide vote.
Ballots were distributed that listed all N employees in alphabetical order, and we were told to write a 2 next to N/4 names, a 1 next to N/2 names, and write a zero next to the remaining N/2 names.
For each person their numbers from all the ballots were totaled, and the 1/4 with the highest totals got 2 shares, the 1/4 with the lowest totals got 0 shares, and everyone else got 1 share.
In addition, the people with the 8 highest votes would be put on a committee that would advise the owner on the direction of the company.
The owner's expectation when he launched this was that I would be the top vote getter nearly every time, and would always be in the top 8 so be on the committee of 8 which he was planning for me to run.
I ended up not being in the the top 8. I wasn't even in the top 25%. I don't remember for sure, but it was either somewhere in the bottom half of the 1 share group or it was in the 0 share group.
The reason was simple. Although I wrote the core functionality of all our products my role was not really visible to people other than other developers. It was the junior developers who wrote the GUIs. It was the junior developers who did most of the interaction with the testers. When a test found a bug they'd report it to the junior developer who was, to the testers, the face of the project. If the bug turned out to be in code the junior developer would let me know.
So every developer put me down for 2 on their ballot, but to everyone else I was just some guy in development doing some unknown work so I didn't get many votes from anyone else.
At least the owner immediately recognized that this profit sharing scheme was flawed and dropped it. Since he reacted so quickly I didn't gloat too much over the fact that this was exactly what I told him was going to happen when he first proposed this scheme.
(Some gloating was necessary because the owner and I had been best friends since we met in college about 15 years earlier, and the obligation to rip on your best friend when they do something stupid after you told them it would not work is stronger than the rule that you shouldn't tell your boss "I told you so!").
umvi
So there wasn't a way to capture the fact that Tim was helping with so many JIRA stories in a way that makes Tim's work more visible? Like maybe pair programming stories that Tim owns and moves on the board (assuming Tim's time is best used in a mentorship capacity)?
It kind of rubs me the wrong way that Tim can just fly under the radar and do whatever he wants without any paper trail or accountability while the rest of the team has to clearly convey what they are working on