Comment stupid metrics (Score 2) 66
The only thing worse than not having metrics...
Big companies are almost always stupid.
Organizations measure what is easy for them to measure, not what actually matters. The drift between what matters and what gets measured always produces dumb behaviors.
Measuring token usage is perhaps a proxy for "is this person trying to use AI?". That question is only interesting if you are quite confident that at some point, your staff will be able to grow the business value they produce as a result of using AI.
What you actually should measure is business value delivered. Companies are bad at that (Because its difficult), So instead they Measure what is easy to measure, and they don't test the assumption that AI helps staff deliver more BV.
At my small software company, having developers use Claude is delivering tremendous BV. We are getting higher quality PRs, code reviews, and built software, and we are getting it faster. We are getting MUCH faster bug investigations. Overall time to fix is down. We are shipping faster and we are paying down technical debt.
(we may be accumulating new technical debt we don't see yet. time will tell)