Ask HN: How much efficiency improvement would you estimate Copilot gives you?
I work at a tech company with about 400 employees.
The company rolled out access to GitHub Copilot for everyone about a year ago.
After a year they did a survey of employees who used it and asked to estimate what percent efficiency gain they get from using it.
What surprised me is the average across responses was around 10% gain in work efficiency.
With all the talk of AI replacing programmers I was surprised to see it that low.
Does that match your experience?
I've tried to calculate this.
I used copilot for 14 days and every time I used copilot and found it benefits me (saved me time), I wrote a check mark on a sheet of paper. Every time I tried to use it and it led me down a rabbit hole or simply didn't do much, I wrote a minus sign.
In the end there were roughly the same number of checks and minuses. Granted-- it wasn't the best experiment in the world and quite subjective.
10%-ish is about right. In Fred Brook's No Silver Bullet essay he makes the point that software development has multiple phases, say:
* requirements gathering
* design
* coding
* testing
* deployment
and suppose these all make an equal contribution. If you managed to make a tool that can do the coding in 0% of the time you still have to do 80% of the work!