Only that the compiler works in a defined algorithmic way that can always be expected to work, at worst it uses more cpu registers than needed or something. AI on the other hand just spews garbage in a fundamentally statistical way and despite the enormous efforts to create tools that manipulate it into working more predictably, it still sucks so much of the time.
Another difference is that you are critically thinking when “instructing” a compiler via the code, but you only convince yourself that you think critically when you’re instructing an AI, it’s not the same and it actively makes you a worse engineer every time you decide to use it instead of thinking.
Cognitive Surrender. I can feel it happening every time I use this employer-mandated Cursor crap. I’m fighting it as hard as I can. Every AI slop pull request I have to review makes me die a little more inside.
(Edit: I’m glad this was so well received, but people are hating on my “assembler isn’t as difficult as people think” post below. Oh well.)
“Over the last few months, we have stopped getting AI slop security reports in the curl project,” said Daniel Stenberg, founder and lead developer of curl, in a social media post. “They’re gone. Instead, we get an ever-increasing amount of really good security reports, almost all done with the help of AI.”
It’s not the form that’s the issue. It’s the required depth of that thought, when you are actually doing the work yourself you need to go all the way in your thinking otherwise it simply won’t work. When you’re vibe coding, your thinking only goes as far as you see necessary so the AI has enough context to give something useful in return. It’s like comparing the thought process of an analyst and of the engineer that implements the spec, any engineer knows that difference.
but you only convince yourself that you think critically when you’re instructing an AI, it’s not the same and it actively makes you a worse engineer every time you decide to use it instead of thinking.
So, if you’re a lumberjack and you used hand tools to chop down trees, and due to the amount of time and effort required to chop down a tree with hand tools you traditionally thought carefully about how you’re cutting the tree before and while you were cutting it (not guaranteed - some lumberjacks still made stupid / unthinking cuts) - then along come gas powered chainsaws. Now you have options: you can spend more time thinking about the cuts before you make them and make even better - more careful cuts which make the process safer and easier to extract the logs after you fell them, or you can just go out and chop down 10x as many trees in the same time as before - thinking less than you did before because it’s just so damn amazing how many trees you can cut down in a day now.
Software development carries less immediate risk to the software developers than arborists experience in their field work. Software development also manifests many of its risks on a longer time horizon, more difficult to predict or even link to the development work. But the choices with AI development are very much the same: are you going to take the time to understand what your powerful new tool is doing, or are you just going to fire it up and show off how fast it does what it does?
Management comes into play in both fields, and bad management pushing either kind of field workers to cut staff and “increase productivity” by arbitrary (bad) metrics has been demonstrated over and over to produce bad results.
There are no such things as “AI experts” at this time - it’s far too new for that, much like “expert software systems developers” in the 1980s, the methods are just barely beginning to be developed. Applied judiciously, with proper care, it is already a powerful tool - but like they used to say even back in the early 1970s: “To Err is Human; To Really Foul Things Up Requires a Computer”
Not comparable at all. Power tools work deterministically. A powered chainsaw is not going to have a 0.1% chance of chopping a completely different tree on the other side of the forest. Of course accidents happen; your hand can slip. But a proper comparison would be if you got a computer to look at a large number of powered chainsaws and then generate its own in CAD based on what it’s seen, and then you use that generated power tool. Which, for something as potentially dangerous as a powered chainsaw, you most likely wouldn’t want to do, and would want to have careful human oversight over every part of design.
There’s a certain amount of random in how a chainsaw operates… will a certain cut cause the chain to be thrown or not? That has a lot to do with how you use the saw, maintain the tension on and lubrication in the chain, what you’re trying to cut, etc. and the same goes for the LLMs. They are based on statistics and “heat” of how far down the list of top results they choose for their answer, but you are the LLM operator, you choose what to do with its output, how much agency give it, how thoroughly you review and test its output before using it.
Inexperienced idiots would use no chain lube while cutting a 20" dbh standing hardwood with a 14" saw and just doing a straight plunge cut on the downwind / leaning to side of the tree where they will bind the bar inbetween the base and the rest of the tree if they’re lucky enough to even get that far. With enough perseverence they just might drop the tree on their house. It’s the same with LLMs, or traditional programming. Put the local high school chess club in charge of the ICBM targeting software? You get what you deserve.
I don’t agree. LLMs are by design probabilistic. Chainsaws aren’t designed to be probabilistic, and any functionality that is probabilistic (aside from philosophical questions about what it is possible to be certain about, YKWIM) is aimed to be minimised. You’re supposed to be able to give the same model the same prompt twice and get two different answers. You’re not meant to be able to use a chainsaw the same way on the same object and have it cut significantly differently. You’re inherently leaving much more to chance by using LLMs to generate code, and creating more work for yourself as you have to review LLM code, which is generally lower quality than human-written code.
And that’s a strength as compared with the machines that have attempted 100% determinisim since the days of Lady Ada and Charles Babbage. It also makes it a different beast which must be handled differently than a rigid machine.
You’re supposed to be able to give the same model the same prompt twice and get two different answers
Like creative writers. The Late Show monologue wouldn’t be very good if the writers used the exact same formula every night.
You’re inherently leaving much more to chance by using LLMs to generate code
Not if you give proper (complete and testable) requirements. I’d argue that LLMs are no more “unpredictable” than a pool of randomly selected human programmers.
This is where the power of diversity / randomness comes into play: with proper (complete and testable) requirements, the randomized agents, be they LLMs or consultants for hire, will iterate until they meet the requirements or give up / run out of time or resources.
and creating more work for yourself as you have to review LLM code
That’s a matter of practices - and code review is a good practice, but in the world where LLMs are writing the code - only LLM code reviewers are going to be capable of keeping with the flood of code that the LLMs are producing: https://youtu.be/pzkwn3hu1Cc?t=60
which is generally lower quality than human-written code.
That has been changing, rather quickly over the past year. The size of problems that LLMs are solving as well as humans has been increasing steadily for many months.
Now, having said all that, I’m paid to produce code, so I do review everything the LLMs make because nobody is asking me (yet) to make anything at super-human speed, so I’m not asking LLMs to make anything at super-human speed for the overall development process. I’ve been doing this for about 6 months, and the quality of what they produce, and the complexity of things they are able to successfully produce, has been steadily increasing throughout that time. Six months ago, I couldn’t ask an LLM to make more than a simple sub-module in things I was working on and get a reasonable result. Today, most things I’m tasked with - I can have the LLM develop a set of requirements for the whole problem statement, and then implement to those requirements, develop meaningful tests (six months ago the LLM generated tests were garbage, lately they’re getting to be on-par to better than what my human test department colleagues make), and do self-reviews and refinements to a point where the code meets our standards better than code written by our human programmers.
One of the most productive prompts you can give an LLM is: “Review these requirements; identify gaps, ambiguities, conflicts or any other problems that may hinder implementation. Report all findings and suggest potential corrections.” You won’t get the same result every time, and repeating that prompt in a fresh context window on the “corrected” requirements often leads to additional refinements, but eventually you do end up with a good set of self-consistent and complete requirements. The thing you have to do is read those (extensive) requirements and ensure that they reflect what you are thinking correctly, because any “hallucinations” that creep into the requirements will then be implemented in the code and the tests and sail right on through to the finished product.
I feel like this says more about neuroscience than it does about LLMs. :D
But seriously, my teams’ and individual experiences with LLMs have been mixed, at best. Even with advanced prompt training, the tools are just not there yet for our work.
I looked at LLM tools for software development a year ago, it was clearly unhelpful then. Showing some inklings of promise, but “just not there for our work” yet.
I looked six months ago and the advancement was dramatic, while it was helpful sometimes and not others six months ago it was clearly improving at an impressive pace. Mind you, I’ve been dabbling with “AI” since the 1980s, built a software neural net in 1991 and tried to make it do something useful back then, so… obviously what we’ve got now is DRAMATICALLY better and faster improving that it was waaay back like that.
Over the past six months it has become solidly “better” for a lot of uses than the methods it replaces. Now, I also notice big players like Google have been “enshittifying” their previous services for a few years leading up to this, so a lot of the “good stuff” I get from AI now is just what I used to get from basic search or “voice assistant” a few years back, but even ignoring that phenomenon - the frontier models really are better than anything that came before in a lot of ways.
Also, starting six months ago, I actively engaged in learning how to use the LLM based tools - and I believe much of the improvement I have experienced is due to me learning how to use the tools better, in addition to the tools themselves improving.
Only that the compiler works in a defined algorithmic way that can always be expected to work, at worst it uses more cpu registers than needed or something. AI on the other hand just spews garbage in a fundamentally statistical way and despite the enormous efforts to create tools that manipulate it into working more predictably, it still sucks so much of the time.
Another difference is that you are critically thinking when “instructing” a compiler via the code, but you only convince yourself that you think critically when you’re instructing an AI, it’s not the same and it actively makes you a worse engineer every time you decide to use it instead of thinking.
Cognitive Surrender. I can feel it happening every time I use this employer-mandated Cursor crap. I’m fighting it as hard as I can. Every AI slop pull request I have to review makes me die a little more inside.
(Edit: I’m glad this was so well received, but people are hating on my “assembler isn’t as difficult as people think” post below. Oh well.)
“Over the last few months, we have stopped getting AI slop security reports in the curl project,” said Daniel Stenberg, founder and lead developer of curl, in a social media post. “They’re gone. Instead, we get an ever-increasing amount of really good security reports, almost all done with the help of AI.”
https://www.theregister.com/2026/04/06/ai_coding_tools_more_work/
“If you’re thinking without writing, you only think you’re thinking.”
It’s not the form that’s the issue. It’s the required depth of that thought, when you are actually doing the work yourself you need to go all the way in your thinking otherwise it simply won’t work. When you’re vibe coding, your thinking only goes as far as you see necessary so the AI has enough context to give something useful in return. It’s like comparing the thought process of an analyst and of the engineer that implements the spec, any engineer knows that difference.
So, if you’re a lumberjack and you used hand tools to chop down trees, and due to the amount of time and effort required to chop down a tree with hand tools you traditionally thought carefully about how you’re cutting the tree before and while you were cutting it (not guaranteed - some lumberjacks still made stupid / unthinking cuts) - then along come gas powered chainsaws. Now you have options: you can spend more time thinking about the cuts before you make them and make even better - more careful cuts which make the process safer and easier to extract the logs after you fell them, or you can just go out and chop down 10x as many trees in the same time as before - thinking less than you did before because it’s just so damn amazing how many trees you can cut down in a day now.
Software development carries less immediate risk to the software developers than arborists experience in their field work. Software development also manifests many of its risks on a longer time horizon, more difficult to predict or even link to the development work. But the choices with AI development are very much the same: are you going to take the time to understand what your powerful new tool is doing, or are you just going to fire it up and show off how fast it does what it does?
Management comes into play in both fields, and bad management pushing either kind of field workers to cut staff and “increase productivity” by arbitrary (bad) metrics has been demonstrated over and over to produce bad results.
There are no such things as “AI experts” at this time - it’s far too new for that, much like “expert software systems developers” in the 1980s, the methods are just barely beginning to be developed. Applied judiciously, with proper care, it is already a powerful tool - but like they used to say even back in the early 1970s: “To Err is Human; To Really Foul Things Up Requires a Computer”
https://quoteinvestigator.com/2010/12/07/foul-computer/
Not comparable at all. Power tools work deterministically. A powered chainsaw is not going to have a 0.1% chance of chopping a completely different tree on the other side of the forest. Of course accidents happen; your hand can slip. But a proper comparison would be if you got a computer to look at a large number of powered chainsaws and then generate its own in CAD based on what it’s seen, and then you use that generated power tool. Which, for something as potentially dangerous as a powered chainsaw, you most likely wouldn’t want to do, and would want to have careful human oversight over every part of design.
There’s a certain amount of random in how a chainsaw operates… will a certain cut cause the chain to be thrown or not? That has a lot to do with how you use the saw, maintain the tension on and lubrication in the chain, what you’re trying to cut, etc. and the same goes for the LLMs. They are based on statistics and “heat” of how far down the list of top results they choose for their answer, but you are the LLM operator, you choose what to do with its output, how much agency give it, how thoroughly you review and test its output before using it.
Inexperienced idiots would use no chain lube while cutting a 20" dbh standing hardwood with a 14" saw and just doing a straight plunge cut on the downwind / leaning to side of the tree where they will bind the bar inbetween the base and the rest of the tree if they’re lucky enough to even get that far. With enough perseverence they just might drop the tree on their house. It’s the same with LLMs, or traditional programming. Put the local high school chess club in charge of the ICBM targeting software? You get what you deserve.
I don’t agree. LLMs are by design probabilistic. Chainsaws aren’t designed to be probabilistic, and any functionality that is probabilistic (aside from philosophical questions about what it is possible to be certain about, YKWIM) is aimed to be minimised. You’re supposed to be able to give the same model the same prompt twice and get two different answers. You’re not meant to be able to use a chainsaw the same way on the same object and have it cut significantly differently. You’re inherently leaving much more to chance by using LLMs to generate code, and creating more work for yourself as you have to review LLM code, which is generally lower quality than human-written code.
And that’s a strength as compared with the machines that have attempted 100% determinisim since the days of Lady Ada and Charles Babbage. It also makes it a different beast which must be handled differently than a rigid machine.
Like creative writers. The Late Show monologue wouldn’t be very good if the writers used the exact same formula every night.
Not if you give proper (complete and testable) requirements. I’d argue that LLMs are no more “unpredictable” than a pool of randomly selected human programmers.
This is where the power of diversity / randomness comes into play: with proper (complete and testable) requirements, the randomized agents, be they LLMs or consultants for hire, will iterate until they meet the requirements or give up / run out of time or resources.
That’s a matter of practices - and code review is a good practice, but in the world where LLMs are writing the code - only LLM code reviewers are going to be capable of keeping with the flood of code that the LLMs are producing: https://youtu.be/pzkwn3hu1Cc?t=60
That has been changing, rather quickly over the past year. The size of problems that LLMs are solving as well as humans has been increasing steadily for many months.
Now, having said all that, I’m paid to produce code, so I do review everything the LLMs make because nobody is asking me (yet) to make anything at super-human speed, so I’m not asking LLMs to make anything at super-human speed for the overall development process. I’ve been doing this for about 6 months, and the quality of what they produce, and the complexity of things they are able to successfully produce, has been steadily increasing throughout that time. Six months ago, I couldn’t ask an LLM to make more than a simple sub-module in things I was working on and get a reasonable result. Today, most things I’m tasked with - I can have the LLM develop a set of requirements for the whole problem statement, and then implement to those requirements, develop meaningful tests (six months ago the LLM generated tests were garbage, lately they’re getting to be on-par to better than what my human test department colleagues make), and do self-reviews and refinements to a point where the code meets our standards better than code written by our human programmers.
One of the most productive prompts you can give an LLM is: “Review these requirements; identify gaps, ambiguities, conflicts or any other problems that may hinder implementation. Report all findings and suggest potential corrections.” You won’t get the same result every time, and repeating that prompt in a fresh context window on the “corrected” requirements often leads to additional refinements, but eventually you do end up with a good set of self-consistent and complete requirements. The thing you have to do is read those (extensive) requirements and ensure that they reflect what you are thinking correctly, because any “hallucinations” that creep into the requirements will then be implemented in the code and the tests and sail right on through to the finished product.
LLMs surpass experts in predicting experimental outcomes. BrainGPT, an LLM we tuned on the neuroscience literature, performed better yet.
https://www.nature.com/articles/s41562-024-02046-9
I feel like this says more about neuroscience than it does about LLMs. :D
But seriously, my teams’ and individual experiences with LLMs have been mixed, at best. Even with advanced prompt training, the tools are just not there yet for our work.
I looked at LLM tools for software development a year ago, it was clearly unhelpful then. Showing some inklings of promise, but “just not there for our work” yet.
I looked six months ago and the advancement was dramatic, while it was helpful sometimes and not others six months ago it was clearly improving at an impressive pace. Mind you, I’ve been dabbling with “AI” since the 1980s, built a software neural net in 1991 and tried to make it do something useful back then, so… obviously what we’ve got now is DRAMATICALLY better and faster improving that it was waaay back like that.
Over the past six months it has become solidly “better” for a lot of uses than the methods it replaces. Now, I also notice big players like Google have been “enshittifying” their previous services for a few years leading up to this, so a lot of the “good stuff” I get from AI now is just what I used to get from basic search or “voice assistant” a few years back, but even ignoring that phenomenon - the frontier models really are better than anything that came before in a lot of ways.
Also, starting six months ago, I actively engaged in learning how to use the LLM based tools - and I believe much of the improvement I have experienced is due to me learning how to use the tools better, in addition to the tools themselves improving.