Web Development

This alarm's being rung for over a year now, so "calling it now" means finally reading the writing on the wall

This is very obvious unless you are in tech leadership, in which case your job is now to push this at all costs and suppress dissenting voices.

geohot

Now there's a name I haven't heard in a long time. George Hotz was the guy who first jailbroke iOS and the PlayStation 3 and made the towelroot exploit for early versions of Android, before legal threats drove him out of the scene.

They are not the automated from 0 to 100 coders that some people claim them to be. But they are quite capable, definitely much more capable than what anyone could have imagined ten years ago. Given well defined problems they can excel at even relatively complex tasks. I pointed Claude at a latex file of a somewhat complicated nonparametric statistical estimate calculation to look for any mistakes and it was actually able to find some. I then pointed it at a code that replicates the calculations and it was also able to correctly identify some issues with the code. I think this is the way one should use LLMs, not let it loose on coding tasks. In the former way you won't even be able to burn through your first tier account quota where as in the latter the LLM will likely end up getting in weird loops burning tokens like there is no tomorrow. Also this method of sane usage of LLMs is much more suitable for open local LLMs. I don't think there is any doubt anymore that LLMs can be very useful tools, not just for doing stuff but learning it too. People should move past the stage of invalid criticisms like "they are just stochastic parrots" and move to more serious matters like environmental impact, greedy fucking CEOs pretending LLMs are replacements for humans, degredation of skills, getting lazy at checking AI code, ethics of capitalizing on collective human knowledge and the unsustainable AI bubble that tech companies are pushing for.

if it's broken in a way that can't be detected, is it actually broken?

all software is broken in some way. if the rate of bugs generated by llm and the severity of those bugs drops below the rate you would expect from a human programming team, then llm is offering something competitive.

You know the feeling that you want to rewrite a project? But you know that most rewrites are a bad idea.

Be it your own, old code. Or code you inherited.

There is a small chance that the world realizes that they went in the wrong direction and nothing can get fixed. That will be the time of rewrites.

No, I don't expect this to be very likely. The agent code will remain, and human programmers get yelled at for not fixing it fast enough.

No one paying the bills cares

It's so nice to see some people speaking reason. If only any of those people ran multibillion dollar companies.

It's going to be a wonderful time to be a Freelance Senior Developer and above in a few years.

remember, when you interview for a job and they ask you, "do you have any questions?", you ask;

• has AI ever been used to develop your product?
• what percentage of your product has been written by agenetic AI?
• is the use of AI tracked as a performance indicator?

But they do work, maybe not as a full replacement but my god the amount of boilerplate I can avoid in creating unit tests from scratch. Extracting and finding information in the code base is also useful, not everything is an easy text search of tracing a few code paths. It's an incredible tool for these kinds of work.

If it becomes harder to tell the difference then it also means it's closer to matching reality. And todays AI can do very impressive "reasoning", managing to debug complex issues I have had.

The most important part is that you as developer is fully responsible and can stand behind what they do and deliver using AI agents.

Unpopular prediction: AI agents are going to get better at coding. Not great, but halfway decent at cranking out basic features. Once everything levels out in like 3-5 years, AI agents will be a cherished part of the toolbox most software developers. It will be useful for skimming code, it will be useful for tedious parts of tasks that are just a degree off from boilerplate.

People are definitely gonna try to use it for things more complicated than that, and that'll be a mistake, and it will be costly, but the far side of it could be pretty cool actually. Admittedly I have an optimistic disposition.

part of the issue as well is that when they get something completely broken, people just re roll the output until they get something that’s broken in ways they don’t notice. Or re roll parts of it, or tell the system to judge if the output is broken and re roll the parts that it judges are broken automatically. Or increase the size of the context window to get it closer to that upper limit of accuracy.

All this together can get a more functional output with less effort, and as people find these tricks it gives them the illusion of an upward trend in capability, like this is all solvable issues that will improve as time goes on. Big problem with that though, theses tricks and methods explode the compute cost rapidly. That’s all fine and dandy when everyone is getting their compute costs for these tools subsidized by these model providers, but eventually they will need to charge the real cost of running this. The compute providers that host the model providers are also running at a loss, trying to help grow the market segment and maximize their market share. And then places that have the datacenters in them are giving tax breaks and discount utilities to attract new construction.

Everyone except the people making the chips is selling at a loss, and as people pile on usage to make up for the fundamental limitations of these systems, the demand balloons, validating to the providers at all levels that this is a growing market they should invest more in to.

But eventually… they need to make money. The bill comes due on all the debt and investment. What happens to the people who have fully embraced these to run their businesses? Or to all the people who have built their skill set around using these systems? It’s a crisis, a series of crisis, each time a debt wall gets hit by someone in the supply chain. A half decade of technical debt that just got really expensive to deal with, and not enough experienced people to handle it, since all the grey beared retired and not enough new people got brought in to replace them because the entry level work was automated.