Just to make this clear up front: I was always very skeptical of LLMs (I'll never call them AI) and still see them as fundamentally limited and unreliable. See my post history if you don't believe me.
Having said that: I work in business software and the models have progressed to the point where typing code and writing classes and functions by hand has become uneconomical. We review the output closely to avoid the idiocy of "vibe-coding", but the quality has improved to the point where we rarely have to outright reject solutions. The discussions have moved to high-level architectural points, where we just request better implementations than the first-shot. We get them, review again and move on. Maybe there are deeper, latent problems beeing embedded in the code-base through this process, but we can't spot any although we are looking intently.
And just like that, my profession has died. Nobody in my team writes code anymore. Things that would have taken days are now done in minutes. We are all just considering design and architecture. We are changing our processes since they are not fit for the new way of doing things. Scrum? What's even the point anymore? We can work through a sprint-backlog that would've taken two weeks in two days now. We do reviews 90% of the time. When we request changes they come back in minutes.
I have a feeling like the ground has been taken from below me. My son wanted to become a developer, that isn't a real job anymore. Everyone around me feels the same way, including our management. We are a small team and a small company, everyone is effective and our jobs are not in danger as long as our product is succesful. But my profession is essentially dead. It's a craft I honed for many, many years and the craft itself is now automated. I wanted to improve my typing skills to code faster, this is now a ridiculous endeavour. We don't know what comes next, but I now only have my analytical skills and knowledge of good architecture to fall back on.
And it all changed so fast. I don't think this will transfer to other professions so easily, but for programming LLMs ended it all in the span of the last four months. I thought I knew what my future holds as a senior dev, now I can barely see past christmas.This is a nightmare.
I mean, yeah. Asking "do xyz make no mistakes" won't get you what you want. Come up with a plan, decide on what constraints matter, pick libraries you want to use. Decide what doesn't matter. Generally around now I feed it my plan and ask it "do you think I've missed anything?" and read what it suggests and see if I agree with any of it.
From there, plan out what incremental progress looks like. Maybe start with the UART/I2C/SPI library and just reading firmware version off of each child component. Then ask it to go through the plan, increment by increment. Review everything as it does it, watch the console and redirect it if it's making poor choices or just things you know will box yourself in in the future.
On my best day in university my project this weekend would have been weeks of work. It took me a day, not including soldering and packaging after the electronics were functional. It's just undeniably faster at this point. Even if the only thing I used it for was to open the serial port, watch the logs, and fix any bugs I introduced, it would be faster. But you can feed it whole datasheets and it'll construct a client library for your device essentially perfectly on the first try.
I have serious ethical qualms about how LLMs were created, how they're going to be used, the inequality they create, the finances of the industry behind them, the environmental impact... but from a purely technical perspective they can be very powerful tools. But you need to already have domain knowledge to be able to massage it properly - "everyone is engineer" still isn't reality. Someone with experience and good intuition though...