-8
submitted 1 week ago* (last edited 6 days ago) by robertdegallan@lemmy.org to c/programming@programming.dev

I've been running deep research with agents for the past few weeks on one question: what is the most promising future for cheap, fully local AI-driven software engineering?

After weeks of deep-research agent runs, the most recent and promising stack for cheap, fully local AI-driven software engineering found by 5 agents was:

Qwen4 27B + CoDoc

Turns out that the answer was David and the rock.

  1. Qwen4 27B (David). Quantized, it should fit on a single 24 GB consumer GPU. The coding capability at this size is now good enough for the bulk of day-to-day engineering work provided the agent has the right context.

  2. CoDoc (the rock). This is a proposed standard where every software library ships AI-optimized documentation inside the package itself: docs/README.md as entry point, docs/index.md as catalog, plain markdown throughout. Version-matched by construction, offline, deterministic. In the JavaScript ecosystem npm install puts the docs on disk next to the code.

According to the research agents: the bottleneck for a 27B local model is not raw intelligence, it's the context. A small model hallucinating a v5 API is a context problem, not a capability problem and CoDoc fixes the context for free.

The evidence in the white-paper points in the same direction:

  • Vercel's own agent evals on Next.js 16 APIs (absent from training data): 100% pass rate with bundled docs + an AGENTS.md pointer, vs 53% without. The on-demand "skill" variant maxed at 79% and was never even invoked in 56% of cases.
  • The unknown European IT company, Amadeus, claims that they have observed reduction in hallucinated APIs, the use of deprecated patterns and repeated correction cycles but they do not provide any rigorous evidence beyond these statements. They say that they intend to develop CoDocBench to address this gap.

Now the Goliath part. The big labs' and hyperscalers' moat was never the model: open weights are commoditizing fast. Their moat is the paid context layer: RAG pipelines, hosted doc intermediaries, MCP servers, enterprise knowledge add-ons, etc. According to the research agents: CoDoc collapses that entire layer into npm install or the equivalent in each ecosystem. A standard that makes context free, local, and version-pinned is an existential matter for anyone monetizing context scarcity.

And now the best part, the timing :) The AI bubble is popping as I type this. Goliath's capital expenditure thesis requires inference to keep getting more expensive and cloud dependency to keep deepening. With David and his rock, a decent modern laptop doing your engineering agent's job for electricity money is Goliath's worst nightmare.

The standard itself comes from an unknown European IT company, not from any lab, which is exactly why I believe it adds to its credibility (despite its lack of first-hand evidence): it has no incentive to route your context through a metered API or sell you tokens, quite the contrary, it has all the incentives to make inference as cheap as possible.

I have read CoDoc's white-paper and while it is far from perfect, I do mostly agree with the output of the research agents that picked up on Qwen4 27B + CoDoc as the most promising future for cheap, fully local AI-driven software engineering.

And you, do you think that it is game over for the AI Goliaths?

you are viewing a single comment's thread
view the rest of the comments
[-] FizzyOrange@programming.dev -1 points 3 days ago

Yeah so Opus 5.5 (which is what I use basically 100% of the time currently) gets 100% anyway. As do most of the other frontier models.

Would you care to show some evidence, or is it a vibe?

It's an very conservative estimate based on my experience. Almost all open source software these days keeps their docs in the same Git repo as their source. I haven't checked these, but go and look at e.g. LLVM, Git, libpng, SDL, Qt, etc. if you don't believe me.

Show us the evidence.

Nah, too much work and you wouldn't believe me anyway. I've seen it with my eyes though. Hell, frontier models are happy to reverse engineer binaries without prompting in order to get things done. Reading a bit of source code is nothing to them. (Does it violate a "no reverse engineering" license agreement if AI does it of its own volition lol...)

Would you like to find this out at runtime

No I'm not dumb enough to use dynamic typing.

[-] robertdegallan@lemmy.org 0 points 18 hours ago* (last edited 18 hours ago)

You say Opus 5.5 "gets 100% anyway" but did you read the footnote?

  • AGENTS.md points agents to bundled Next.js documentation. Both conditions can access these docs. With AGENTS.md shows the success rate with that initial guidance.

All models including the frontier models had the bundled docs available the whole time. Your 100% just shows that the AGENTS.md pointer wasn't necessary for some of them. Which is, if anything, another point for CoDoc ;)

Now here's the part you skipped: GLM5.2 (open-weight model) also hit 100% with the AGENTS.md guidance, at 32% less cost than the Opus 5.5 you use "basically 100% of the time." So the frontier model isn't magic. The context is. And that context is free with CoDoc for all models.

"It's a very conservative estimate based on my experience." So it was a vibe after all... And your vibe, built on:

  • "I haven't checked these, but go and look at e.g. LLVM, Git, libpng, SDL, Qt, etc."

Cherry-picking five eyeballed repos as evidence for "50% of software" is just hilarious. And you're mixing up two different things. I didn't ask whether docs live in the same Git repo. I asked whether libraries ship complete, AI-optimized documentation inside the package artifact. So here we have someone who eyeballs five repos and calls it a "very conservative estimate." ๐Ÿ‘

"Frontier models are happy to reverse engineer binaries without prompting. Reading a bit of source code is nothing to them." Great, here we have someone advocating for letting frontier models waste compute power and frivolously consume your tokens to do what co-packaged documentation does for free. How about we put some of those data centers next to your place?

And by the way, two things:

  1. The paper disarms your argument of models looking at source-code, section 10.5 if you care to read it. There is a reason why humans have built documentation wikis, sites and portals for years.

  2. You might want to read this: Claude Opus 5.5 Reverse Engineering Refusals

"No I'm not dumb enough to use dynamic typing." The point was never about typing. It was about a model hallucinating your v5 API signatures.

Since you use Opus 5.5 basically 100% of the time, let me recall something for you. On June 12th, a single export control letter switched Fable 5 off for every customer worldwide for 17 days. Opus 5.5 is one letter away from the same fate. Besides, even if they do not share the same fate via the export control letter, the company behind the model you rely on is not profitable (it is actually very far from it). As you may know, it will soon IPO to finance their operations and service their existing debt. The irony: they have frontier models under their fingertips, yet their own models cannot solve their business model or their financing problems. You are more than welcome to put all your oranges in one basket.

TLDR: a vibed estimate from five unchecked repos, completely dismissing the benefits of CoDoc for ecosystems with dynamically-typed languages, and a frontier model that "gets 100% anyway" which sure, it does: with the bundled docs that CoDoc ships.

I'll be here if you want to come back with solid evidence. If, however, you do not want to show the evidence behind your claims and you just wanna play the trust me bro game, please play it with some naive people. It ain't gonna work with me ;)

this post was submitted on 09 Nov 2026
-8 points (31.8% liked)

Programming

29290 readers
611 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 3 years ago
MODERATORS