view the rest of the comments
News
Welcome to the News community!
Rules:
1. Be civil
Attack the argument, not the person or their instance. No racism/sexism/bigotry. Good faith argumentation only. This includes accusing another user of being a bot or paid actor. Trolling is uncivil and is grounds for removal and/or a community ban. Do not respond to rule-breaking content; report it and move on. Keep on-topic
2. All posts should contain a source (url) that is as reliable and unbiased as possible and must only contain one link.
Obvious biased sources will be removed at the mods’ discretion. Supporting links can be added in comments or posted separately but not to the post body. Sources may be checked for reliability using Wikipedia, MBFC, AdFontes, GroundNews, etc.
3. No bots, spam or self-promotion.
Only approved bots, which follow the guidelines for bots set by the instance, are allowed.
4. Post titles should be the same as the article used as source. Clickbait titles may be removed.
Posts which titles don’t match the source may be removed. If the site changed their headline, we may ask you to update the post title. Clickbait titles use hyperbolic language and do not accurately describe the article content. When necessary, post titles may be edited, clearly marked with [brackets], but may never be used to editorialize or comment on the content.
5. Only recent news is allowed.
Posts must be news from the most recent 30 days.
6. All posts must be news articles.
No opinion pieces, Listicles, editorials, videos, press releases, or celebrity gossip will be allowed. All posts will be judged on a case-by-case basis. Mods may use discretion to pre-approve videos or press releases from highly credible sources that provide unique, newsworthy content not available or possible in another format.
7. No duplicate posts.
If an article has already been posted, it will be removed. Different articles reporting on the same subject are permitted. If the post that matches your post is very old, we refer you to rule 5.
8. Misinformation is prohibited.
Misinformation / propaganda is strictly prohibited. Any comment or post containing or linking to misinformation will be removed. If you feel that your post has been removed in error, credible sources must be provided.
9. No link shorteners or news aggregators.
All posts must link to original article sources. You may include archival links in the post description. News aggregators such as Yahoo, Google, Hacker News, etc. should be avoided in favor of the original source link. Newswire services such as AP, Reuters, or AFP, are frequently republished and may be shared from other credible sources.
10. Don't copy entire article in your post body
For copyright reasons, you are not allowed to copy an entire article into your post body. This is an instance wide rule, that is strictly enforced in this community.
I mean, that was the joke for Amazon for years, but....look where they are now.
But I just don't understand the massive hype around LLMs. The cost per 1M tokens has dropped, significantly, since 2023 or so. And I don't think there is any reason to expect it not to continue. If you host it yourself, it's basically free other than hardware and electricity.
Sure, with agents like claude code, you may use a bit more tokens or use it more often, etc, and the best models are more expensive. But not every task needs the best model or even a paid model at all. And people seem to be finding ways to fan out things to prefer local models for simple stuff, escalate to something paid, etc.
I just don't get the business model at all, but maybe I'm just too stupid to understand. It's also possible we have an Enron-style situation, but spanning multiple companies and/or industries....
Amazon was building a platform, that now they use as a rentier. Bad for society, good for Amazon.
But LLM models don't have that. You can swap one out for another with little effort - hell, I do it daily. And with that, it means that you're a commodity . . . which also means you can't ever get those initial losses back if your competitor (like Chinese models) doesn't have them.
Have you seen openrouter.ai? They are more or less based around that idea, I think. Routing to whatever is best fit - most likely on price, I'd think (?) and/or currently up.
I have not set it up myself, but I think there is a way to have opencode or the like prefer local models for certain things, then escalate to something remote (presumably paid) if needed.
Also, opencode.ai gives you so many free tokens per day on some of their models.
The problem is that it's just not commercially viable, because you can't just say "We make money on inference", because that's a stupid thing to say. That's like saying the cash register at the supermarket is really profitable, because you sell a product for a higher price than it costs you to buy it. It may be true, but that's not how any of this works.
You can't just stop developing new products, because if you do, your existing product becomes worthless. If Ford just started selling their current models, they would make a lot of profit right now... and then none in a few years as their competitors sell newer, fancier cars. If OpenAI stops training new models, they will make a lot of money right now, and then nothing in 3 to 6 months from now.
Token pricing could double, and be free for OpenAI, and it still wouldn't save them, because you can't judge a business on how much money they earn at the cash register.
There is no businessmodel. The trick to keep burning cash, keep the stock high, keep the compensation for the people at the top flowing. The company can burn all the capital it wants, because the people at the top will keep getting paid.
One layer below that, there's a whole bunch of business idiots who think LLMs will let them save on paying their employees, so they're very invested in it.
One layer below that are a bunch of senior managemers who are just as stupid, but whose performance goals were just tied LLM use, so they've got a great reason to push it on everyone.
A problem the 2 big labs have, with OpenAI the bigger problem, is forced compute leases. The training is basically free because they have way too much GPUs already rented.
indeed, i refused to buy stock because it was costing $1 to make 80c in sales, what sort of business is that? and yet here we are, to be fair their cloud business is the big profit centre and that came later