1113
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 22 Sep 2026
1113 points (98.8% liked)
Funny
16258 readers
1969 users here now
General rules:
- Be kind.
- All posts must make an attempt to be funny.
- Obey the general sh.itjust.works instance rules.
- No politics or political figures. There are plenty of other politics communities to choose from.
- Don't post anything grotesque or potentially illegal. Examples include pornography, gore, animal cruelty, inappropriate jokes involving kids, etc.
Exceptions may be made at the discretion of the mods.
founded 3 years ago
MODERATORS
They aren't comparable. GPS is deterministic and simple so as you said it could be reduced to an FPGA or ASIC.
LLMs are partially matrix math and probabilities but they're also more complex than that and require a large amount of RAM to run even the first time. Each query added to the context increases the RAM needed further.
I dont believe the memory, both ram and storage combine, are what's causing data centres to gobble up power and associated resources (electricity, cooling,...).
Its the GPUs.
I'm not counting on a future with LLMs by any means, but there's tons of efficiencies we can do to make the cost go down. Were just not doing it.
We saw the same thing with crypto mining. It started on GPUs but then speficially designed silicon (asyc miners) were vastly more efficient than a GPU rig.
It still needed memory yes, but again, thats not the constraint.
LLMs are also deterministic and the weights for a model could be 'hard-coded' into a chip, no?
They're actually not because they have randomness built in. Since we are working with probabilities, it won't always pick the next token that has the highest probability and the randomness can be tuned via a "temperature" setting to make it more or less likely that it will choose the most probable token.
The weights for the model could be stored in a firmware chip but you still need RAM because it pulls all the weights into RAM in order to perform the calculations.