this post was submitted on 01 Dec 2024
45 points (88.1% liked)

Futurology

1886 readers
54 users here now

founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] pennomi@lemmy.world 3 points 1 month ago

It depends. A lot of LLMs are memory-constrained. If you’re constantly thrashing the GPU memory it can be both slower and less efficient.