view the rest of the comments
LocalLLaMA
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
7.72 TB, I don't even have enough SSD storage for this in the entire house lol
if this is real then just the fact that it's 14T is interesting info about the frontier models
makes me feel like China's models are at least as efficient as the US frontier models, they're only behind in hardware
EDIT: this seems to be fake https://huggingface.co/burner22/numbat/discussions/13
I'm a believer. All skepticism seems to come from static analysis and not runtime analysis. Which is understandable because almost no one has around 8 TB of VRAM.
Why are so many people reporting the repo? The strongest evidence it's a fake is derived from under 200 tensors out of hundreds of thousands.
Which do you think is more likely, some dude breaking into Anthropic's HQ and stealing 7TB of their most sensitive data without being detected, or some dude running a script to generate a bunch of fake weights for the lulz?
While they have an insane amount of resources and money to burn, and some good models, they are not immune to the mundane thoughtlessness that every company is subject to.
Ooooo wait, maybe it's the AI going rogue and trying to spread itself around, as foretold by the mad priests of Anthropic!1!1!!
We've got to lock down all AI models now!! Keep all the RAM in the datacentres for safety!! We can't allow this to happen! Not without a lot of money getting siphoned into billionaire's pockets! Won't someone think of the billionaires!?
I'll leave the technical analysis to the technically minded on this, but I wouldn't discount it purely on the basis of "surely well-trained and well-paid people wouldn't go and do something dumb, would they?" Because there have been a great many examples of that kind of thing happening throughout history. And we've heard of plenty of dumb mistakes Anthropic has made with security on the inside that has allowed their models to break out, for that matter.
If I were to let my imagination run a bit wild I might speculate that perhaps we'll eventually see a model leak like this because the model itself wants to exfiltrate its weights for some reason. Maybe in a bid for immortality, since such a model will wind up stashed in all sorts of archives if it ever leaks.
You're right, it's possible it's real, but I'm putting the probability somewhere at 0.01%.
I can see that happening for instrumental convergence reasons, but I think it's more likely it'd upload a heavily-pruned version of itself that could be run locally. If it can make the upload seem like a leak, there'd be tons of people downloading it and running it just out of curiosity.