this post was submitted on 31 Jan 2025
352 points (94.7% liked)
Open Source
33247 readers
496 users here now
All about open source! Feel free to ask questions, and share news, and interesting stuff!
Useful Links
- Open Source Initiative
- Free Software Foundation
- Electronic Frontier Foundation
- Software Freedom Conservancy
- It's FOSS
- Android FOSS Apps Megathread
Rules
- Posts must be relevant to the open source ideology
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
- !libre_culture@lemmy.ml
- !libre_software@lemmy.ml
- !libre_hardware@lemmy.ml
- !linux@lemmy.ml
- !technology@lemmy.ml
Community icon from opensource.org, but we are not affiliated with them.
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
What???? Whoever wrote this sounds like he has 0 understanding of how it works. There is no "more privacy-friendly version" that could be developed, the models are already out and you can run the entire model 100% locally. That's as privacy-friendly as it gets.
Operated, yes. Trained, no. The model is MIT licensed, China has nothing on you when you run it yourself. I expect better from a company whose whole business is on privacy.
To be fair, most people can't actually self-host Deepseek, but there already are other providers offering API access to it.
There are plenty of step-by-step guides to run Deepseek locally. Hell, someone even had it running on a Raspberry Pi. It seems to be much more efficient than other current alternatives.
That's about as openly available to self host as you can get without a 1-button installer.
You can run an imitation of the DeepSeek R1 model, but not the actual one unless you literally buy a dozen of whatever NVIDIA’s top GPU is at the moment.
A server grade CPU with a lot of RAM and memory bandwidth would work reasonable well, and cost "only" ~$10k rather than 100k+...
I saw posts about people running it well enough for testing purposes on an NVMe.
Those are not deepseek R1. They are unrelated models like llama3 from Meta or Qwen from Alibaba "distilled" by deepseek.
This is a common method to smarten a smaller model from a larger one.
Ollama should have never labelled them deepseek:8B/32B. Way too many people misunderstood that.
I'm running deepseek-r1:14b-qwen-distill-fp16 locally and it produces really good results I find. Like yeah it's a reduced version of the online one, but it's still far better than anything else I've tried running locally.
Have you compared it with the regular qwen? It was also very good
The main difference is speed and memory usage. Qwen is a full-sized, high-parameter model while qwen-distill is a smaller model created using knowledge distillation to mimic qwen's outputs. If you have the resources to run qwen fast then I'd just go with that.
I think you're confusing the two. I'm talking about the regular qwen before it was finetuned by deep seek, not the regular deepseek
I haven't actually used that one, but doesn't the same point apply here too? The whole point of DeepSeek is in distillation that makes runtime requirements smaller.
No cause I was already running regular (non-deepseek) qwen 14B, admittedly a heavily quantized and uncensored version, so I was just curious if it would be any better
The big difference with R1 at least is that it is trained using reinforcement learning, and it gives you an explanation of how it arrived at a solution. I find the thinking bit is really fascinating cause it really reads like a train of thought. Here's an example of what it outputs when I ask it to write a react component with a text input and a button:
First, I'll start by setting up the basic structure of a React component. I'll import React and useState since we'll need state management for the input value.
Next, I'll create a functional component called TextInputButton. Inside this component, I'll use the useState hook to manage the text input's value. The initial state can be an empty string.
Then, I'll define a handler function, handleChange, that updates the state whenever the user types into the input. This function will take the event object and set the new value using setTextInputValue.
I also need a function to handle the button click. For now, I'll use handleClick with an alert message as a placeholder. The actual functionality can be added later based on requirements.
In the return statement, I'll structure the JSX. I'll include an input element of type text, binding its value to the state variable and attaching the handleChange function to the onChange event. This ensures that the state always reflects the current input value.
Below the input, I'll add a button element. On clicking this button, it will trigger handleClick. I'll also give it a class for styling purposes, like 'button' which can be styled with CSS if needed.
Finally, I'll export the component so it can be used elsewhere in the application. I'll make sure to include comments explaining each part for clarity.
Here's a simple React component with a text input and a button:
This component:
You can use this component in your application like:
The input value is controlled through the
text
state variable, and you can access it usingtext
. When the button is clicked, thehandleClick
function will execute whatever logic you need.I think they mean privacy friendly version of the infrastructure could be developed.
Unfortunately it is you who have 0 understanding of it. Read my comment below. Tldr: good luck to have the hardware
I understand it well. It's still relevant to mention that you can run the distilled models on consumer hardware if you really care about privacy. 8GB+ VRAM isn't crazy, especially if you have a ton of unified memory on macbooks or some Windows laptops releasing this year that have 64+GB unified memory. There are also websites re-hosting various versions of Deepseek like Huggingface hosting the 32B model which is good enough for most people.
Instead, the article is written like there is literally no way to use Deepseek privately, which is literally wrong.
So I've been interested in running one locally but honestly I'm pretty confused what model I should be using. I have a laptop with a 3070 mobile in it. What model should I be going after?
as I said in my original comment, it's not only VRAM that matters.
I honestly doubt that even gamer laptops can run these models with a usable speed, but even if we add up the people who have such a laptop, and those who have a PC powerful enough to run these models, they are tiny fractions of those that use the internet on the world. it is basically not available to those that want to use it. ot is available to some of them, but not nearly all who may want it
Is it Open Source? I cannot find the source code. The official repository https://github.com/deepseek-ai/DeepSeek-R1 only contains images, a PDF file, and links to download the model. But I don't see any code. What exactly is Open Source here?
I don't see the source either. Fair cop.
Thanks for confirmation. I made a top level comment too, because this important information gets lost in the comment hierarchy here.
Open source is in general wrong term in all of these "open source" LLM's (like LLAMA and R1), the model is shared, but there is no real way of reproducing the model. This is because the training data is never shared.
In my mind open source means that you can reproduce the same binary from source. The models are shared for free, but not "open".
Obviously you need lots of GPUs to run large deep learning models. I don't see how that's a fault of the developers and researchers, it's just a fact of this technology.
and that is not what I was complaining about
Down votes be damned, you are right to call out the parent they clearly dont articulate their point in a way that confirms they actually understand what is going on and how an open source model can still have privacy implications if the masses use the company's hosted version.
There are already other providers like Deepinfra offering DeepSeek. So while the the average person (like me) couldn't run it themselves, they do have alternative options.
which probably also collects and keeps everything you say in the chat. just look in ublock origin's expanded view to see their approach to privacy, by having a look at all the shit they are pushing to your browser