Can someone explain how this works?
this post was submitted on 06 Oct 2023
760 points (100.0% liked)
196
16542 readers
1928 users here now
Be sure to follow the rule before you head out.
Rule: You must post before you leave.
founded 1 year ago
MODERATORS
In a nutshell (eli5) swap is sorta "very slow ram" which is actually just a section of your hard drive /SSD/some other bizarre medium. It is generally used to temporarily store information that might be needed later but would waste valuable "fast ram" which is your actual ram sticks.
What's going on here is this user mounted Google drive in a way that the operating system can interact more directly with it, and it appears to have a set amount of space. Because we've mounted Google Drive we can tell our operating system to use it as swap....very very slow silly swap, but swap nonetheless.
So that's exactly what they did, they told the operating system to set aside X amount of Google Drive for swap, and when looking at the resource monitor we can see the "swap" appears as "more ram".
Hope that helps, please ask if I confused you :)
I'm pretty sure google drive just acts like a syncing tool in the same way as dropbox, so this would still act like a normal swap drive, presumably.
That said, I've only used swap partitions so I'm not sure how it works when you point it at a directory, but I guess it depends how this person set it up.
They've used google-drive-ocamlfuse to mount the drive in Linux, which if I recall correctly is direct access, rather than the way it works in Windows
No they didn't. They used rclone and mounted Google drive in vfs-cach mode which means it's firstly written to the drive before it's synced to the cloud
My mistake, they chose a different implementation
So, if they did use a direct storage mode connection instead of this cache method with a Gbit+ Internet connection, do you think it would be faster than swap on an HDD?
I can't imagine a time when anyone would ever make a cloud swap drive. You want the swap to be as fast as possible
No. It also has to be written to GDrive's storage
It makes sense that you can do this, as it's how Chromebooks work, and they run on ~~Chrome OS~~ ~~Android~~ Linux
Oh cool
You're right, this wouldn't really work, but it's still a funny idea.
Time to use my floppy as a swap space 😎
I've used my floppy in a few swap spaces myself if ya know what I mean....
Did it play the Imperial March?
Can I watch?
Thanks <3
So RAM is just memory space that is accessible very quickly by applications that need it. Many applications use it to just temporarily keep information while you have it open. If you have a lot of applications open or you have an application that stores a lot of things even if it doesn't immediately need it all, you can run out of it quickly.
A swap space is a Linux tool (though there are similar things in Windows) that reserves a spot on your hard disk to move data to and from RAM, to keep data that isn't actively being used off the RAM and data that is being used on it. That way you're not as likely to run out of memory from memory-hogging applications.
There's a separate tool to use Google drive space as a disk in Linux. Then, once it's configured as a disk, you can make it act as a swap space. Of course, this is very slow even compared to normal hard disks, but the point is it is possible to do it.
All you need to know is that RAM is all about being extremely fast.
Swap is disk space set aside to be used as virtual Ram (part of your disk become RAM ) for when you have no more real ram. It's useless on modern computers because RAMs are at worst of worst 100 times faster than the fastest SSD. If you have no ram it's just best to crash the OS than making it slow down to a crawl or freeze.
This person use their Google drive as swap which make it even far more useless because even the slowest ssd is probably faster than the best commercial internet connection even with a perfect latency.
In short, it's extremely cursed.
I don't know about useless.
I'm much happier now that my work laptop slows down when I compile something during a zoom meeting, vs when it used to run out of memory and crash.
If you have no ram it’s just best to crash the OS than making it slow down to a crawl or freeze.
That very much depends on what you're doing. Even with rather random access patterns (e.g. compiling) swapping out doesn't crash performance terribly, mostly because my RAM isn't exactly under-dimensioned (I used the rule of thumb "one gig per hardware thread, round up"). For more regular access patterns such as merging SDXL models (which definitely eats all my 16G) the impact is even less. For, dunno, highly complex and irregular datalog queries over a database four times as large as your RAM -- yeah that won't be nearly as fast.
What you also want to do (under linux) is enable the earlyoom daemon. Those freezes are Linux being way too nice and not killing processes until literally every cached and buffered thing has been purged, also heavily-used ones.
Then you'll see behaviour such as switching tabs in the browser, or bringing up a minimised terminal or something actually taking a second or two because they got swapped out. But it's nowhere close to unusable and, due to a 3G/s SSD, a way better experience than in the 90s with a couple megabytes of RAM and swapping to a glorified flywheel.
I could by more RAM, DDR4 prices have pretty much tanked after all, OTOH I swap out like fifteen minutes every other month. Not worth it, I have coffee to make.
While an interesting read, my comment isn't directed at advanced users that know their stuff. Also, not everyone own a beast, most users have a medium or low tier PC. Funnily, 16GB of RAM for your usage seems low. Double that and you'll never need swap unless you want to play with the biggest local LLMs via Llama.ccp. But then, you likely have a GPU beast.
Nah I'm only dabbling around with SDXL, I have a 4G RX5500. Four years ago a Ryzen 3600 was certainly respectable (and slightly more expensive than the GPU) but it's nowhere near high end, it was slam dunk in the middle of the price/performance optimum and is generally sufficient for my workloads.
I don't think I'll upgrade this box (short of an SSD or such) before either GPU prices are sane again, and/or CPUs actually become noticeably faster. All the AM4 ones certainly don't really seem to be worth it. See I'm an old fart millennial, I'm used to a "two years later get twice the performance at half the price" kind of cadence and the two years have been steadily getting longer.
Sadly, i don't think GPU prices will ever drop. The only reason would be if a new competitor comes up with something on par or better than the current tech. For CPUs, we've reached the limit already. Nonetheless, their's still some hope with a tech called optical computing.
Computer power increase became meaningless because companies juste use it as an excuse to not optimize their software anymore. The best exemple is how everyone basically uses a whole browser (chromium ) just to show some GUI. The steam client and discord are two big software that comes in mind using this but it's spreading fast and I found a GUI for aria2c that weights more than 60mb while aria2c itself is a few megs, but even worse some manufacturers are using it for their mouse and keyboard drivers for god sake.
I’m old enough to remember that my swap drives used to be on spinning drives that were slower than my gigabit fiber. Well, I’m actually older than that but still. If I really needed to run some unoptimized task that required a lot of memory I could consider trying it and walking away.
I'm old enough to remember when my first computer had a 210MB hard drive and 4MB of ram.
Fancy! My first computer had 16K RAM and saved to cassette.
Lording it over us poor Vic-20 users :( I remember wishing I could have that computer. I ended up with a commodore 64 soon after.
Nice! In case you didn't know, you can run those classic systems on emulators now, for yours it'd be this: https://en.wikipedia.org/wiki/VICE
The first PC I owned was a 486 with 8MB of RAM which no one else had. I remember I could make a RAM drive and copy the entire install of games into the RAM drive and they would be so fast. Imagine being able to cache 3 floppy disks worth of data!
I’m old enough to remember that my swap drives used to be on spinning drives that were slower than my gigabit fiber.
be me
the year is 2023
have a spinning drive and no gigabit fiber. ( gigabit fiber doesn't exist where i live )
become sad.
Joke aside, i don't think swap is worth it because i think nobody is willing to have their computer locked for tens of hours or even days for something to finish considering it's literally impossible to know if the app is working or is hung ( search the halting problem for more info).
A normal swap space is a partition on a hard drive that is used as overflow for ram. It allows the computer to continue working if all the ram is being used.
As far as using google drive as a swap, i have no idea how that would be implemented, but if it's real, the computer would just upload overflow memory to g drive.
Thanks :)
Modern computers are set up so that they can use the SSD/hard drive as extra, much slower RAM. Typically, when normal RAM is full, and you need more, a page of data in RAM will be swapped for a page of data on disk. On Unix, they end up in something called the swap file or swap partition, and on Windows, the equivalent is called the page file. In the screenshot, someone's mounted their Google Drive as a filesystem, and told their computer to use it as the swap partition, so instead of swapping to disk, it swaps to the cloud. This is obviously way slower, but they're effectively now using the cloud as RAM.
it doesn't. LTT tried it and explains why
Well it does. But its just not practical.
https://youtu.be/JcJSW7Rprio You might enjoy this, impractical hard drives nobody asked for.
I have a hunch it's a Tom 7 aka suckerpinch video.
Edit : yep, it is.
Truly one of the most underrated creators. His reverse NES emulation video floored me.
He's but a bored computer god. Jokes aside, he's not doing it for the views although his videos often reach 1 million views so he isn't that underrated. He's an inspiration to make stuff for the sake of it.
Here is an alternative Piped link(s):
https://piped.video/JcJSW7Rprio
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I'm open-source; check me out at GitHub.
Here is an alternative Piped link(s):
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I'm open-source; check me out at GitHub.
cursed
This brings back memories.
When I was in university, I worked in the computer services department (IT services for the campus). A social science prof down the hall would use a program called SPSS (Statistics Package for Social Sciences) to do various statistics on data sets.
It was available on two platforms: PC & VAX
The PC had about 4 MB of RAM. The VAX (a large centralized computer) had much more (not sure the amount, but I'm guessing around 256 MB at least). The data sets they were using would sometimes require more RAM to process than the PC had (even with swap space), and would give an "out of memory" error.
They always came down the hall and would ask us what to do. The answer was always the same, "Some combination of: get more RAM/swap space for the PC, shrink the data set, simplify the complexity of the query or run it on the VAX server."
They finally started saying, "We connect to VAX, get more memory?"
I don't know if they thought using telnet to connect to the VAX actually caused the local PC to gain RAM or not but it was an inside pun for the department for a couple of years ( ͡° ͜ʖ ͡°) .
Ahh yes. rclone