Hello self-hosters! :) I'd like to tap into your experience
tl;dr: Is it really a bad idea to use MariaDB with data over NFS?
My setup
- RAID NFS NAS on RPi4
- "CPU machine", just an old laptop for now
The idea is that NAS stores the data and RAID mitigates single point of failure. Laptop is only for running the stuff, all docker-compose are under git, uploaded elswhere. Laptop fails - I buy a new refurbished one and put everything back together from data on NAS and git
Problem
But Nextcloud uses DB. ATM it sits on the laptop. And I'm thinking how to somehow move that data to the NAS.
At the beginning NAS was just for NC files, connected as NC "external storage" over SFTP. But there were issues with this setup and NFS4 turned out to not require Kerberos to work.
Since NFS docker volumes have been working quite fine, I thought about just moving the data to the NAS. But then I read this:
Generally, we recommend not to use NFS (Network File System) with MariaDB, for these reasons:
MariaDB data and log files on NFS volumes can become locked and unavailable for use. Locking issues may occur in cases where multiple instances of MariaDB access the same data directory, or when MariaDB is shut down improperly, for instance, due to a power outage. In particular, sharing a data directory among MariaDB instances is not recommended.
Data inconsistencies due to messages received out of order or lost network traffic. To avoid this issue, use TCP with hard and intr mount options.
And I feel discouraged
- I could move whole DB (I mean including the process) to the NAS, but then operating the DB becomes NAS' burden and breaks the responsibility separation
I'm not a fan of this idea - I could treat NAS as first stage of backup for that DB data and just send the data to NAS once a day
Also a bit meh. Getting back up from server failure will require copying the data - Maybe it's not that bad to use MariaDB with NFS?
If there is a problem with server-NAS connection, I actually don't want Nextcloud to work at all. It should fail, so I know I need to fix something
So, what are your experiences? Is running the DB on one machine, using the data from another machine over NFS a shot in the foot?
Not nextcloud, but I have several containers set up to use 'third party' database - ie, not running inside the container or compose group, sometimes on a second machine. It definitely complicates the setup. I haven't noticed any performance issues, but my services are low load.
I would think that the db running on the Pi with storage on the Pi could be faster than the db running on the laptop with storage on the Pi. The Pi db would presumably have higher bandwidth to the Pi storage and only have to transfer the selected data over network, where the laptop db has to do all the db sorting, filter, and processing over the network, although I suppose the db would try to keep all the data in RAM. I know there's limitations to Pi's internal bus bandwidth, Maybe you can compare the hdparm performance of the NAS RAID vs the NFS mount.
I think so too, but I'm mindful of the 1GB RAM per TB rule. I'm afraid to choke the NAS and on Pi I can't put in more RAM