Hello everybody, happy Monday.
I’m hoping to get a little help with my most recent self-hosting project. I’ve created a VM on my Proxmox instance with a 32GB disk and installed Ubuntu, Docker, and CosmOS to it. Currently I have Gitea, Home Assistant, NextCloud, and Jellyfin installed via CosmOS.
If I want to add more services to Cosmos, then I need to be able to move the containers from the VM’s 32GB disk into an NFS Share mounted on the VM which has something like 40TB of storage at the moment. My hope is that moving these Containers will allow them to grow on their own terms while leaving the OS disk the same size.
Would some kind of link allow me to move the files to the NFS share while making them still appear in their current locations in the host OS (Ubuntu 24.04). I’m not concerned about the NFS share not being available, it runs on the same server virtualizing everything else and it’s configured to start before everything else so the share should be up and running by the time the server is in any situation. If anyone can see an obvious problem with that premise though, I’d love to hear about it.
In general, container root filesystems and the images backing them will not function on NFS. When deploying containers, you should be mounting data volumes into the containers rather than storing things on the container root filesystems. Hopefully you are already doing that, otherwise you’re going to need to manually copy data out of the containers. Personally, if all you’re talking about is 32 gigs max, I would just stop all of the containers, copy everything to the new NFS locations, and then re-create the containers to point at the new NFS locations.
All this said though, some applications really don’t like their data stored on NFS. I know Plex really doesn’t function well when it’s database is on NFS. But, the Plex media directories are fine to host from NFS.
I’m still early enough in that if something’s wrong or not ideal about the config, I can go scorched earth and have the whole thing back up and running in an hour or two.
Is there a better filesystem that I could share out for this kind of thing? My RAID Array is run through OpenMediaVault if that helps.
Docker’s documentation for supported backing filesystems for container filesystems.
In general, you should be considering your container root filesystems as completely ephemeral. But, you will generally want low latency and local. If you move most of your data to NFS, you can hopefully just keep a minimal local disk for images/containers.
As for your data volumes, it’s likely going to be very application specific. I’ve got Postgres databases running off remote NFS, that are totally happy. I don’t fully understand why Plex struggles to run it’s Database/Config dir from NFS. Disappointingly, I generally have to host it on a filesystem and disk local to my docker host.
The key with NFS is performance. Its not that you can’t store disk dependent apps on NFS. You can but you need to be aware of latency and speed.