Selfhosted

Are you running rootless or rootfull containers?

At least with rootless containers (I have not used roortfull) I think its quite pointless to create separate users for the services, as you could use just the subuids (and/or SELinux) to get container separation, as you already seem to do .

I run my quadlets with uidmap and gidmaps, so I get explicit control of the mappings. It can be a tad tedious at times but I think it's worth the hassle.

You can use configuration management software like Ansible or Puppet. I wouldn't write my own scripts when there are good Ansible modules.

Are you using one (system) user per quadlet? (Why?) Each of my quadlets is owned by the same (non-root) user and the containers are rootless. In that case, podman deals with (sub)id-allocation. selinux-labels take care of volume scopes and podman networks are as isolated as possible. Seems enough for me; never even considered using different users in that case.

I approach it the same as I did with Docker, one user per container.

I started with rootless but networking within Podman is moot with multiple users per container. And one user for all containers to get networking has to lead to subUID clashes (and thus escape vectors) - unless someone can explain how not…

But root Podman is just as secure anyway, and easier, so I just roll with UserNS=auto and use idmap= on the volumes to enable writing as the specified user for the container. And networking in Podman works because it’s one user space. By default UserNS=auto gives 1024 subUIDs to a container. I had to up that to the expected max of 65535 for Frigate to work (example bit I use with idmap, UserNS with the size and group mapping, and GroupAdd):

Volume=/mnt/camraz:/media/frigate:rw,noexec,nosuid,nodev,Z,idmap=uids=@1111-0-1;gids=@1111-0-1#1020-1020-1  
UserNS=auto:size=65535,uidmapping=105:@105:1,gidmapping=105:@105:1  
GroupAdd=105  

So what's going on here is on the Volume= I have the camera folder mounted to where Frigate wants it, and I specify that it accesses that volume with idmap= and frigate's UID/GID 1111 and the camera group I made at GID 1020.

The order for the idmap is @${hostUID}-${containerUID}-${num2map}. The @ symbol means that 1111 is an absolute host UID; drop that and 1111 is pulled from the containers subUID/subGID list. Most containers run as root (0) so you just map to that user.

Side quest: For containers that have one of those mini-hypervisors like LinuxServers images and their "S6" thing, I worked around them starting with a root user but then requiring moving to a non-root user by specifying PUID=1 and PGID=1 then setting idmap=uids=@1999-0-2;gids=@1999-0-2 and reserving 1999 as username containerNameStartup and 2000 as containerName. This idmap= maps host 1999 to container 0 but maps two UID/GIDs consecutively, so it also maps host 2000 to container 1 keeping everything lined up perfectly - while LinuxServer images still don't get host root like they so desperately want.

GroupAdd=105 adds in the group 105 to the container, which is notably not the host group 105. UserNS is needed to complete that. This just makes that group exist in the container.

UserNS=auto:size=65535,gidmapping=105:@105:1 accomplishes the subUID/subGID size allocation to the expected default max of 65535 (because Frigate needs it) and the group connection from host 105 to container 105. Note that this syntax ~~helpfully~~ uses the opposite order as idmap=. It goes ${containerUID}:@${hostUID}:${num2map} and uses colons instead of dashes.

Every other container is cool with the default 1024 (just UserNS=auto). The subUIDs are pulled from a non-existent user named containers that you need to enable for Podman root to work with UserNS, and it has like 2 ~~million~~ billion or something with their recommended setup, so it’s good on subUIDs/subGIDs.

Command to set up root Podman with UserNS=auto:

sudo echo "containers:2147483647:2147483648" >> /etc/subuid  
sudo echo "containers:2147483647:2147483648" >> /etc/subgid  

You can also have UserNS=auto and run the container as a specific host user directly instead of "nobody" with:

UserNS=auto:uidmapping=0:@1111:1,gidmapping=0:@1111:1

Same idea as how I mapped in the host group 105 above. Where the root user 0 is mapped to the host (via @) user 1111 so the container runs as the user. I don't use that because the idmap= works perfectly and then the container doesn't even run as the user that controls the files directly. But maybe there will be a use for that sometime(?).

And I do have a fuckton of users; Debian once complained it ran out of numbers or something after like 20 users, so I just ran the first thing I found to make the UID limit some really big number, and I never thought about it again! No idea what it was..

**Edit: ** I cleaned it up and added examples

Also I mostly work with UIDs not users per-say, so I specify UID 1111 has access to Frigate files. I just make a user named frigate to make it a bit easier for me. When you do ls -l it says frigate instead of 1111; life is easier.

Overall, it has been a lot of work to divine these things because the documentation and useful examples for Podman are confusing and non-existent. I think I've gotten most of this idmap= and UserNS= stuff working from reading GitHub issues that drop little tidbits. But after I get it working as I want it, it's solid, and I can now rinse-and-repeat in the future. No Docker daemon to fuck with or get hacked, and it's reliable since I just let systemd start these quadlets up.

I'm not 100% done, but I'm close. Outstanding issue right now is that on server restart sometimes some Podman networks don't actually start up correctly and then containers that want them fail to start because of that (I have to restart the Podman network via systemctl to get them to exist in Podman). But eventually I'll find the stupid GitHub issue where someone mentions something relevant for that, and then I'll be golden!

I have a Server with ~16 podman services, each their own user, network namespace and uids. This is managed using NixOS and Home manager (which supports quadlets) but I am changing my setup to a single node k3s cluster with user namespaces because that seems simpler to manage. Here a snippet for how the subuids/subuids are defined:

users.users.<username> = {
        subUidRanges = [{
            startUid = 100000+65536*( config.users.users.<username>.uid - 999);
            count = 65536;
        }];
        subGidRanges = [{
            startGid = 100000+65536*( config.users.users.<username>.uid - 999);
            count = 65536;
        }];
        home = "[...]";
        isNormalUser = true;
        linger = true;
        group = "users";
        openssh.authorizedKeys.keys = config.users.users.root.openssh.authorizedKeys.keys;
    };