Clarify /usr/local's replacement for local systemwide overrides etc.

[silverhadch]

There doesn't seem to be a local system-wide override directory defined for a read-only /usr. On a writable /usr, the /usr/local subdirectory exists, but on a system with a read-only /usr, there is no place defined for the local sysadmin to mask or override /usr's content.

Consider defining /opt/local as a replacement for /usr/local, or any other directory in fact. A way for local admins to mask vendor software, etc., from an immutable /usr is missing.

There are similar discussions at the FDO FHS:
https://gitlab.freedesktop.org/FHS/fhs-spec/-/issues/5#note_2789528
https://gitlab.freedesktop.org/FHS/fhs-spec/-/issues/5#note_3086478

[keszybz]

There is no defined integration between /opt and the rest of the system. Thus, software that is installed in /opt actually needs to hook into paths under /usr, e.g. to provide binaries that are in $PATH. So the spec says that /opt is discouraged. So I don't think we'll want to add a role for /opt/local.

In general, local modifications are supposed to be in /etc. Why wouldn't that work for you?

[silverhadch]

There is no defined integration between /opt and the rest of the system. Thus, software that is installed in /opt actually needs to hook into paths under /usr, e.g. to provide binaries that are in $PATH. So the spec says that /opt is discouraged. So I don't think we'll want to add a role for /opt/local.

In general, local modifications are supposed to be in /etc. Why wouldn't that work for you?

We cant put binaries and data into /etc. The current integration for system wide overwrites is /usr/local which is read only and not defined in the spec. /usr/local is missing an replacement in the spec. We cant mask binaries, .desktop files, libaries etc. without it. Currently we have set up all the integration and make /opt/local do /usr/local's job but since we want to comply with the LFHS and it doesnt define anything for this, this Issue was created. We want to know if there will be a standard defined for /usr/local's replacement so we can change the integration to that directory or if local systemwide overrides are unsupported and we need to drop the Idea of complying with this spec as it doesnt fit our needs.

[bluca]

We cant put binaries and data into /etc.

I agree with this, it wouldn't fit well. I think having a new location, and then having a symlink in /usr/local for compat, would be the best approach here.
What the location might be is of course the big bikesheddy question...

[cgwalters]

For ostree systems we've historically recommended /usr/local -> /opt/usrlocal...it's been OK, what about standardizing that suggestion?

[silverhadch]

For ostree systems we've historically recommended /usr/local -> /opt/usrlocal...it's been OK, what about standardizing that suggestion?

I think they Fedora Atomic has started to use /var/usrlocal instead. I have even seen some say /local at the root level.

We have
/opt/local
/var/local

Any other directory would change the symantic to radically. Even /var is more meant for state then binaries. /opt however is being used already similiary to /usr/local by third party vendors. So maybe extend it to the local admin too?

Just my two cents.

[berrange]

/opt however is being used already similiary to /usr/local by third party vendors. So maybe extend it to the local admin too?

IMHO it is preferable to keep local sysadmin overrides well separated from the location that 3rd party vendors use for their adhoc software install processes.

[silverhadch]

/opt however is being used already similiary to /usr/local by third party vendors. So maybe extend it to the local admin too?

IMHO it is preferable to keep local sysadmin overrides well separated from the location that 3rd party vendors use for their adhoc software install processes.

Thats why it should be a subfolder called local. As far as I know there isnt any closed source third party software called local. And its just an idea. It doesnt have to be /opt/local. Can be anywhere else.

[silverhadch]

I really would like it to be /opt since its one of the locations supported by systemd-sysext and we rely on sysext to be able to mask binaries in certain conditions and thus needing to be either /usr or /opt (or /etc but that is just wrong). But since we want /usr/local to not be encouraged so only realistic solution would be /opt.

[poettering]

Isn't stuff like systemd-sysext the better solution to your problem? I.e. if you want to replace /bin/ls then place the replacement in /var/lib/extensions/foobar/usr/bin/ls and do systemd-sysext refresh?

[silverhadch]

Isn't stuff like systemd-sysext the better solution to your problem? I.e. if you want to replace /bin/ls then place it in /var/lib/extensions/foobar/usr/bin/ls and do systemd-sysext refresh?

Yeah but we are Distromaintainers. For our stuff we can do that but our Users might try to install something that wants to go into the read only /usr/local. And even then we want it to be bidirectional. Its more about the Users/Local Admins accessing /usr/local.

[keszybz]

I changed my view on this after reading the discussion in the other tickets.

There is no defined integration between /opt and the rest of the system

This is true, but paradoxically, for /opt/local (symlinked from /usr/local), such integration would be provided. We have integration for /usr/local/bin, systemd config dirs, /usr/local/lib64, etc, etc, so if /usr/local is symlinked to somewhere else, things would still continue to work. So the main reason against /opt doesn't actually apply in this narrow case.

As far as I know there isnt any closed source third party software called local.

Yeah, I think we can assume that /opt/local is not used. It's certainly be crazy for any vendor to try to use that as their directory.

We have /opt/local /var/local

Any other directory would change the symantic to radically. Even /var is more meant for state then binaries. /opt however is being used already similiary to /usr/local by third party vendors. So maybe extend it to the local admin too?

Yeah, /var has a different purpose. /usr and /opt are the only two directories intended for "software", so I think actually using /opt/local for this purpose is better than /var or trying to introduce a completely new top-level directory.

[poettering]

Yeah but we are Distromaintainers. For our stuff we can do that but our Users might try to install something that wants to go into the read only /usr/local. And even then we want it to be bidirectional. Its more about the Users/Local Admins accessing /usr/local.

i don't grok this? can't you just tell users to put their stuff in /var/lib/extensions/? and what do you mean by "bidirectional"?

and why do you think /opt/local would be better to communicate to your users than /var/lib/extensions/?

not following?

[silverhadch]

Yeah but we are Distromaintainers. For our stuff we can do that but our Users might try to install something that wants to go into the read only /usr/local. And even then we want it to be bidirectional. Its more about the Users/Local Admins accessing /usr/local.

i don't grok this? can't you just tell users to put their stuff in /var/lib/extensions/? and what do you mean by "bidirectional"?

and why do you think /opt/local would be better to communicate to your users than /var/lib/extensions/?

not following?

We want /usr/local to remain the familiar path, but backed by a writable directory (in our case, /opt/local). It’s also much more convenient.

“Bidirectional” means that /usr/local acts as a visible, compatible path for software, scripts, and users, but writes actually go into /opt/local — so both /usr/local and /opt/local remain consistent.

The main point is convenience and having /usr/local symlinked out of the image. What we are asking here is what the location for this symlink target is supposed to be.

We also want this for local, system-wide software that we don’t ship on the mkosi image and that isn’t supposed to go into /usr/bin. We could do this via a mutable sysext, allowing writes in and out of /usr/local, but that would completely bypass immutability — making all of /usr, not just the override, writable.

Thus, we want a new location outside of /usr and then symlink /usr/local to that location for compatibility and usability.

[lnussel]

Using /opt seems right and wrong at the same time.

Right because it's wild west there anyway. In reality there seems to be some common pattern where you create a subdirectory and then do whatever you want in there. No rules about directory structure, ownership, no separation of code, config, data and logs (even though /var/opt and /etc/opt are supposed to exist). There have been attempts to integrate with OS vendor package managers but that was a bad idea. Adding another location for the admin to do whatever the admin wants doesn't look like it makes things worse.

Wrong because if some standard reserves a hierarchy in /opt (which likely also means pre-populating it with bin, lib etc.) we're asking OS vendors to touch a location they stayed out of so far. With that in mind a new location with clear rules (like $PATH, man etc integration) somewhere in /var makes more sense. 3rd party add-on managers like flatpak, snap, docker, systemd extensions etc also put their software "packages" in /var anyway. So IMO /opt isn't really needed for anything. It should be phased out and if at all symlinked to somewhere in /var.