feat(deployment): Add new frontend system option

[gashcrumb]

This change adds a useNewFrontendSystem option to configure the RHDH deployment to load the new frontend system app-next package instead of the old frontend system app package. This change also adds the relevant plugins in this mode that are required for logging in using other authentication providers than the guest login as well as source control authentication support. Tests that want to use the new frontend system simply need to set useNewFrontendSystem to "true" in the options object for rhdh.configure().

Assisted-By: Cursor Desktop

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

[subhashkhileri]

I went through the full deployment flow and have a few suggestions to simplify things and fix a couple of merge-order issues. Most of these come from aligning NFS with how the existing config layers (common → auth → user) already work.

For context, the default deployment pipeline follows a consistent pattern across all three config types: merge layer files via mergeYamlFilesIfExists in a specific order (package defaults first, user overrides last), and the later layers win. Auth configs follow this exactly — they're just extra paths spliced into the merge list. NFS should work the same way rather than using a separate loading mechanism.

The custom loader module might be unnecessary. new-frontend-system-plugins.ts exists purely for the env-var overrides (RHDH_E2E_NFS_APP_AUTH_PACKAGE, RHDH_E2E_NFS_APP_INTEGRATIONS_PACKAGE), but users can already override these OCI refs by putting different versions in their workspace's dynamic-plugins.yaml — that's the standard pattern for all other plugins. Could we drop the module, its tests, and the exported constants, and just use mergeYamlFilesIfExists like we do for everything else?

includes gets duplicated. The NFS dynamic-plugins YAML has includes: [dynamic-plugins.default.yaml], but it's always deepMerged into an already-assembled config that already has that includes entry from common/dynamic-plugins.yaml. Since the merge strategy uses byKey: "package" and strings get null merge keys in mergeArraysByKey, they just get appended — so you end up with includes: [dynamic-plugins.default.yaml, dynamic-plugins.default.yaml]. Since this file is never consumed standalone, could we just drop the includes from it?

envsubst ordering. In _applySecrets(), the NFS secrets are merged after envsubst runs on the base secrets. Right now that's fine since the NFS values are all literals (app-next, "true"), but if anyone adds a $VAR reference to the NFS secrets later it'll land as a literal string on the cluster. The default pipeline runs envsubst on the final merged result — common + auth + user all get substituted together. What if we splice the NFS secrets path into the merge layer list (alongside common and auth) before envsubst runs? That way it follows the same pipeline as everything else.

Merge order for dynamic plugins. Currently deepMerge(config, nfsPlugins) means NFS wins over user config — so if a workspace has a different app-auth version in their dynamic-plugins.yaml, the NFS default overwrites it. Could we swap to deepMerge(nfsPlugins, config) so NFS acts as defaults and user config takes precedence? That way workspaces can override NFS plugin versions the same way they override auth plugins today.

Config directory organization. The NFS files in config/common/ aren't really "common" (always applied) — they're conditional on the NFS flag, same as auth/ is conditional on the auth provider. The helm value file (value_file.new-frontend.yaml) is also sitting in config/helm/ but only applies to NFS. Could we give them their own directory to match the existing pattern?

config/
├── common/                    ← always applied
├── auth/                      ← per auth provider
├── helm/                      ← helm-specific
├── operator/                  ← operator-specific
└── new-frontend-system/       ← NFS-specific
    ├── secrets.yaml
    ├── dynamic-plugins.yaml
    └── value_file.yaml

Then in constants.ts, newFrontendSystem groups all three paths (secrets, dynamicPlugins, valueFile) under one key — same as how helm and operator group theirs.

Auto-detection from project name. Instead of always requiring useNewFrontendSystem: true in configure(), what if we auto-detect it when the Playwright project name ends with -app-next (since project name = namespace) or when a USE_NEW_FRONTEND_SYSTEM=true env var is set? Explicit true/false would still take precedence via ??:

const namespace = input.namespace ?? this.deploymentConfig.namespace;
const useNewFrontendSystem =
  input.useNewFrontendSystem ??
  (namespace.endsWith("-app-next") ||
    process.env.USE_NEW_FRONTEND_SYSTEM === "true");

This way existing tests that already use -app-next as a project name suffix don't need any code changes — they'd just start getting NFS behavior automatically. And the USE_NEW_FRONTEND_SYSTEM env var gives us the ability to run all tests with the new frontend system without touching individual test configs.

[gashcrumb]

Thanks @subhashkhileri that was very helpful, have updated these changes accordingly. Lemme know if there's any other concerns.