Skip to content

fix(ui): recall the right VAE and text encoder per model base - #9489

Open
Pfannkuchensack wants to merge 2 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/metadata-recall-anima-gating
Open

fix(ui): recall the right VAE and text encoder per model base#9489
Pfannkuchensack wants to merge 2 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/metadata-recall-anima-gating

Conversation

@Pfannkuchensack

Copy link
Copy Markdown
Member

Summary

Model-specific VAEs and text encoders were recalled into the wrong Redux slot, or not recallable at all, depending on the model base. This fixes the gating, wires the missing handlers into the recall UI, and adds the Anima denoise node needed to produce that metadata from a workflow.

Several bases keep their standalone components in dedicated params slots (params.fluxVAE, params.animaVaeModel, …) but record them in the shared metadata.vae / metadata.qwen3_encoder fields. Whether a handler may fire therefore depends on two axes: where the image came from (metadata.model.base) and which base is currently selected (which decides which slot is live). Neither axis was checked consistently.

What was broken

Symptom Cause
Anima images showed a duplicate VAE row, and "recall all" also wrote the Anima VAE into the SD params.vae slot VAEModel excluded z-image, flux2, krea-2 — but not anima
Anima and FLUX.2 Klein images pushed their encoder into the Z-Image slot and cleared zImageQwen3SourceModel Qwen3EncoderModel had no base gate at all, yet dispatched hard into the Z-Image slots
A Krea-2 image recalled while Anima was selected wrote its VAE into the Anima slot The Anima handlers gated only on the selected base, never on the image's provenance
FLUX.1: the VAE recall button did nothing The graph reads params.fluxVAE, the generic handler writes params.vae — no handler ever dispatched fluxVAESelected
Anima, Z-Image and FLUX.1 VAE / encoder had no row and no recall button in the metadata panel The handlers were missing from the hand-maintained IMAGE_METADATA_ACTION_HANDLERS
Anima workflows could only record metadata by hand-chaining Metadata Item Linked nodes metadata_linked.py had a self-recording denoise node for SD, FLUX.1 and Z-Image, but not for Anima

The first three were partly self-healing — the modelSelected listener clears foreign slots on the next model switch — but the panel stayed wrong and a user's Z-Image encoder selection was silently dropped.

Changes

features/metadata/parsing.tsx

  • New BASES_WITH_DEDICATED_VAE_HANDLER set replaces the base !== 'x' && … chain in VAEModel, documenting per base which slot and handler own it. qwen-image and wan are deliberately absent — they write qwen_image_vae / wan_vae_model and never collide.
  • New Flux1VAEModel handler dispatching fluxVAESelected.
  • Qwen3EncoderModelZImageQwen3EncoderModel, gated on base === 'z-image'. A variant-based check would not work here: isQwen3EncoderModelConfig is defined as variant !== 'qwen3_06b', which Klein encoders satisfy too — and the slot choice is a property of the main model, not of the encoder.
  • Both Anima handlers now check provenance via assertMetadataModelBase. The VAE additionally asserts parsed.base === 'anima' (meaningful, since isAnimaVAEModelConfig is base-driven); the encoder deliberately gets no base assert, since Anima encoders are identified by variant — with a comment saying so, to stop a future cleanup from "fixing" it.

ImageMetadataViewer/ImageMetadataActions.tsx — list Flux1VAEModel, ZImageVAEModel, ZImageQwen3EncoderModel, ZImageQwen3SourceModel, AnimaVAEModel, AnimaQwen3EncoderModel. Listing Flux1VAEModel is mandatory, not cosmetic: without it, adding flux to the exclusion set would remove the FLUX.1 VAE row outright.

invocations/metadata_linked.py — new anima_denoise_meta node ("Denoise - Anima + Metadata"), mirroring z_image_denoise_meta with two Anima specifics: the CFG value is recorded as cfg_scale (Z-Image writes guidance, which for Anima would land in the FLUX guidance param), and the standalone VAE / Qwen3 encoder are optional inputs, since Anima cannot derive them from the transformer field. Unconnected inputs write nothing rather than a null the recall handlers would fail to resolve. schema.ts regenerated.

Tests

  • parsing.test.tsx: gating for AnimaVAEModel (incl. rejecting Krea-2 provenance and metadata with no model at all), AnimaQwen3EncoderModel (incl. pinning that any encoder base parses), ZImageQwen3EncoderModel, Flux1VAEModel; the generic VAEModel matrix extended by flux and anima.
  • ImageMetadataActions.test.tsx: a toContain check for the six newly listed handlers, plus a guard test that diffs the whole registry against the UI list. Its allowlist is split into "hidden by design" (CreatedBy, ImageSize) and a commented snapshot of pre-existing gaps (HiDiffusion, Wan, Qwen-Image sub-models, Gemini/OpenAI/Seedream, Z-Image seed variance) — meant to shrink, not grow.
  • test_denoise_noise_inputs.py: the seed-selection case matching the FLUX / Z-Image tests, plus one that pins the metadata keys against the recall contract (cfg_scale set, guidance absent, model / VAE / encoder recorded) and one that checks unconnected component inputs are omitted.

pnpm lint, pnpm lint:tsc, the full frontend suite (1809 tests / 144 files), ruff, and the touched pytest modules all pass.

Out of scope

  • T5EncoderModel has the same missing base gate and writes into the FLUX.1 T5 slot. Left alone deliberately.
  • Krea-2, Qwen-Image and FLUX.2 still have no *_denoise_meta counterpart either; only Anima is added here.
  • Note that the provenance check requires model (with base: "anima") in the metadata for the Anima VAE / encoder rows to recall. The new node always writes it; a hand-built Metadata Item Linked chain that omits it will not recall those two fields.

Manual verification

  1. Anima image → metadata panel shows one VAE row and one Qwen3 Encoder row, each with a working recall button.
  2. "Recall all" on an Anima image while SDXL is selected: model switches to Anima, both Anima slots hold the values from the image (not the listener's defaults), the SD VAE slot stays empty, Z-Image slots untouched.
  3. FLUX.1: the VAE recall now lands in the FLUX VAE select.
  4. An Anima workflow using Denoise - Anima + Metadata, with the loader's VAE and Qwen3 Encoder connected and its metadata output wired into the L2I node, produces an image whose parameters recall in full.
  5. SDXL and FLUX.2 Klein behave as before, in particular no VAE row disappeared.

Several bases write their standalone VAE / text encoder into the shared
`metadata.vae` and `metadata.qwen3_encoder` fields but recall them into their
own params slot. The generic handlers did not account for that:

- `VAEModel` excluded z-image, flux2 and krea-2 but not anima, so an Anima
  image rendered a duplicate VAE row and additionally wrote its VAE into the
  shared `params.vae` slot, which no Anima graph reads.
- `Qwen3EncoderModel` had no base gate at all while dispatching hard into the
  Z-Image slots, so Anima and FLUX.2 Klein images pushed their encoder into
  `zImageQwen3EncoderModel` and cleared `zImageQwen3SourceModel` on the way.
- The Anima handlers gated only on the currently selected base, never on the
  image's own `metadata.model.base`, so a Krea-2 image recalled while Anima was
  selected wrote its VAE into the Anima slot.

Replace the growing `base !== 'x' && ...` chain with a documented
`BASES_WITH_DEDICATED_VAE_HANDLER` set, rename `Qwen3EncoderModel` to
`ZImageQwen3EncoderModel` and gate it on z-image, and harden both Anima handlers
with `assertMetadataModelBase` (plus a base assert on the VAE, which is
base-identified - the encoder is variant-identified, so it deliberately gets
none).

The same slot mismatch existed for FLUX.1, whose VAE lives in `params.fluxVAE`:
its recall silently landed in `params.vae` and had no effect on generation. Add
a dedicated `Flux1VAEModel` handler for it.

Finally, `AnimaVAEModel`, `AnimaQwen3EncoderModel`, `Flux1VAEModel` and the
three Z-Image handlers were missing from IMAGE_METADATA_ACTION_HANDLERS, so
those parameters had neither a row nor a per-parameter recall button. List them,
and add a guard test that diffs the UI list against the handler registry with an
explicit allowlist, so the list cannot drift behind again.
Workflows only carry recallable parameters if the graph builds the metadata
itself. `metadata_linked.py` offers two ways to do that: chaining Metadata Item
Linked nodes, or a denoise node that writes its own metadata - but the latter
existed only for SD, FLUX.1 and Z-Image.

Add `anima_denoise_meta`, mirroring `z_image_denoise_meta`, with two Anima
specifics:

- The CFG value is recorded as `cfg_scale`, matching both the Anima graph
  builder and the UI's recall handler. Z-Image writes `guidance`, which for
  Anima would land in the FLUX guidance param instead.
- The VAE and Qwen3 encoder are exposed as optional inputs. Anima loads both
  standalone, so unlike SD/FLUX they cannot be derived from the transformer
  field, and without them a recalled Anima image would be missing exactly the
  two components that are hardest to pick by hand. Unconnected inputs write
  nothing rather than a null the recall handlers would fail to resolve.
@github-actions github-actions Bot added python PRs that change python files invocations PRs that change invocations frontend PRs that change frontend files python-tests PRs that change python tests labels Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files python-tests PRs that change python tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant