Skip to content

feat(vertex_ai): support GenerateContentConfig labels for Dify app_id propagation#3168

Draft
mas-sakai wants to merge 13 commits into
langgenius:mainfrom
mas-sakai:feat/vertex-ai-app-id-labels
Draft

feat(vertex_ai): support GenerateContentConfig labels for Dify app_id propagation#3168
mas-sakai wants to merge 13 commits into
langgenius:mainfrom
mas-sakai:feat/vertex-ai-app-id-labels

Conversation

@mas-sakai
Copy link
Copy Markdown

Note

This is a Draft PR. It depends on
langgenius/dify-plugin-sdks#313
being merged and a new SDK released. pyproject.toml / uv.lock currently
pin dify_plugin to the #313 branch so the Gemini route can import
get_current_session(). The pin will be reverted to a released version
constraint (dify_plugin>=0.3.0,<0.6.0) before this PR is marked Ready
for review. The CI "Validate Dependencies" / "Check Plugin Install"
steps may fail until then.

Summary

Enable Cloud Billing visibility per Dify app for the Vertex AI plugin (Gemini
route). When the new enable_request_metadata credential is set to enabled,
the plugin attaches dify_app_id and dify_source to
GenerateContentConfig.labels, so operators can break Vertex AI cost down by
labels.dify_app_id in GCP Cloud Billing without any ETL.

Related issues / PRs:

Scope (intentionally narrow)

  • Gemini route only (models/llm/llm.py::_generate). The Anthropic-on-Vertex
    route (_generate_anthropic) and other providers are out of scope.
  • Label keys: dify_app_id and dify_source only. dify_tenant_id /
    dify_user_id are deferred to a follow-up to keep the surface minimal.
  • Default-off: enable_request_metadata defaults to disabled, so existing
    users who do not touch their credentials are completely unaffected. With
    disabled, the control flow is identical to the previous version.

Implementation notes

  • New helper models/llm/_labels.py with normalize_label_value and
    build_dify_labels. Output conforms to the GCP labels spec (lowercase,
    [a-z0-9_-] only, 63-char cap). UUID app_ids pass through unchanged.
  • get_current_session() is imported locally inside _generate and wrapped in
    try/except ImportError, so the plugin keeps working against older
    dify_plugin releases (no hard runtime dependency on fix getnumtoken retrun list[int] #313 once it ships).
  • 9 unit tests added in tests/test_labels.py (passthrough, character
    replacement, truncation, casing, empty input, None / empty short-circuits
    in build_dify_labels).

Change Type

  • Documentation / non-plugin change
  • Non-LLM plugin (tools, extensions, datasource, etc.)
  • LLM plugin

Screenshots / Videos

🚧 Cloud Billing breakdown screenshots (by labels.dify_app_id) will be
added once E2E verification completes
(currently testing on GCP project
dify-vertex-labels-poc). All 9 unit tests pass (uv run pytest tests/).

Before After

LLM Plugin Checklist

Areas affected by this change (check all that apply)
  • Message flow (system messages, user ↔ assistant turn-taking)
  • Tool interaction flow (multi-round usage, Agent App and Agent Node)
  • Multimodal input (images, PDFs, audio, video, etc.)
  • Multimodal output (images, audio, video, etc.)
  • Structured output (JSON, XML, etc.)
  • Token consumption metrics
  • Other LLM functionality (reasoning, grounding, prompt caching, etc.) — request metadata / Cloud Billing labels
  • New models / model parameter fixes

Version

  • Bumped top-level version in manifest.yaml (not the one under meta) — 0.0.530.0.54
  • dify_plugin>=0.3.0,<0.6.0 is declared in pyproject.toml and locked in uv.lock (or kept in requirements.txt for legacy plugins without uv.lock) — SDK docs

⚠️ The dify_plugin constraint is currently a temporary git-branch pin
targeting #313.
It will be reverted to dify_plugin>=0.3.0,<0.6.0 before this PR leaves
Draft.

Testing

  • Local deployment — Dify version: 1.14.x (E2E Cloud Billing breakdown verification in progress on GCP project dify-vertex-labels-poc)
  • SaaS (cloud.dify.ai)

Unit tests: uv run pytest tests/ — 9 passed (label normalization helper).

mas-sakai and others added 5 commits May 19, 2026 14:28
Add `_labels.py` with `normalize_label_value` and `build_dify_labels`
helpers that produce GCP-compliant Vertex AI label dicts. UUID app_ids
pass through unchanged; other strings are lowercased, restricted to
`[a-z0-9_-]`, and truncated to 63 characters. Unit tests cover passthrough,
character replacement, truncation, casing, empty input, and the `None`/
empty short-circuits in `build_dify_labels`.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When `enable_request_metadata` is `enabled` in credentials, attach
`dify_app_id` and `dify_source` to `GenerateContentConfig.labels` for the
Gemini route only. The `get_current_session` import is local to the call
site and wrapped in `try/except ImportError` so the plugin keeps working
against older `dify_plugin` releases. With the default `disabled` setting,
control flow is identical to the previous version.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an opt-in `enable_request_metadata` credential (select, defaults to
`disabled`) so operators can choose to attach Dify metadata to Vertex AI
requests for Cloud Billing breakdown. Existing users who do not touch
their credentials are unaffected.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Point `dify_plugin` at the `feat/pass-session-to-model-plugins` branch of
`ryuta-kobayashi-ug/dify-plugin-sdks` (PR langgenius#313) so the Gemini route can
import `get_current_session()`. Refresh `uv.lock` accordingly. This pin
is temporary for the review period; revert to a release-version
constraint once langgenius#313 is merged and a new SDK is published.

Also add `PR_BODY.md` as a draft of the upstream PR body so it is
available alongside the branch.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mas-sakai mas-sakai temporarily deployed to models/vertex_ai May 20, 2026 05:45 — with GitHub Actions Inactive
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the capability to track Cloud Billing costs on a per-application basis by attaching Dify metadata as labels to Vertex AI requests. The implementation includes a utility for normalizing label values according to GCP constraints, logic in the Gemini model to extract session metadata, and a new configuration setting to enable this feature. Feedback suggested optimizing the label normalization helper by truncating long input strings prior to processing to enhance performance and reduce memory usage.

Comment thread models/vertex_ai/models/llm/_labels.py
mas-sakai and others added 2 commits May 22, 2026 09:56
Remove `PR_BODY.md` (and the local `PR_BODY_EN.md` working copy) from
the repository. The content lives in the PR description on GitHub; the
checked-in copies are working notes that would otherwise land in main
with the merge.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The descriptive text for the `enable_request_metadata` credential was
placed in `placeholder`, but `type: select` is rendered as a dropdown
and does not surface placeholder text. Move the description to `help`
so it actually shows in the credentials UI, matching the convention
used by the openai provider's select credentials.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mas-sakai mas-sakai temporarily deployed to models/vertex_ai May 22, 2026 00:57 — with GitHub Actions Inactive
@mas-sakai mas-sakai marked this pull request as ready for review May 26, 2026 02:46
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request labels May 26, 2026
mas-sakai and others added 2 commits May 26, 2026 12:07
Broaden the exception catch around dify_plugin.get_current_session() to
`Exception` so that contextvars LookupError, RuntimeError, or any other
runtime issue in best-effort telemetry cannot break model generation
(previously only ImportError was caught, which left the runtime path
unprotected).

In normalize_label_value, coerce non-string input via str() before the
empty-check so numeric `0` becomes "0" instead of being dropped as
falsy. Adds a regression test.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Surface zh_Hans copy for the new credential's label, enabled/disabled
option labels, and help text. Phrasing follows the Cloud Billing /
labels framing of the existing help body, not the Bedrock CloudWatch
copy.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…labels per review

- Widen `normalize_label_value` / `build_dify_labels` type hints to
  `Any` to match the str() coercion the implementations already do.
- Reject only `None` and `""` in `build_dify_labels`; other falsy
  values (e.g. numeric 0) flow through to normalization rather than
  being silently dropped, matching the design intent for the helper.
- Centralize the session lookup + label injection in a new
  `apply_dify_labels_if_enabled` helper that merges into existing
  `config_kwargs["labels"]` instead of overwriting it. Aligns with the
  OpenAI plugin's `apply_dify_metadata_if_enabled` structure.
- Add tests covering the merge-with-existing path, the non-dict
  fallback, the disabled/missing credential no-ops, the session-lookup
  failure swallowing, and the falsy-but-non-empty input case.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels May 26, 2026
Bump vertex_ai manifest from 0.0.54 to 0.0.56 to leapfrog main's 0.0.55
so the marketplace registration does not collide. Adopt upstream's
refreshed dependency floors in pyproject.toml while keeping the
dify_plugin git+ pin to sdks langgenius#313, then regenerate uv.lock from the
merged pyproject. pytest passes (16/16).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mas-sakai mas-sakai requested a review from a team May 27, 2026 01:09
@mas-sakai mas-sakai temporarily deployed to models/vertex_ai May 27, 2026 01:11 — with GitHub Actions Inactive
@mas-sakai mas-sakai marked this pull request as draft May 27, 2026 01:13
Addresses gemini-code-assist suggestion on langgenius#3168: bound the cost of
lower() and the regex sub() by slicing the input down to 63 chars first.
Keep the trailing truncation because lower() can still expand certain
Unicode characters (e.g. German sharp s -> "ss"), so the final cap acts
as a defense-in-depth net.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mas-sakai mas-sakai temporarily deployed to models/vertex_ai May 27, 2026 01:20 — with GitHub Actions Inactive
mas-sakai added a commit to mas-sakai/dify-official-plugins that referenced this pull request May 27, 2026
Applies the same ordering fix discussed on langgenius#3168 to the Bedrock helper:
slice down to 256 chars before the regex sub() so pathological inputs do
not incur unbounded regex work. The character set used by sub() replaces
1:1 by length, so the trailing truncation is no longer needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@WH-2099 WH-2099 removed the request for review from a team May 27, 2026 14:18
…malize per review

Per gemini-code-assist on langgenius#3233. Aligns 4 plugins on the same
'no side effects on existing args' principle.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant