Skip to content

feat(outlink): support rich format in outlink messages - #7353

Open
Nixieboluo wants to merge 24 commits into
labring:mainfrom
Nixieboluo:feat/outlink-media-support
Open

feat(outlink): support rich format in outlink messages#7353
Nixieboluo wants to merge 24 commits into
labring:mainfrom
Nixieboluo:feat/outlink-media-support

Conversation

@Nixieboluo

@Nixieboluo Nixieboluo commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR introduces the shared FastGPT runtime and media utilities used by rich-message outlink providers.

The outlink runtime exposes a provider-neutral async iterable response API. Provider integrations normalize incoming platform messages into FastGPT chat queries and consume ordered start, chunk, done, and error events without duplicating workflow execution, chat persistence, usage accounting, or common error handling.

In addition to the shared runtime, this PR adds the personal WeChat channel implementation and the shared utilities required by the coordinated provider migrations in the companion Pro PR.

What changed

Shared outlink runtime

  • Extracted outlink workflow execution into the shared runOutlinkRuntime service.
  • Added the async iterable responder contract used by provider adapters.
  • Kept duplicate-message detection, outlink limits, workflow file limits, workflow dispatch, chat persistence, and workflow usage accounting inside the shared runtime.
  • Ensures responder failures terminate the response stream so later workflow chunks are not retained by an abandoned consumer.

Personal WeChat channel

  • Added iLink QR-code login, status polling, token lifecycle handling, and long-poll message processing.
  • Added personal WeChat text, image, voice, and file message normalization.
  • Added encrypted/CDN media download handling and deferred resource resolution after workflow file limits are known.
  • Preserves WeChat message_id losslessly as a string before it is used for grouping and BullMQ reply job IDs.

Shared query, media, and configuration utilities

  • Added provider configuration loading with channel matching and Zod validation.
  • Added the Feishu provider credential schema, including the optional event encryption key.
  • Added shared helpers for composing queries and quoting referenced messages.
  • Added a byte-limited readable stream wrapper for externally downloaded files.
  • Updated audio transcription to accept a general Node.js Readable.
  • Moved STT usage recording into the service package and made its source explicit.

Related channel support

  • Includes the WeCom route and configuration changes required by the shared runtime migration.
  • Includes the shared DingTalk access-token support required by the coordinated provider work.

APIs introduced

Outlink runtime

  • runOutlinkRuntime<T>({ outLinkConfig, message, respond })

    • Runs an outlink chat through the standard FastGPT workflow runtime.
    • Persists the chat round and workflow usage data.
    • Sends ordered response events to the provider adapter.
  • OutlinkMessage

    • Normalized provider message containing chatId, messageId, chatUserId, and FastGPT query items.
  • OutlinkResponseEvent

    • { type: 'start' }
    • { type: 'chunk'; content: string }
    • { type: 'done'; content: string }
    • { type: 'error'; content: string }
  • OutlinkResponder

    • Consumes AsyncIterable<OutlinkResponseEvent> and delivers events through a provider-specific transport.
  • OutlinkProviderMessageHandler<T>

    • Shared handler signature used to connect a provider implementation to runOutlinkRuntime.

Provider configuration

  • FeishuAppSchema

    • Validates appId, appSecret, and optional encryptKey.
  • loadOutlinkProviderConfig<T>({ shareId, channel, appSchema })

    • Loads an outlink by share ID and publish channel.
    • Validates the stored provider configuration before returning a typed result.

Query and media utilities

  • composeOutLinkQuery(...queries)

    • Merges text into one ordered text item and preserves file items.
  • citeOutLinkQuery(query)

    • Wraps referenced text in <Cite> while preserving attached files.
  • createOutLinkFileLimitStream({ source, maxBytes })

    • Enforces the actual downloaded byte count while streaming.
    • Destroys the source stream when the limit is exceeded or consumption ends.
  • OutLinkFileSizeExceededError

    • Typed error containing the configured byte limit.

Audio and usage

  • aiTranscriptions({ model, fileStream, filename, headers })

    • Accepts a general Node.js Readable as the audio input.
  • pushWhisperUsage({ teamId, tmbId, duration, source })

    • Records STT usage with an explicit API or outlink source.

Compatibility

outlinkInvokeChat and packages/service/support/outLink/runtime/utils.ts are intentionally removed.

All channel integrations covered by this PR pair use runOutlinkRuntime through provider adapters. This is an internal service API migration and does not add a new public HTTP endpoint.

Companion PR

The companion Pro PR migrates Feishu, DingTalk, Official Account, and WeCom providers to this shared runtime:

https://github.com/labring/fastgpt-pro/pull/1027

Validation

  • Covers lossless WeChat uint64 message ID parsing and grouping.
  • Covers response-stream cleanup after responder failure.
  • Covers provider-specific streaming, identity, media, and validation behavior in the companion Pro PR.

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 26.87% 2499 / 9299
🔵 Statements 26.75% 2624 / 9807
🔵 Functions 22.46% 504 / 2243
🔵 Branches 21.65% 1159 / 5351
File CoverageNo changed files found.
Generated in workflow #2186 for commit 7f6aeb5 by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

Admin Preview Image Ready!

ghcr.io/labring/fastgpt-pr:admin_4cec6a2337853ddfc13f1575359cfe7d430299ba

🕒 Time: 2026-08-11 18:01:13 (UTC+8)

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

Build Successful - Preview fastgpt Image for this PR:

ghcr.io/labring/fastgpt-pr:fastgpt_7f6aeb5d64d9fe3fbadbf6a6cb9648c7e7d2f401

🕒 Time: 2026-08-12 16:16:56 (UTC+8)

@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch from 47ec0f5 to 179e105 Compare July 24, 2026 02:49
@c121914yu
c121914yu self-requested a review July 26, 2026 09:20

@c121914yu c121914yu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

基于 179e105 的定向复核,新增 runtime 的主流程测试已通过,但建议合并前先确认上面两处兼容性问题。另有一个未阻断的架构风险:Readable.push() 的背压结果没有被处理,慢 responder 可能积累事件;建议补充慢/失败 responder 和 Redis 故障场景测试。FastGPT 定向测试 4 个 suite、19 个测试通过;git diff --check 通过。

Comment thread packages/service/support/outLink/runtime/service.ts
Comment thread packages/service/support/outLink/runtime/service.ts
@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch from 7e89f39 to 02a85f9 Compare August 4, 2026 03:36
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
Signed-off-by: Nixieboluo <me@sagirii.me>
@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch from 342f3e7 to 19df6e7 Compare August 4, 2026 06:15
@c121914yu

Copy link
Copy Markdown
Collaborator

基于当前 head 19df6e7e 的复核,建议合并前处理或明确以下问题:

  1. [阻断] 微信消息 ID 存在 uint64 精度丢失。 ilinkClient.ts#L52-L58 将超过 Number.MAX_SAFE_INTEGERmessage_id 定义为 numbergetUpdates 又直接 JSON.parsemessageParser.ts#L29-L56 转成字符串时精度已经丢失。相邻 ID 可能变成同一个值,进而导致 BullMQ jobId 冲突或 runtime 误判重复消息。建议在 HTTP 边界无损解析为字符串,并补超过安全整数范围的相邻 ID 测试。

  2. [兼容性] PR 描述与最终实现不一致。 描述称 outlinkInvokeChat 仍作为 deprecated compatibility adapter 保留,但最终 head 已删除 runtime/utils.ts,实现中也没有该符号,现有设计文档仍引用旧接口。请恢复兼容 wrapper,或明确这是破坏性内部 API 变更并同步 PR 描述、设计文档及所有外部消费者。

  3. [运行时风险] responder 队列没有背压控制。 runtime/service.ts#L188-L201 忽略 Readable.push() 的返回值;慢速卡片更新或 Redis responder 下,workflow chunk 会持续堆积。建议增加有界队列/背压策略,并覆盖慢 responder 与投递失败场景。

  4. [测试缺口] 1dad863 删除了原有微信 messageParser 测试以及本 PR 中新增的 adapter 测试,最终没有覆盖大整数 ID、消息分组、AES/CDN 媒体和 runtime adapter。当前 CI 通过不能覆盖上述行为风险。

此外,本 PR 实际包含个人微信协议/二维码状态/富媒体、企微迁移及钉钉共享 token 等用户可见或跨模块改动,范围明显大于正文中的 shared runtime 描述。建议拆分,或补充按渠道列出的需求和验收矩阵。

@Nixieboluo

Copy link
Copy Markdown
Collaborator Author

修复:

  1. message_id 现在通过一个自定义的 reviver 解析为 string。
  2. PR 描述已更新。
  3. 现在在 responder 失败时会正确关闭流。背压控制可以不加,因为模型输出速度和输出数据量不大,且忽略chunk并不影响计费和持久化等流程。
  4. 补充了一些测试。

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Docs Preview Deployed!

🔗 👀 Click here to visit preview

ghcr.io/labring/fastgpt-docs-pr:7f6aeb5d64d9fe3fbadbf6a6cb9648c7e7d2f401

🕒 Time: 2026-08-12 16:10:29 (UTC+8)

…edia-support

# Conflicts:
#	.agents/design/outlink/wechat-clawbot.md
#	.agents/design/outlink/wechat-polling-refactor.md
#	packages/service/core/dataset/apiDataset/dingtalkDataset/api.ts
#	packages/service/support/outLink/runtime/utils.ts
#	packages/service/support/outLink/wechat/mq.ts
#	packages/service/support/outLink/wechat/type.ts
#	packages/service/test/core/dataset/apiDataset/dingtalkDataset/api.test.ts
#	packages/service/test/support/outLink/runtime/utils.test.ts
#	pro
@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch 2 times, most recently from a614d8e to 7037f4a Compare August 5, 2026 10:28
Signed-off-by: Nixieboluo <me@sagirii.me>
@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch from 7037f4a to 3d443a0 Compare August 5, 2026 10:30
@c121914yu

Copy link
Copy Markdown
Collaborator

[P1] 微信 iLink 大整数修复仍不兼容仓库声明支持的 Node 20

当前实现依赖 JSON.parse reviver 的第三个 context 参数及 context.source。这个能力在 Node 20.19.0 中不存在;用本次测试中的 uint64 message_id 复现时,contextundefinedgetUpdates 会直接抛错。与此同时,projects/app@fastgpt/serviceengines.node 仍声明为 >=20.19.0,因此按仓库支持版本部署的源码或私有化环境会无法轮询微信消息。

这里有可降级兼容的无损方案:在 @fastgpt/service 显式声明大整数 JSON 解析器作为运行时依赖,例如使用 json-bigint({ storeAsString: true }) 解析原始响应,再把每条消息的 message_id 统一通过 String(...) 归一化。Node 20.19 下实测,18446744073709551615 会保持为精确字符串,而普通数值字段仍是 number。当前锁文件中的 json-bigint 仅来自开发依赖的传递依赖,不能直接依赖它。另一种方案是统一把 app、service、镜像和 CI 的最低 Node 契约提升到支持该 API 的版本,但这会成为更大范围的破坏性变更。

请补一条在最低支持版本 Node 20.19 下运行的 uint64 响应测试,避免当前 Node 24 环境把兼容问题隐藏掉。

@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch 2 times, most recently from 42b665d to 11e0009 Compare August 6, 2026 02:57
@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch 3 times, most recently from bccf7e7 to 8ff980c Compare August 11, 2026 02:56
@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch from 8ff980c to 31e1d05 Compare August 11, 2026 03:09
Signed-off-by: Nixieboluo <me@sagirii.me>
@Nixieboluo
Nixieboluo force-pushed the feat/outlink-media-support branch from 9fd2568 to 4cec6a2 Compare August 11, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants