Skip to content

feat(protocol,onebot): raise video size limit to 1.5GB with Highway-first fallback - #35

Merged
Stlara-F merged 1 commit into
devfrom
feat/large-video-upload-fallback
Jun 26, 2026
Merged

Stlara-F merged 1 commit into
devfrom
feat/large-video-upload-fallback

Conversation

@Stlara-F

@Stlara-F Stlara-F commented Jun 26, 2026 •

Copy link
Copy Markdown
Owner

概述

对大于 100MB 小于 1.5GB 的视频上传做两层兜底:不再硬拒绝(throw),改为警告后先用 Highway 尝试上传;若 Highway 失败,自动降级为文件上传。

变更清单

文件路径 变更描述
packages/protocol/src/highway/video-upload.ts 导出 MAX_VIDEO_SIZE;MAX_VIDEO_SIZE_HARD=1.5GB;>100MB 时 warn+proceed;新增 getVideoSourceSize
packages/protocol/src/element-builder.ts makeVideoElem try/catch,group 降级为 makeGroupFileElem
packages/onebot/src/modules/message-actions.ts sendPrivateMessage catch 大视频错误后转为 file 走文件管道

设计决策

  • Layer 1 (Highway): 将 hard limit 提升到 1.5GB(低于 Node.js fs.readFileSync 约 2GiB 上限)
  • Layer 2 (File): group 在 element-builder 降级为 file element;private 在 message-actions 转为 file(element 层不支持 private file)
  • fallback 仅触发于 size 相关错误(error message 中匹配 size limit/too large/413/exceed)
  • 远端视频(size 不可推断)也会在 size 错误时触发降级

Review 修复记录

Round 修复内容
R1 4GB 降至 1.5GB 硬上限(fs.readFileSync 约 2GiB 限制)
R1 catch 收窄为 size 相关错误 + 错误消息启发式检查
R1 处理所有超量视频而非仅第一个
R1 远端视频 sz===null 时通过 isSizeErr 触发降级
R2 两个 throw 站点错误消息从硬编码 4096 MB 改为引用 MAX_VIDEO_SIZE_HARD 常量

测试验证

  • pnpm typecheck: 全部包通过
  • pnpm test: core 447/448 passed(1 个 bridge-private-routing 预存超时波动,与本 PR 无关)

自检

  • 基于 dev 分支
  • pnpm typecheck + pnpm test 通过
  • 一个 PR 只做一件事
  • Commit 格式符合 conventional commits

@sourcery-ai

sourcery-ai Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

审阅者指南

将视频上传的软限制从 100MB 提升至 4GB,并加入两层回退机制:首先尝试使用 Highway 上传(最高 4GB),如果失败,则在群聊和私聊消息中降级为文件上传;同时新增用于检测大小的辅助方法,并在 element 构建和 OneBot 发送路径中增加针对性的错误处理。

私聊视频上传降级为文件的序列图

sequenceDiagram
  actor Client
  participant OneBot as sendPrivateMessage
  participant MsgAPI as bridge.apis.message
  participant FileAPI as bridge.apis.file

  Client->>OneBot: sendPrivateMessage(userId, elements)
  OneBot->>OneBot: split elements into allFileElements & nonFileElements
  alt nonFileElements not empty
    OneBot->>MsgAPI: sendPrivate(userId, nonFileElements)
    MsgAPI-->>OneBot: error (video upload failed)
    OneBot->>OneBot: getVideoSourceSize(videoElement)
    alt size > MAX_VIDEO_SIZE
      OneBot->>OneBot: log.warn fallback to file upload
      OneBot->>OneBot: convert video element to file MessageElement
      OneBot->>OneBot: update allFileElements & nonFileElements
      opt nonFileElements still not empty
        OneBot->>MsgAPI: sendPrivate(userId, nonFileElements)
        MsgAPI-->>OneBot: lastReceipt
        OneBot->>OneBot: logSentMessage(false, userId, nonFileElements)
      end
    else no large video element
      OneBot-->>Client: rethrow error
    end
  end
  opt allFileElements not empty
    OneBot->>FileAPI: uploadPrivate / sendC2cFile(allFileElements)
    FileAPI-->>OneBot: file upload receipts
  end
  OneBot-->>Client: final lastReceipt and file results
Loading

文件级变更

Change Details Files
将 Highway 视频上传大小限制放宽至 4GB,在 100MB 设置软警告阈值,并引入共享辅助方法用于确定视频源大小。
  • 导出现有的 100MB MAX_VIDEO_SIZE 常量,并新增 4GB 的 MAX_VIDEO_SIZE_HARD 硬限制。
  • 新增 getVideoSourceSize,用于从元素元数据或本地文件系统路径推导视频大小。
  • 更新本地路径的暂存逻辑:对超过 4GB 的文件进行硬拒绝;对 100MB 到 4GB 之间的文件仅做警告(不抛错)。
  • 修改远程/二进制加载逻辑,使其允许最高 4GB 的文件,并在暂存完成后再次应用硬限制,将之前针对 >100MB 的错误转换为警告。
packages/protocol/src/highway/video-upload.ts
在 Highway 视频上传失败时,为群消息添加从视频元素回退到文件元素的机制。
  • 用 try/catch 包裹 makeVideoElem 的视频上传路径,以拦截 uploadVideoMsgInfo 失败。
  • 当群消息且存在 fileId 时,如果失败,则记录控制台警告,并委托给 makeGroupFileElem 以文件元素的形式发送。
  • 对于非群消息或无 fileId 的情况,保持原有错误行为:直接原样重新抛出错误。
packages/protocol/src/element-builder.ts
在 Highway 视频上传失败时,为私聊消息添加从大视频元素回退到文件上传的机制。
  • 修改 sendPrivateMessage 以使用可变的文件元素和非文件元素数组,以便在出错时进行调整。
  • 用 try/catch 包裹初始的非文件 sendPrivate 调用,以检测上传失败。
  • 在出错时,使用 getVideoSourceSize 和 MAX_VIDEO_SIZE 检测是否由超过软限制大小的视频元素触发了失败。
  • 如果找到这样的视讯:记录警告,将该视频转换为文件 MessageElement,追加到文件元素列表中,重新发送剩余的非文件元素,然后继续执行现有的文件上传流程;否则,重新抛出原始错误。
packages/onebot/src/modules/message-actions.ts

技巧与命令

与 Sourcery 交互

  • 触发新的审查: 在 pull request 上评论 @sourcery-ai review。
  • 继续讨论: 直接回复 Sourcery 的审查评论。
  • 从审查评论生成 GitHub issue: 在审查评论下回复,要求 Sourcery 从该评论创建一个 issue。你也可以直接回复审查评论 @sourcery-ai issue 来从中创建 issue。
  • 生成 pull request 标题: 在 pull request 标题中任意位置写上 @sourcery-ai,即可在任意时间生成标题。你也可以在 pull request 上评论 @sourcery-ai title 来(重新)生成标题。
  • 生成 pull request 摘要: 在 pull request 正文中任意位置写上 @sourcery-ai summary,即可在你想要的位置生成 PR 摘要。你也可以在 pull request 上评论 @sourcery-ai summary 来在任意时间(重新)生成摘要。
  • 生成审阅者指南: 在 pull request 上评论 @sourcery-ai guide,即可在任意时间(重新)生成审阅者指南。
  • 一次性解决所有 Sourcery 评论: 在 pull request 上评论 @sourcery-ai resolve,即可将所有 Sourcery 评论标记为已解决。如果你已经处理了所有评论且不想再看到它们,这会很有用。
  • 一次性忽略所有 Sourcery 审查: 在 pull request 上评论 @sourcery-ai dismiss,即可忽略所有现有的 Sourcery 审查。特别适用于希望从头开始一轮新的审查时——别忘了再评论一次 @sourcery-ai review 来触发新的审查!

自定义你的体验

访问你的 dashboard 以:

  • 启用或停用诸如 Sourcery 自动生成的 pull request 摘要、审阅者指南等审查功能。
  • 更改审查语言。
  • 添加、删除或编辑自定义审查指令。
  • 调整其他审查设置。

获取帮助

Original review guide in English

Reviewer's Guide

Raises the video upload soft limit from 100MB to 4GB with a two-layer fallback: first attempting Highway upload up to 4GB, and on failure degrading to file uploads for group and private messages, including size introspection helpers and targeted error handling in element building and OneBot message sending paths.

Sequence diagram for private video upload fallback to file

sequenceDiagram
  actor Client
  participant OneBot as sendPrivateMessage
  participant MsgAPI as bridge.apis.message
  participant FileAPI as bridge.apis.file

  Client->>OneBot: sendPrivateMessage(userId, elements)
  OneBot->>OneBot: split elements into allFileElements & nonFileElements
  alt nonFileElements not empty
    OneBot->>MsgAPI: sendPrivate(userId, nonFileElements)
    MsgAPI-->>OneBot: error (video upload failed)
    OneBot->>OneBot: getVideoSourceSize(videoElement)
    alt size > MAX_VIDEO_SIZE
      OneBot->>OneBot: log.warn fallback to file upload
      OneBot->>OneBot: convert video element to file MessageElement
      OneBot->>OneBot: update allFileElements & nonFileElements
      opt nonFileElements still not empty
        OneBot->>MsgAPI: sendPrivate(userId, nonFileElements)
        MsgAPI-->>OneBot: lastReceipt
        OneBot->>OneBot: logSentMessage(false, userId, nonFileElements)
      end
    else no large video element
      OneBot-->>Client: rethrow error
    end
  end
  opt allFileElements not empty
    OneBot->>FileAPI: uploadPrivate / sendC2cFile(allFileElements)
    FileAPI-->>OneBot: file upload receipts
  end
  OneBot-->>Client: final lastReceipt and file results
Loading

File-Level Changes

Change Details Files
Relax video size limits for Highway uploads to 4GB with a soft warning threshold at 100MB and introduce a shared helper to determine video source size.
  • Export the existing 100MB MAX_VIDEO_SIZE constant and introduce a new 4GB MAX_VIDEO_SIZE_HARD limit.
  • Add getVideoSourceSize to derive video size from element metadata or local filesystem paths.
  • Update local-path staging to hard-reject files over 4GB while only warning (not throwing) for files between 100MB and 4GB.
  • Change remote/binary loading to allow files up to 4GB and enforce the hard limit again after staging, converting prior >100MB errors into warnings.
packages/protocol/src/highway/video-upload.ts
Add group-message fallback from video elements to file elements when Highway video upload fails.
  • Wrap makeVideoElem video upload path in try/catch to intercept uploadVideoMsgInfo failures.
  • On failure for group messages with a fileId, log a console warning and delegate to makeGroupFileElem to send as a file element instead.
  • Re-throw the error unchanged for non-group or non-fileId cases to preserve existing error behavior.
packages/protocol/src/element-builder.ts
Add private-message fallback from large video elements to file upload when Highway video upload fails.
  • Change sendPrivateMessage to use mutable file and non-file element arrays so they can be adjusted on error.
  • Wrap the initial non-file sendPrivate call in try/catch to detect upload failures.
  • On error, detect if a video element over the soft size limit triggered the failure using getVideoSourceSize and MAX_VIDEO_SIZE.
  • If such a video is found, log a warning, convert that video into a file MessageElement, append it to the file elements list, resend remaining non-file elements, and then continue with the existing file upload pipeline; otherwise, rethrow the original error.
packages/onebot/src/modules/message-actions.ts

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

本次修改调整了视频上传的大小校验阈值:新增硬上限,超过软上限时改为告警并继续处理,同时新增获取视频源大小的函数。视频元素在群聊上传失败时可回退为文件元素。私聊发送流程在上传失败时会识别超大视频并转为文件元素后继续发送。

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed 标题准确概括了提升视频上限并增加 Highway 优先回退的主要改动。
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed PR 描述包含概述、变更清单、设计决策、测试验证和自检,基本覆盖模板要求,且与实现内容一致。

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - 我发现了 2 个问题,并给出了一些总体反馈:

  • 在 makeVideoElem 中,你直接使用了 console.warn,而该模块/调用栈的其他部分都依赖结构化日志工具(例如 moduleLog/log);建议将这个 warning 通过现有的日志工具输出,以保持一致性和可配置性。
  • getVideoSourceSize 在消息发送路径中(被 sendPrivateMessage 使用)执行了同步的 fs.existsSync/fs.statSync 调用,对于大型或较慢的文件系统,这可能会阻塞事件循环;建议改用异步的 fs.promises 版本,或者把这部分工作从热点路径中移出。
供 AI Agent 使用的提示
Please address the comments from this code review:

## Overall Comments
- In `makeVideoElem` you’re using `console.warn` directly while the rest of the module/stack relies on structured loggers (e.g. `moduleLog`/`log`); consider routing this warning through the existing logging utility for consistency and configurability.
- `getVideoSourceSize` performs synchronous `fs.existsSync`/`fs.statSync` calls in the message send path (used by `sendPrivateMessage`), which can block the event loop for large or slow filesystems; consider switching to the async `fs.promises` variants or deferring this work off the hot path.

## Individual Comments

### Comment 1
<location path="packages/protocol/src/element-builder.ts" line_range="391-393" />
<code_context>
+    try {
+      lastReceipt = await ref.bridge.apis.message.sendPrivate(userId, nonFileElements);
+      logSentMessage(false, userId, nonFileElements);
+    } catch (err) {
+      // Highway upload failed — check if a large video element triggered it
+      // and fall back to file upload for that element (private messages
</code_context>
<issue_to_address>
**suggestion:** Use the existing logging mechanism instead of `console.warn` for consistency and configurability.

In this package we standardize on the module logger rather than `console.*`. Please switch this warning to `moduleLog.warn` (or the equivalent here) so log formatting, levels, routing, and observability remain consistent with the rest of the system.
</issue_to_address>

### Comment 2
<location path="packages/onebot/src/modules/message-actions.ts" line_range="552-555" />
<code_context>
+      // Highway upload failed — check if a large video element triggered it
+      // and fall back to file upload for that element (private messages
+      // cannot carry file elements through the element pipeline).
+      const videoIdx = nonFileElements.findIndex(e => {
+        if (e.type !== 'video') return false;
+        const sz = getVideoSourceSize(e);
+        return sz !== null && sz > MAX_VIDEO_SIZE;
+      });
+      if (videoIdx >= 0) {
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Size-based fallback may miss some large-video failures when `getVideoSourceSize` returns null.

Because the fallback only checks `getVideoSourceSize` against `MAX_VIDEO_SIZE`, any video whose size can’t be inferred (`sz === null`) will never trigger the fallback—even if the upload actually failed due to a size limit. If the bridge exposes a specific error code or message for size-related failures, consider also using that signal so the fallback covers these remote-only cases.

Suggested implementation:

```typescript
      // Highway upload failed — check if a large video element triggered it
      // and fall back to file upload for that element (private messages
      // cannot carry file elements through the element pipeline).
      // Also consider explicit size-limit errors from the bridge so that
      // videos whose size can't be inferred locally can still trigger
      // the fallback.
      const isSizeLimitError =
        err instanceof Error &&
        /size limit|too large|413/i.test(err.message);

      const videoIdx = nonFileElements.findIndex(e => {
        if (e.type !== 'video') return false;
        const sz = getVideoSourceSize(e);
        if (sz === null) {
          // If we know the remote failed with a size-limit error, treat
          // size-unknown videos as candidates for fallback as well.
          return isSizeLimitError;
        }
        return sz > MAX_VIDEO_SIZE;
      });

```

If the bridge exposes a more specific way to detect size-related failures (for example, a structured error type with a `code` like `HIGHWAY_SIZE_LIMIT` or a boolean flag), replace the `isSizeLimitError` heuristic with that canonical check, e.g.:

```ts
const isSizeLimitError =
  typeof err === 'object' &&
  err !== null &&
  'code' in err &&
  (err as any).code === 'HIGHWAY_SIZE_LIMIT';
```

This should be aligned with however other parts of the codebase detect Highway size-limit errors.
</issue_to_address>

Sourcery 对开源项目是免费的 —— 如果你觉得我们的评审有帮助,欢迎分享 ✨
帮我变得更有用!请对每条评论点 👍 或 👎,我会根据你的反馈改进后续的评审。
Original comment in English

Hey - I've found 2 issues, and left some high level feedback:

  • In makeVideoElem you’re using console.warn directly while the rest of the module/stack relies on structured loggers (e.g. moduleLog/log); consider routing this warning through the existing logging utility for consistency and configurability.
  • getVideoSourceSize performs synchronous fs.existsSync/fs.statSync calls in the message send path (used by sendPrivateMessage), which can block the event loop for large or slow filesystems; consider switching to the async fs.promises variants or deferring this work off the hot path.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `makeVideoElem` you’re using `console.warn` directly while the rest of the module/stack relies on structured loggers (e.g. `moduleLog`/`log`); consider routing this warning through the existing logging utility for consistency and configurability.
- `getVideoSourceSize` performs synchronous `fs.existsSync`/`fs.statSync` calls in the message send path (used by `sendPrivateMessage`), which can block the event loop for large or slow filesystems; consider switching to the async `fs.promises` variants or deferring this work off the hot path.

## Individual Comments

### Comment 1
<location path="packages/protocol/src/element-builder.ts" line_range="391-393" />
<code_context>
+    try {
+      lastReceipt = await ref.bridge.apis.message.sendPrivate(userId, nonFileElements);
+      logSentMessage(false, userId, nonFileElements);
+    } catch (err) {
+      // Highway upload failed — check if a large video element triggered it
+      // and fall back to file upload for that element (private messages
</code_context>
<issue_to_address>
**suggestion:** Use the existing logging mechanism instead of `console.warn` for consistency and configurability.

In this package we standardize on the module logger rather than `console.*`. Please switch this warning to `moduleLog.warn` (or the equivalent here) so log formatting, levels, routing, and observability remain consistent with the rest of the system.
</issue_to_address>

### Comment 2
<location path="packages/onebot/src/modules/message-actions.ts" line_range="552-555" />
<code_context>
+      // Highway upload failed — check if a large video element triggered it
+      // and fall back to file upload for that element (private messages
+      // cannot carry file elements through the element pipeline).
+      const videoIdx = nonFileElements.findIndex(e => {
+        if (e.type !== 'video') return false;
+        const sz = getVideoSourceSize(e);
+        return sz !== null && sz > MAX_VIDEO_SIZE;
+      });
+      if (videoIdx >= 0) {
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Size-based fallback may miss some large-video failures when `getVideoSourceSize` returns null.

Because the fallback only checks `getVideoSourceSize` against `MAX_VIDEO_SIZE`, any video whose size can’t be inferred (`sz === null`) will never trigger the fallback—even if the upload actually failed due to a size limit. If the bridge exposes a specific error code or message for size-related failures, consider also using that signal so the fallback covers these remote-only cases.

Suggested implementation:

```typescript
      // Highway upload failed — check if a large video element triggered it
      // and fall back to file upload for that element (private messages
      // cannot carry file elements through the element pipeline).
      // Also consider explicit size-limit errors from the bridge so that
      // videos whose size can't be inferred locally can still trigger
      // the fallback.
      const isSizeLimitError =
        err instanceof Error &&
        /size limit|too large|413/i.test(err.message);

      const videoIdx = nonFileElements.findIndex(e => {
        if (e.type !== 'video') return false;
        const sz = getVideoSourceSize(e);
        if (sz === null) {
          // If we know the remote failed with a size-limit error, treat
          // size-unknown videos as candidates for fallback as well.
          return isSizeLimitError;
        }
        return sz > MAX_VIDEO_SIZE;
      });

```

If the bridge exposes a more specific way to detect size-related failures (for example, a structured error type with a `code` like `HIGHWAY_SIZE_LIMIT` or a boolean flag), replace the `isSizeLimitError` heuristic with that canonical check, e.g.:

```ts
const isSizeLimitError =
  typeof err === 'object' &&
  err !== null &&
  'code' in err &&
  (err as any).code === 'HIGHWAY_SIZE_LIMIT';
```

This should be aligned with however other parts of the codebase detect Highway size-limit errors.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +391 to +393
} catch (err) {
if (isGroup && element.fileId) {
console.warn('[ElemBuilder] video upload failed, falling back to file element: %s', err instanceof Error ? err.message : String(err));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: 为了保持一致性和可配置性,请使用现有的日志机制,而不是 console.warn。

在这个包里我们标准化使用模块级日志器,而不是 console.*。请将此处的 warning 切换为 moduleLog.warn(或此处的等价实现),这样日志的格式、级别、路由和可观测性就能与系统中其他部分保持一致。

Original comment in English

suggestion: Use the existing logging mechanism instead of console.warn for consistency and configurability.

In this package we standardize on the module logger rather than console.*. Please switch this warning to moduleLog.warn (or the equivalent here) so log formatting, levels, routing, and observability remain consistent with the rest of the system.

Comment thread packages/onebot/src/modules/message-actions.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/protocol/src/highway/video-upload.ts (1)

291-303: 🩺 Stability & Availability | 🟠 Major

把视频读取改成流式,或把硬上限降到 2GiB 以下

fs.readFileSync(local) 仍会把整段视频一次性装入内存;后续 computeHashes(staged.bytes) 和 computeVideoSha1Blocks(staged.bytes) 也都是基于这份完整字节数组处理,SHA1_STREAM_BLOCK_SIZE 只是分块计算,不会避免 OOM。fs.readFileSync 本身也有约 2GiB 的单次读取上限,所以 MAX_VIDEO_SIZE_HARD = 4GiB 实际不可达。建议改成流式读取/流式 SHA1,或者把硬上限调到明显低于 2GiB 的安全值。

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/protocol/src/highway/video-upload.ts` around lines 291 - 303, The
upload staging path in video-upload.ts still loads the entire file into memory
via the local file handling in the upload helper, so fix this by either
switching the staging/hash flow to stream-based processing or lowering the hard
size limit to a safe value below the single-read ceiling. Update the logic
around the function that returns staged video bytes, and ensure downstream
consumers like computeHashes and computeVideoSha1Blocks do not require a full
Uint8Array if you choose the streaming approach.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/onebot/src/modules/message-actions.ts`:
- Around line 557-568: The catch in message sending is too broad and treats any
`sendPrivate` failure as a large-video highway upload failure. Update the
fallback logic in `message-actions.ts` around the `sendPrivate` call and the
`videoIdx`/`findIndex` handling so it only retries as file upload when the error
clearly indicates highway/too-large upload failure, and otherwise rethrow the
original error. If multiple oversized local videos can appear, adjust the
`nonFileElements` processing to handle all matching video elements rather than
only the first one.
- Around line 552-556: The video fallback check in message-actions.ts only
triggers when getVideoSourceSize returns a known size above MAX_VIDEO_SIZE, so
remote http(s) videos with null size are skipped after sendPrivate fails. Update
the video handling around nonFileElements.findIndex (and the related sendPrivate
fallback path) to also treat “video present but size unknown” as a fallback
case, or ensure the video parser populates fileSize before this check so remote
large videos can fall back correctly.

---

Outside diff comments:
In `@packages/protocol/src/highway/video-upload.ts`:
- Around line 291-303: The upload staging path in video-upload.ts still loads
the entire file into memory via the local file handling in the upload helper, so
fix this by either switching the staging/hash flow to stream-based processing or
lowering the hard size limit to a safe value below the single-read ceiling.
Update the logic around the function that returns staged video bytes, and ensure
downstream consumers like computeHashes and computeVideoSha1Blocks do not
require a full Uint8Array if you choose the streaming approach.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 56df6d80-67db-46d7-a25f-4207ea8caa19

📥 Commits

Reviewing files that changed from the base of the PR and between 70fd40e and d0cc6bd.

📒 Files selected for processing (3)
  • packages/onebot/src/modules/message-actions.ts
  • packages/protocol/src/element-builder.ts
  • packages/protocol/src/highway/video-upload.ts

Comment thread packages/onebot/src/modules/message-actions.ts Outdated
Comment thread packages/onebot/src/modules/message-actions.ts Outdated
@Stlara-F
Stlara-F force-pushed the feat/large-video-upload-fallback branch from d0cc6bd to e44ec93 Compare June 26, 2026 17:39

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/protocol/src/highway/video-upload.ts`:
- Around line 35-36: The hard-limit constant, the error messages, and the PR
target are inconsistent in the video upload flow. In video-upload.ts, reconcile
MAX_VIDEO_SIZE_HARD with the intended limit (1.5GB or 4GB), then update the
validation/error text in the upload checks and the related helpers so they all
derive from the same constant instead of hardcoding 4096 MB. Use
MAX_VIDEO_SIZE_HARD as the single source of truth in the functions that emit the
size-limit errors to keep the message and behavior aligned.
- Line 305: The video upload path still does full in-memory reads and hashing,
causing peak memory to scale with the size limit. Update the flow around
stageVideoSource, loadBinarySource, computeHashes, and computeVideoSha1Blocks to
use streaming or chunked processing instead of fs.readFileSync/full Uint8Array
accumulation, and avoid extra copies when writing the temp file and hashing. If
full buffering must remain, re-evaluate MAX_VIDEO_SIZE_HARD against the
deployment memory budget.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a14466ef-32e6-438a-ba64-6039a6b64331

📥 Commits

Reviewing files that changed from the base of the PR and between d0cc6bd and e44ec93.

📒 Files selected for processing (3)
  • packages/onebot/src/modules/message-actions.ts
  • packages/protocol/src/element-builder.ts
  • packages/protocol/src/highway/video-upload.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • packages/protocol/src/element-builder.ts
  • packages/onebot/src/modules/message-actions.ts

Comment thread packages/protocol/src/highway/video-upload.ts
}

const loaded = await loadBinarySource(source, 'video', MAX_VIDEO_SIZE);
const loaded = await loadBinarySource(source, 'video', MAX_VIDEO_SIZE_HARD);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Node.js buffer.constants.MAX_LENGTH value on 64-bit, and maximum size of Buffer / Uint8Array

💡 Result:

In Node.js, the value of buffer.constants.MAX_LENGTH has evolved alongside updates to the V8 engine [1]. On 64-bit systems, as of Node.js v22.0.0 and later (which includes V8 12.4 and newer), buffer.constants.MAX_LENGTH is 8 PiB (2^53 - 1 bytes, or exactly 9,007,199,254,740,991 bytes) [1]. Prior to Node.js v22.0.0, the limit was 4 GiB (2^32 bytes) [1]. Buffer instances in Node.js are subclasses of Uint8Array [2][3]. Consequently, the maximum size allowed for a Buffer is governed by this same constant [4][2]. Attempting to allocate a Buffer or Uint8Array larger than buffer.constants.MAX_LENGTH will result in an ERR_OUT_OF_RANGE error [4][2]. Note that while the theoretical V8 limit may be very large, practical limits on memory allocation and system architecture may still affect the actual maximum size you can successfully allocate in a real-world environment.

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate the relevant file and surrounding helpers.
git ls-files packages/protocol/src/highway/video-upload.ts
echo '--- outline ---'
ast-grep outline packages/protocol/src/highway/video-upload.ts --view expanded
echo '--- line count ---'
wc -l packages/protocol/src/highway/video-upload.ts
echo '--- relevant slices ---'
sed -n '1,260p' packages/protocol/src/highway/video-upload.ts | cat -n
echo '--- second slice ---'
sed -n '260,420p' packages/protocol/src/highway/video-upload.ts | cat -n

Repository: Stlara-F/SnowLuma

Length of output: 18529


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect helper implementations referenced by the review comment.
rg -n "loadBinarySource|stageVideoSource|computeHashes|computeVideoSha1Blocks|MAX_VIDEO_SIZE_HARD" packages/protocol/src -S

Repository: Stlara-F/SnowLuma

Length of output: 2824


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Check Node limit semantics with a small read-only probe if node is available.
node - <<'JS'
const { constants } = require('buffer');
console.log(JSON.stringify({
  node: process.version,
  MAX_LENGTH: constants.MAX_LENGTH,
  MAX_STRING_LENGTH: constants.MAX_STRING_LENGTH,
}));
JS

Repository: Stlara-F/SnowLuma

Length of output: 235


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Read only the most relevant regions once outputs are available.
sed -n '1,220p' packages/protocol/src/highway/video-upload.ts | cat -n
sed -n '220,380p' packages/protocol/src/highway/video-upload.ts | cat -n

Repository: Stlara-F/SnowLuma

Length of output: 14213


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the package's Node engine and the relevant implementation paths.
git ls-files package.json packages/protocol/package.json
echo '--- package.json / engine hints ---'
for f in package.json packages/protocol/package.json; do
  if [ -f "$f" ]; then
    echo "### $f"
    rg -n '"engines"|"node"|MAX_VIDEO_SIZE_HARD|loadBinarySource|stageVideoSource|computeHashes|computeVideoSha1Blocks' "$f" -n -S || true
  fi
done

Repository: Stlara-F/SnowLuma

Length of output: 327


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect loadBinarySource / computeHashes to confirm memory behavior and copies.
sed -n '1,280p' packages/protocol/src/highway/utils.ts | cat -n

Repository: Stlara-F/SnowLuma

Length of output: 11951


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Probe Node's Buffer/Uint8Array limits against the repo's engine floor.
node - <<'JS'
const { constants } = require('buffer');
console.log(JSON.stringify({
  node: process.version,
  maxLength: constants.MAX_LENGTH,
  maxTypedArrayLength: constants.MAX_LENGTH, // Buffer and Uint8Array share the same practical bound here
}));
JS

Repository: Stlara-F/SnowLuma

Length of output: 243


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Confirm the exact source lines that materialize bytes in memory.
rg -n "readFileSync|Buffer\.from\(|new Uint8Array\(|loadBinarySource\(" packages/protocol/src/highway -S

Repository: Stlara-F/SnowLuma

Length of output: 5203


这里仍然是全量读入 + 全量 hash,峰值内存会随上限一起抬高。
本地路径在 stageVideoSource 里仍是 fs.readFileSync,远端路径也会在 loadBinarySource 里累积成完整 Uint8Array;后面还要再写临时文件并跑 computeHashes / computeVideoSha1Blocks。当前 Node 22+ 的 Buffer / Uint8Array 长度上限已远高于 4GB,主要风险是 OOM 和额外拷贝开销;建议改成流式 hash / 分块处理,或把上限与部署内存配额重新对齐。

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/protocol/src/highway/video-upload.ts` at line 305, The video upload
path still does full in-memory reads and hashing, causing peak memory to scale
with the size limit. Update the flow around stageVideoSource, loadBinarySource,
computeHashes, and computeVideoSha1Blocks to use streaming or chunked processing
instead of fs.readFileSync/full Uint8Array accumulation, and avoid extra copies
when writing the temp file and hashing. If full buffering must remain,
re-evaluate MAX_VIDEO_SIZE_HARD against the deployment memory budget.

…back

Round 3: fix hardcoded 4096 MB in error messages
- Use MAX_VIDEO_SIZE_HARD constant instead of hardcoded 4096 MB
- Both throw sites now derive limit from the same constant
@Stlara-F
Stlara-F force-pushed the feat/large-video-upload-fallback branch from e44ec93 to 2f203a6 Compare June 26, 2026 17:46
@Stlara-F Stlara-F changed the title feat(protocol,onebot): raise video size limit to 4GB with Highway-first fallback feat(protocol,onebot): raise video size limit to 1.5GB with Highway-first fallback Jun 26, 2026
@Stlara-F
Stlara-F merged commit 91e479d into dev Jun 26, 2026
8 checks passed
@Stlara-F
Stlara-F deleted the feat/large-video-upload-fallback branch June 30, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant