Skip to content

vulkan int8 kernels#6751

Open
nihui wants to merge 48 commits into
Tencent:masterfrom
nihui:vulkan-int-2
Open

vulkan int8 kernels#6751
nihui wants to merge 48 commits into
Tencent:masterfrom
nihui:vulkan-int-2

Conversation

@nihui

@nihui nihui commented May 28, 2026

Copy link
Copy Markdown
Member

No description provided.

Implement Vulkan int8 paths for convolution, convolutiondepthwise, innerproduct, and gemm with scalar fallback and integer dot product shader branches. Add goal.md and enable existing int8 unit tests for Vulkan coverage.
@tencent-adm

Copy link
Copy Markdown
Member

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@nihui nihui marked this pull request as draft May 28, 2026 02:35
@codecov-commenter

codecov-commenter commented May 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.27871% with 489 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.78%. Comparing base (b16501a) to head (cd08386).

Files with missing lines Patch % Lines
src/layer/vulkan/convolution_vulkan.cpp 77.21% 206 Missing ⚠️
src/layer/vulkan/gemm_vulkan.cpp 62.01% 158 Missing ⚠️
src/layer/vulkan/padding_vulkan.cpp 84.75% 50 Missing ⚠️
src/gpu.cpp 70.58% 25 Missing ⚠️
src/layer/vulkan/convolutiondepthwise_vulkan.cpp 96.44% 16 Missing ⚠️
src/layer/vulkan/innerproduct_vulkan.cpp 97.11% 9 Missing ⚠️
src/layer/convolution.cpp 40.00% 6 Missing ⚠️
src/layer/x86/convolution_x86.cpp 50.00% 5 Missing ⚠️
src/layer/arm/convolution_arm.cpp 50.00% 3 Missing ⚠️
src/layer/loongarch/convolution_loongarch.cpp 62.50% 3 Missing ⚠️
... and 3 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6751      +/-   ##
==========================================
- Coverage   95.78%   94.78%   -1.00%     
==========================================
  Files         946      946              
  Lines      410528   412347    +1819     
==========================================
- Hits       393237   390861    -2376     
- Misses      17291    21486    +4195     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions github-actions Bot added the doc label May 28, 2026
@nihui nihui mentioned this pull request May 29, 2026
6 tasks
@nihui nihui closed this Jun 1, 2026
@nihui nihui reopened this Jun 1, 2026
@nihui nihui marked this pull request as ready for review June 4, 2026 03:55
@nihui nihui changed the title [WIP] vulkan int8 kernels vulkan int8 kernels Jun 4, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 46154093ca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/layer/vulkan/shader/padding_int8.comp

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd9a06adcb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/net.cpp

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ee43f88d03

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

VkMat B = B0;

// Runtime int8 blobs do not carry scale metadata, so reject before recording int8 pipelines.
if (!constantA && A.elembits() == 8)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fall back before rejecting dynamic int8 Gemm inputs

When an int8 Gemm has a non-constant A input that is already an 8-bit VkMat (for example fed by a previous requantized int8 layer), this new Vulkan path is selected because the old support_vulkan = false guard was removed, but the first thing it does is return -1. There is no CPU fallback at this point in execution, so extraction fails for a graph shape that the loader now advertises as Vulkan-capable; either keep this case off Vulkan during loading or handle the supplied int8 scale path here.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants