Skip to content

Fix Float CCT Bugs on L3 #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Victor-Jung merged 3 commits into from
Feb 27, 2025
Merged

Fix Float CCT Bugs on L3 #37

Victor-Jung merged 3 commits into from
Feb 27, 2025

Conversation

@runwangdl
Copy link
Contributor

@runwangdl runwangdl commented Feb 17, 2025 •
edited by Victor-Jung
Loading

Added

  • Added multiple CCT settings for testing.
  • Added CCT L3 test to CI to ensure correctness for img size of 16 and 32.
  • Added NaN check for deeploytest diff to improve result validation.
  • SINGELE_CORE macro header for PULP

Changed

  • Regenerated CCT ONNX files without "output" & "input" in their names to avoid triggering the dumphex parser bug.
  • Regenerated CCT ONNX file with 3 branches for attention, transforming the attention computation graph into three branches.
  • Changed code generation for Hex output to properly handle float values.
  • (Some changes about profileuntiled should be included in the last PR but are not for no reason, so they are changed again in this PR)

Fixed

  • Updated printinput nodetemplate for float handling.
  • A few kernels have both the SINGLE_CORE macro and multicore support, which leads to error.
  • Mempool float weight with int value casted to float during code generation. This change saved mempool CI a lot of time.

Limitation

  • Since gather tiling is not successful and not used in any kernel now, gather constraint is reverted to last PR version.
  • Larger CCT would require more changes, including DMA size check, maxpool L3 bug. Those would be fixed in the next two PRs.

PR Merge Checklist

  1. The PR is rebased on the latest devel commit and pointing to devel.
  2. Your PR reviewed and approved.
  3. All checks are passing.
  4. The CHANGELOG.md file has been updated.

@runwangdl runwangdl changed the title Fix Float CCT Bugs on L3 [Draft] Fix Float CCT Bugs on L3 Feb 17, 2025
@runwangdl runwangdl force-pushed the PULPCCTL3 branch 11 times, most recently from 380f5d2 to d0c8f0c Compare February 23, 2025 21:12
@runwangdl runwangdl changed the title [Draft] Fix Float CCT Bugs on L3 Fix Float CCT Bugs on L3 Feb 23, 2025
@runwangdl runwangdl force-pushed the PULPCCTL3 branch 2 times, most recently from ea0c646 to 3df3245 Compare February 23, 2025 21:39
@runwangdl runwangdl marked this pull request as ready for review February 23, 2025 21:39
Victor-Jung
Victor-Jung reviewed Feb 24, 2025
View reviewed changes
Copy link
Member

@Victor-Jung Victor-Jung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Run, very good work! One comment on the single-core execution, please propagate this change to every PULPOpen template that does not support multi-core execution.

@runwangdl runwangdl requested a review from Xeratec as a code owner February 26, 2025 15:25
runwangdl and others added 2 commits February 26, 2025 15:27
@runwangdl
Add Tiling Support to All CCT Kernels and Fix CCT Operators on Siracu…
7b9cadf 
…sa Platform for L2 (pulp-platform#35)

* Update CCT onnx without broadcast and Upload CCT two version(16,32)
* Update CCT on PULP with Tiling

---------

Co-authored-by: Victor Jung <33875047+Victor-Jung@users.noreply.github.com>

Update CCT on PULP with Tiling

Add PULPProfileUntiled Pass
@runwangdl
Fixed CCT on L3 Bugs
be7c12d
@runwangdl runwangdl force-pushed the PULPCCTL3 branch 3 times, most recently from 4ac65b5 to 98ce2cf Compare February 26, 2025 15:43
@runwangdl runwangdl force-pushed the PULPCCTL3 branch 5 times, most recently from dbd3666 to 33c5d62 Compare February 26, 2025 22:26
@runwangdl runwangdl force-pushed the PULPCCTL3 branch 2 times, most recently from 9a22998 to d5b1288 Compare February 26, 2025 23:34
@runwangdl
Copy link
Contributor Author

runwangdl commented Feb 27, 2025 •
edited
Loading

Regarding the SINGLE_CORE macro on PULP, a new file named pulp_core.h has been created. Meanwhile, some PULP kernels had a bug where SINGLE_CORE and multicore support coexisted, leading to errors. To resolve this, the SINGLE_CORE macros have been removed from these kernels. This issue was observed in ihardwish, iRMSNorm, and Transpose.

@runwangdl
Copy link
Contributor Author

runwangdl commented Feb 27, 2025

All CI tests have successfully passed this time.

@Victor-Jung
Copy link
Member

Victor-Jung commented Feb 27, 2025

Regarding the SINGLE_CORE macro on PULP, a new file named pulp_core.h has been created. Meanwhile, some PULP kernels had a bug where SINGLE_CORE and multicore support coexisted, leading to errors. To resolve this, the SINGLE_CORE macros have been removed from these kernels. This issue was observed in ihardwish, iRMSNorm, and Transpose.

All clear; I see how you can easily reach a deadlock or wrong behavior for some kernels.

@Victor-Jung Victor-Jung merged commit fb6f0d1 into pulp-platform:devel Feb 27, 2025
110 checks passed
@runwangdl runwangdl deleted the PULPCCTL3 branch April 11, 2025 15:54
FrancescoConti pushed a commit to FrancescoConti/Deeploy that referenced this pull request May 9, 2025
@runwangdl @Victor-Jung @FrancescoConti
Fix Float CCT Bugs on L3 (pulp-platform#37)
8db1c59 
* Add Tiling Support to All CCT Kernels and Fix CCT Operators on Siracusa Platform for L2 (pulp-platform#35)

* Update CCT onnx without broadcast and Upload CCT two version(16,32)

* Fixed CCT on L3 Bugs

---------

Co-authored-by: Victor Jung <33875047+Victor-Jung@users.noreply.github.com>
@Xeratec Xeratec mentioned this pull request Jul 8, 2025
Merged
Xeratec added a commit that referenced this pull request Jul 8, 2025
@Xeratec
Release v0.2.0 (#103)
c7fd2a1 
This release contains major architectural changes, new platform support,
enhanced simulation workflows, floating-point kernel support, training
infrastructure for CCT models, memory allocation strategies, and
documentation improvements.

After merging this into `main`, the release process will proceed with:
- Pushing a Git tag for the release after merging this PR
- Creating a GitHub release with the prepared tag.

Note: Since the release tag references the Docker container tagged with
the release tag (`ghcr.io/pulp-platform/deeploy:v0.2.0`), the CI will
initially fail. The Deeploy Docker image must be built after the release
PR is merged and the CI restarted.

### List of Pull Requests
- Prepare v0.2.0 release
[#102](#102)
- Add Luka as Code Owner
[#101](#101)
- Fix CI, Docker Files, and Documentation Workflow
[#100](#100)
- Chimera Platform Integration
[#96](#96)
- Add Tutorial and Refactor README
[#97](#97)
- Reduce Mean Float Template
[#92](#92)
- Reshape Memory Freeing and Generic Float GEMM Fixes
[#91](#91)
- Prepare for Release and Separate Dependencies
[#90](#90)
- Fix input offsets calculation
[#89](#89)
- Move PULP SDK to main branch/fork
[#88](#88)
- Finite Lifetime for IO Tensors
[#51](#51)
- Improved Memory Visualization and Multi-Layer Tiling Profiling
[#56](#56)
- Fix Linting in CI and Reformat C Files
[#86](#86)
- Fix Broken CMake Flow For pulp-sdk
[#87](#87)
- Refactor Changelog For Release
[#85](#85)
- ARM Docker Container and Minor Bug Fix
[#84](#84)
- Added Kernel for Generic Float DW Conv2D
[#63](#63)
- Autoselect Self-Hosted Runners if the Action is on Upstream
[#81](#81)
- TEST_RECENT linking on MacOS
[#78](#78)
- Add RV32IMF Picolibc support for Siracusa platform
[#66](#66)
- Improve Documentation and VSCode Support
[#76](#76)
- Debug Print Topology Pass and Code Transformation
[#75](#75)
- Find all subdirectories of Deeploy when installing with pip install
[#70](#70)
- Add milestone issue template
[#71](#71)
- Bunch of fixes and changes
[#58](#58)
- Add SoftHier platform
[#65](#65)
- rv32imf_xpulpv2 ISA support for Siracusa platform
[#64](#64)
- One LLVM To Compile Them All
[#60](#60)
- One GVSoC to Simulate Them All
[#59](#59)
- Add Support for CCT Last Layer Training with Embedding Dim 8-128
[#55](#55)
- Add CCT Classifier Training Support
[#53](#53)
- L3 Bugs: DMA Struct Datatype and Maxpool Margin Error
[#45](#45)
- DeepQuant Quantized Linear Support
[#54](#54)
- Implemented Dequant Layer for Generic and Siracusa
[#52](#52)
- Infinite Lifetime Buffers Considered in Tiling & Memory Allocation (+
Visualization) [#44](#44)
- Implemented Quant Layer for Generic and Siracusa
[#49](#49)
- Increase maximal Mchan DMA transfer sizes from 64KiB to 128KiB
[#47](#47)
- Add MiniMalloc and Decouple Memory Allocation and Tiling
[#40](#40)
- Float CCT Bugs on L3
[#37](#37)
- Memory Allocation Strategies and Visualization
[#36](#36)
- Add CODEOWNERS [#42](#42)
- Add Tiling Support to All CCT Kernels and Fix CCT Operators on
Siracusa Platform for L2
[#35](#35)
- Add Fp gemm and Softmax for Snitch platform
[#31](#31)
- Add Float Kernels for CCT
[#29](#29)
- documentation deployment
[#34](#34)
- main.c Float Cast Bugs
[#28](#28)
- Add Float GEMM on PULP with Tiling
[#26](#26)
- Add Float Support & Float GEMM for Generic
[#25](#25)
- GVSOC support for the Snitch Cluster platform
[#23](#23)
- Snitch Cluster Tiling Support
[#22](#22)
- Snitch support integration
[#14](#14)
- Update bibtex citation
[#20](#20)
- the PR template location, bump min python to 3.10, change install
command [#17](#17)
- Add pre-commit for python formatting
[#15](#15)
- FP integration (v2)
[#12](#12)
- shell for sequential tests of Generic, Cortex, and Mempool platforms
[#11](#11)
- Add issue templates
[#10](#10)
- Minor CI and Readme Improvements
[#8](#8)
- Fix GHCR Link for Docker Build
[#7](#7)
- neureka's ccache id
[#6](#6)
- GitHub-based CI/CD Flow
[#4](#4)
- Generic Softmax Kernel
[#2](#2)
- Port GitLab CI [#1](#1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Victor-Jung Victor-Jung Victor-Jung approved these changes

@Xeratec Xeratec Awaiting requested review from Xeratec Xeratec is a code owner

Assignees

@runwangdl runwangdl

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@runwangdl @Victor-Jung