Skip to content

Comments

Add STA to V1#312

Merged
SolitaryThinker merged 16 commits intomainfrom
add_sta
Apr 15, 2025
Merged

Add STA to V1#312
SolitaryThinker merged 16 commits intomainfrom
add_sta

Conversation

@jzhang38
Copy link
Collaborator

@jzhang38 jzhang38 commented Apr 4, 2025

  • add fa3 backend
  • remove all attn dropout args (no one use dropout in 2025, fa3 do not even support dropout)
  • add supported_attn_backend attribute for each model(so text encoder will avoid using STA even if we set the attn backend to STA & WAN will not call STA if we did not figure out a good mask strategy)
  • add prefix for dit module, so each attn knows which layer is it located at.
  • add sta
  • make sure changes does not break other functions (such as Wan)
  • make sure there is no lining error.

@jzhang38 jzhang38 marked this pull request as draft April 4, 2025 02:18
@SolitaryThinker SolitaryThinker marked this pull request as ready for review April 9, 2025 17:31
@SolitaryThinker SolitaryThinker force-pushed the add_sta branch 2 times, most recently from df76d9a to 9d56ca4 Compare April 15, 2025 03:24
@SolitaryThinker
Copy link
Collaborator

pretty sure hf is just down....

@SolitaryThinker SolitaryThinker merged commit 2325adf into main Apr 15, 2025
8 of 14 checks passed
kevin314 pushed a commit to kevin314/FastVideo that referenced this pull request Apr 15, 2025
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
@SolitaryThinker SolitaryThinker deleted the add_sta branch June 30, 2025 22:40
qimcis pushed a commit to qimcis/FastVideo that referenced this pull request Oct 30, 2025
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants