tag:github.com,2008:https://github.com/hpcaitech/ColossalAI/releases Release notes from ColossalAI 2025-06-02T09:08:24Z tag:github.com,2008:Repository/422274596/v0.5.1 2025-06-02T09:08:24Z v0.5.1: Merge pull request #6334 from flybird11111/main <p>[release] release version</p> BurkeHulk tag:github.com,2008:Repository/422274596/v0.5.0 2025-06-04T06:00:47Z Version v0.5.0 Release Today! <h2>What's Changed</h2> <ul> <li>[HotFix] update load lora model Readme; by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/duanjunwen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/duanjunwen">@duanjunwen</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2899941804" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6240" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6240/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6240">#6240</a></li> <li>Update README.md by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Yanjia0/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Yanjia0">@Yanjia0</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3001352017" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6268" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6268/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6268">#6268</a></li> <li>[ci] update ci by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/flybird11111/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/flybird11111">@flybird11111</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2952358081" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6254" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6254/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6254">#6254</a></li> <li>[upgrade]Upgrade transformers by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/flybird11111/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/flybird11111">@flybird11111</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3079367037" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6320" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6320/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6320">#6320</a></li> <li>[release] release version by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/flybird11111/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/flybird11111">@flybird11111</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3092874250" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6330" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6330/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6330">#6330</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.9...v0.5.0"><tt>v0.4.9...v0.5.0</tt></a></p> github-actions[bot] tag:github.com,2008:Repository/422274596/v0.4.9 2025-03-04T01:51:48Z Version v0.4.9 Release Today! <h2>What's Changed</h2> <h3>Release</h3> <ul> <li>[release] update version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2889960650" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6236" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6236/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6236">#6236</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Hotfix</h3> <ul> <li>[hotfix] fix lora load (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2886145994" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6231" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6231/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6231">#6231</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Misc</h3> <ul> <li>[misc] update torch version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2865118473" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6206" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6206/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6206">#6206</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Chat</h3> <ul> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2865583462" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6208" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6208/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6208">#6208</a> from hpcaitech/grpo_dev by <a href="https://api.github.com/users/YeAnbang">YeAnbang</a></li> </ul> <h3>Pre-commit.ci</h3> <ul> <li>[pre-commit.ci] auto fixes from pre-commit.com hooks by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.9...v0.4.8"><tt>v0.4.9...v0.4.8</tt></a></p> github-actions[bot] tag:github.com,2008:Repository/422274596/v0.4.8 2025-02-20T03:37:37Z Version v0.4.8 Release Today! <h2>What's Changed</h2> <h3>Release</h3> <ul> <li>[release] update version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2859513385" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6195" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6195/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6195">#6195</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Doc</h3> <ul> <li>[doc] DeepSeek V3/R1 news (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2862377410" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6199" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6199/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6199">#6199</a>) by <a href="https://api.github.com/users/binmakeswell">binmakeswell</a></li> </ul> <h3>Application</h3> <ul> <li>[application] add lora sft example data (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2859909375" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6198" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6198/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6198">#6198</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[application] Update README (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2859516371" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6196" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6196/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6196">#6196</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> <li>[application] add lora sft example (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2853126872" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6192" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6192/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6192">#6192</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Pre-commit.ci</h3> <ul> <li>Add GRPO and Support RLVR for PPO (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2842147351" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6186" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6186/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6186">#6186</a>) by <a href="https://api.github.com/users/YeAnbang">YeAnbang</a></li> </ul> <h3>Checkpointio</h3> <ul> <li>[checkpointio] fix for async io (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2850064175" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6189" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6189/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6189">#6189</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[checkpointio] fix checkpoint for 3d (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2844396622" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6187" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6187/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6187">#6187</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[checkpointio] gather tensor before unpad it if the tensor is both padded and distributed (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2757516206" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6168" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6168/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6168">#6168</a>) by <a href="https://api.github.com/users/Lemon-412">Lemon Qin</a></li> <li>[checkpointio] support load-pin overlap (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2767016256" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6177" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6177/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6177">#6177</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Hotfix</h3> <ul> <li>[hotfix] fix zero optim save (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2852741961" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6191" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6191/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6191">#6191</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[hotfix] fix hybrid checkpointio for sp+dp (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2825851415" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6184" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6184/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6184">#6184</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> </ul> <h3>Shardformer</h3> <ul> <li>[shardformer] support pipeline for deepseek v3 and optimize lora save (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2847084730" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6188" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6188/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6188">#6188</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[shardformer] support ep for deepseek v3 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2834919982" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6185" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6185/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6185">#6185</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Ci</h3> <ul> <li>[CI] Cleanup Dist Optim tests with shared helper funcs (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2649994038" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6125" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6125/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6125">#6125</a>) by <a href="https://api.github.com/users/Edenzzzz">Wenxuan Tan</a></li> </ul> <h3>Issue template</h3> <ul> <li>[Issue template] Add checkbox asking for details to reproduce error (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2616875056" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6104" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6104/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6104">#6104</a>) by <a href="https://api.github.com/users/Edenzzzz">Wenxuan Tan</a></li> </ul> <h3>Inference</h3> <ul> <li>[Inference]Fix example in readme (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2769161834" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6178" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6178/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6178">#6178</a>) by <a href="https://api.github.com/users/GuangyaoZhang">Guangyao Zhang</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.8...v0.4.7"><tt>v0.4.8...v0.4.7</tt></a></p> github-actions[bot] tag:github.com,2008:Repository/422274596/v0.4.7 2025-01-03T03:53:16Z Version v0.4.7 Release Today! <h2>What's Changed</h2> <h3>Release</h3> <ul> <li>[release] update version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2765370824" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6174" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6174/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6174">#6174</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Pre-commit.ci</h3> <ul> <li>[pre-commit.ci] pre-commit autoupdate (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2633478869" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6113" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6113/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6113">#6113</a>) by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> </ul> <h3>Sharderformer</h3> <ul> <li>[Sharderformer] Support zbv in Sharderformer Policy (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2678774598" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6150" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6150/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6150">#6150</a>) by <a href="https://api.github.com/users/duanjunwen">duanjunwen</a></li> </ul> <h3>Checkpointio</h3> <ul> <li>[checkpointio] support non blocking pin load (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2758591875" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6172" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6172/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6172">#6172</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[checkpointio]support asyncio for 3d (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2693478955" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6152" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6152/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6152">#6152</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[checkpointio] fix async io (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2722453050" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6155" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6155/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6155">#6155</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[checkpointio] support debug log (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2694255365" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6153" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6153/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6153">#6153</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[checkpointio] fix zero optimizer async save memory (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2679229499" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6151" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6151/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6151">#6151</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2678198917" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6149" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6149/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6149">#6149</a> from ver217/hotfix/ckpt by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> <li>[checkpointio] disable buffering by <a href="https://api.github.com/users/ver217">ver217</a></li> <li>[checkpointio] fix pinned state dict by <a href="https://api.github.com/users/ver217">ver217</a></li> <li>[checkpointio] fix size compute by <a href="https://api.github.com/users/ver217">ver217</a></li> <li>[checkpointio] fix performance issue (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2667267656" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6139" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6139/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6139">#6139</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[checkpointio] support async model save (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2655172233" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6131" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6131/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6131">#6131</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>News</h3> <ul> <li>[news] release colossalai for sora (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2756058942" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6166" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6166/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6166">#6166</a>) by <a href="https://api.github.com/users/binmakeswell">binmakeswell</a></li> </ul> <h3>Hotfix</h3> <ul> <li>[hotfix] improve compatibility (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2755696162" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6165" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6165/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6165">#6165</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[Hotfix] hotfix normalization (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2755372719" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6163" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6163/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6163">#6163</a>) by <a href="https://api.github.com/users/duanjunwen">duanjunwen</a></li> <li>[hotfix] fix zero comm buffer init (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2719828404" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6154" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6154/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6154">#6154</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[hotfix] fix flash attn window_size err (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2657676536" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6132" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6132/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6132">#6132</a>) by <a href="https://api.github.com/users/duanjunwen">duanjunwen</a></li> </ul> <h3>Doc</h3> <ul> <li>[doc] add bonus event (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2755628950" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6164" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6164/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6164">#6164</a>) by <a href="https://api.github.com/users/binmakeswell">binmakeswell</a></li> <li>[doc] update cloud link (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2675701123" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6148" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6148/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6148">#6148</a>) by <a href="https://api.github.com/users/Sze-qq">Sze-qq</a></li> <li>[doc] add hpc cloud intro (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2674825664" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6147" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6147/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6147">#6147</a>) by <a href="https://api.github.com/users/Sze-qq">Sze-qq</a></li> </ul> <h3>Device</h3> <ul> <li>[Device]Support npu (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2735095423" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6159" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6159/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6159">#6159</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> </ul> <h3>Fix</h3> <ul> <li>[fix] fix bug caused by perf version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2728884642" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6156" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6156/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6156">#6156</a>) by <a href="https://api.github.com/users/duanjunwen">duanjunwen</a></li> <li>[fix] multi-node backward slowdown (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2657685584" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6134" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6134/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6134">#6134</a>) by <a href="https://api.github.com/users/BurkeHulk">Hanks</a></li> </ul> <h3>Optim</h3> <ul> <li>[optim] hotfix adam load (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2674214127" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6146" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6146/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6146">#6146</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Zerobubble</h3> <ul> <li>[Zerobubble] merge main. (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2667991156" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6142" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6142/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6142">#6142</a>) by <a href="https://api.github.com/users/duanjunwen">duanjunwen</a></li> </ul> <h3>Async io</h3> <ul> <li>[async io]supoort async io (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2658569645" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6137" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6137/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6137">#6137</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> </ul> <h3>Ckpt</h3> <ul> <li>[ckpt] Add async ckpt api (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2657969413" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6136" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6136/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6136">#6136</a>) by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> </ul> <h3>Cli</h3> <ul> <li>[cli] support run as module option (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2657838340" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6135" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6135/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6135">#6135</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Zero</h3> <ul> <li>[zero] support extra dp (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2648804592" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6123" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6123/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6123">#6123</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Coati</h3> <ul> <li>[Coati] Refine prompt for better inference (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2641480070" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6117" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6117/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6117">#6117</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> </ul> <h3>Plugin</h3> <ul> <li>[plugin] support get_grad_norm (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2634851238" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6115" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6115/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6115">#6115</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.7...v0.4.6"><tt>v0.4.7...v0.4.6</tt></a></p> github-actions[bot] tag:github.com,2008:Repository/422274596/v0.4.6 2024-11-04T09:28:04Z Version v0.4.6 Release Today! <h2>What's Changed</h2> <h3>Release</h3> <ul> <li>[release] update version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2628662568" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6109" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6109/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6109">#6109</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Pre-commit.ci</h3> <ul> <li>[pre-commit.ci] pre-commit autoupdate (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2557306329" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6078" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6078/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6078">#6078</a>) by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> </ul> <h3>Checkpointio</h3> <ul> <li>[checkpointio] fix hybrid plugin model save (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2626209174" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6106" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6106/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6106">#6106</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Mcts</h3> <ul> <li>[MCTS] Add self-refined MCTS (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2604985633" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6098" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6098/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6098">#6098</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> </ul> <h3>Doc</h3> <ul> <li>[doc] sora solution news (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2610503230" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6100" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6100/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6100">#6100</a>) by <a href="https://api.github.com/users/binmakeswell">binmakeswell</a></li> </ul> <h3>Extension</h3> <ul> <li>[extension] hotfix compile check (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2610275633" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6099" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6099/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6099">#6099</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Hotfix</h3> <ul> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2601450046" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6096" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6096/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6096">#6096</a> from BurkeHulk/hotfix/lora_ckpt by <a href="https://api.github.com/users/BurkeHulk">Hanks</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.6...v0.4.5"><tt>v0.4.6...v0.4.5</tt></a></p> github-actions[bot] tag:github.com,2008:Repository/422274596/v0.4.5 2024-10-21T02:21:19Z Version v0.4.5 Release Today! <h2>What's Changed</h2> <h3>Release</h3> <ul> <li>[release] update version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2597043312" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6094" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6094/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6094">#6094</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Misc</h3> <ul> <li>[misc] fit torch api upgradation and remove legecy import (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2596667808" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6093" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6093/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6093">#6093</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Fp8</h3> <ul> <li>[fp8] add fallback and make compile option configurable (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2596349994" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6092" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6092/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6092">#6092</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Chore</h3> <ul> <li>[chore] refactor by <a href="https://api.github.com/users/botbw">botbw</a></li> </ul> <h3>Ckpt</h3> <ul> <li>[ckpt] add safetensors util by <a href="https://api.github.com/users/botbw">botbw</a></li> </ul> <h3>Pipeline</h3> <ul> <li>[pipeline] hotfix backward for multiple outputs (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2588275865" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6090" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6090/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6090">#6090</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Ring attention</h3> <ul> <li>[Ring Attention] Improve comments (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2576261671" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6085" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6085/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6085">#6085</a>) by <a href="https://api.github.com/users/Edenzzzz">Wenxuan Tan</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2547662156" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6071" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6071/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6071">#6071</a> from wangbluo/ring_attention by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> </ul> <h3>Coati</h3> <ul> <li>[Coati] Train DPO using PP (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2516071882" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6054" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6054/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6054">#6054</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> </ul> <h3>Shardformer</h3> <ul> <li>[shardformer] optimize seq parallelism (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2577962427" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6086" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6086/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6086">#6086</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[shardformer] fix linear 1d row and support uneven splits for fused qkv linear (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2575117624" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6084" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6084/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6084">#6084</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.5...v0.4.4"><tt>v0.4.5...v0.4.4</tt></a></p> github-actions[bot] tag:github.com,2008:Repository/422274596/v0.4.4 2024-09-19T02:53:35Z Version v0.4.4 Release Today! <h2>What's Changed</h2> <h3>Release</h3> <ul> <li>[release] update version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2525957954" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6062" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6062/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6062">#6062</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Colossaleval</h3> <ul> <li>[ColossalEval] support for vllm (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2516218055" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6056" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6056/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6056">#6056</a>) by <a href="https://api.github.com/users/Camille7777">Camille Zhong</a></li> </ul> <h3>Moe</h3> <ul> <li>[moe] add parallel strategy for shared_expert &amp;&amp; fix test for deepseek (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2526187309" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6063" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6063/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6063">#6063</a>) by <a href="https://api.github.com/users/botbw">botbw</a></li> </ul> <h3>Sp</h3> <ul> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2527550882" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6064" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6064/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6064">#6064</a> from wangbluo/fix_attn by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2523752210" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6061" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6061/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6061">#6061</a> from wangbluo/sp_fix by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> </ul> <h3>Doc</h3> <ul> <li>[doc] FP8 training and communication document (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2513040606" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6050" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6050/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6050">#6050</a>) by <a href="https://api.github.com/users/GuangyaoZhang">Guangyao Zhang</a></li> <li>[doc] update sp doc (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2516200749" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6055" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6055/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6055">#6055</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> </ul> <h3>Fp8</h3> <ul> <li>[fp8] Disable all_gather intranode. Disable Redundant all_gather fp8 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2519188783" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6059" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6059/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6059">#6059</a>) by <a href="https://api.github.com/users/GuangyaoZhang">Guangyao Zhang</a></li> <li>[fp8] fix missing fp8_comm flag in mixtral (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2518665370" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6057" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6057/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6057">#6057</a>) by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[fp8] hotfix backward hook (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2516056852" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6053" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6053/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6053">#6053</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Pre-commit.ci</h3> <ul> <li>[pre-commit.ci] auto fixes from pre-commit.com hooks by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> </ul> <h3>Hotfix</h3> <ul> <li>[hotfix] moe hybrid parallelism benchmark &amp; follow-up fix (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2512985973" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6048" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6048/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6048">#6048</a>) by <a href="https://api.github.com/users/botbw">botbw</a></li> </ul> <h3>Feature</h3> <ul> <li>[Feature] Split cross-entropy computation in SP (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2442555721" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5959" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5959/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5959">#5959</a>) by <a href="https://api.github.com/users/Edenzzzz">Wenxuan Tan</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.4...v0.4.3"><tt>v0.4.4...v0.4.3</tt></a></p> github-actions[bot] tag:github.com,2008:Repository/422274596/v0.4.3 2024-09-10T02:39:50Z Version v0.4.3 Release Today! <h2>What's Changed</h2> <h3>Release</h3> <ul> <li>[release] update version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2496774598" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6041" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6041/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6041">#6041</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Fp8</h3> <ul> <li>[fp8] disable all_to_all_fp8 in intranode (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2501976802" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6045" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6045/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6045">#6045</a>) by <a href="https://api.github.com/users/BurkeHulk">Hanks</a></li> <li>[fp8] fix linear hook (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2502141948" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6046" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6046/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6046">#6046</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[fp8] optimize all-gather (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2500505657" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6043" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6043/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6043">#6043</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[FP8] unsqueeze scale to make it compatible with torch.compile (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2493372641" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6040" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6040/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6040">#6040</a>) by <a href="https://api.github.com/users/GuangyaoZhang">Guangyao Zhang</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2469598544" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6012" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6012/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6012">#6012</a> from hpcaitech/feature/fp8_comm by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2485806262" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6033" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6033/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6033">#6033</a> from wangbluo/fix by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2479664827" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6024" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6024/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6024">#6024</a> from wangbluo/fix_merge by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2479634235" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6023" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6023/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6023">#6023</a> from wangbluo/fp8_merge by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> <li>[fp8] Merge feature/fp8_comm to main branch of Colossalai (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2471451309" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6016" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6016/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6016">#6016</a>) by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> <li>[fp8] zero support fp8 linear. (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2467513735" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6006" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6006/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6006">#6006</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[fp8] add use_fp8 option for MoeHybridParallelPlugin (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2467711615" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6009" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6009/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6009">#6009</a>) by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> <li>[fp8]update reduce-scatter test (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2465027522" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6002" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6002/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6002">#6002</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[fp8] linear perf enhancement by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[fp8] update torch.compile for linear_fp8 to &gt;= 2.4.0 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2465399187" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6004" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6004/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6004">#6004</a>) by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[fp8] support asynchronous FP8 communication (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2462938378" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5997" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5997/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5997">#5997</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[fp8] refactor fp8 linear with compile (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2460535479" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5993" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5993/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5993">#5993</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[fp8] support hybrid parallel plugin (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2455230776" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5982" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5982/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5982">#5982</a>) by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> <li>[fp8]Moe support fp8 communication (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2453225125" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5977" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5977/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5977">#5977</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[fp8] use torch compile (torch &gt;= 2.3.0) (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2454885140" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5979" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5979/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5979">#5979</a>) by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[fp8] support gemini plugin (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2454813186" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5978" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5978/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5978">#5978</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[fp8] support fp8 amp for hybrid parallel plugin (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2452917256" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5975" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5975/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5975">#5975</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[fp8] add fp8 linear (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2450308103" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5967" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5967/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5967">#5967</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[fp8]support all2all fp8 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2437320288" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5953" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5953/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5953">#5953</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>[FP8] rebase main (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2444580403" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5963" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5963/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5963">#5963</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2443923768" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5961" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5961/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5961">#5961</a> from ver217/feature/zeor-fp8 by <a href="https://api.github.com/users/BurkeHulk">Hanks</a></li> <li>[fp8] add fp8 comm for low level zero by <a href="https://api.github.com/users/ver217">ver217</a></li> </ul> <h3>Hotfix</h3> <ul> <li>[Hotfix] Remove deprecated install (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2500293003" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6042" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6042/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6042">#6042</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> <li>[Hotfix] Fix llama fwd replacement bug (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2482513283" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6031" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6031/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6031">#6031</a>) by <a href="https://api.github.com/users/Edenzzzz">Wenxuan Tan</a></li> <li>[Hotfix] Avoid fused RMSnorm import error without apex (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2457556427" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5985" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5985/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5985">#5985</a>) by <a href="https://api.github.com/users/Edenzzzz">Edenzzzz</a></li> <li>[Hotfix] README link (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2447828624" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5966" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5966/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5966">#5966</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> <li>[hotfix] Remove unused plan section (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2439246455" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5957" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5957/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5957">#5957</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> </ul> <h3>Colossalai/checkpoint_io/...</h3> <ul> <li>[colossalai/checkpoint_io/...] fix bug in load_state_dict_into_model; format error msg (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2474635858" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6020" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6020/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6020">#6020</a>) by <a href="https://api.github.com/users/flymin">Gao, Ruiyuan</a></li> </ul> <h3>Colossal-llama</h3> <ul> <li>[Colossal-LLaMA] Refactor latest APIs (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2482391053" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6030" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6030/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6030">#6030</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> </ul> <h3>Plugin</h3> <ul> <li>[plugin] hotfix zero plugin (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2488899778" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6036" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6036/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6036">#6036</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[plugin] add cast inputs option for zero (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2465087310" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6003" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6003/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6003">#6003</a>) (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2476880398" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6022" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6022/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6022">#6022</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[plugin] add cast inputs option for zero (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2465087310" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6003" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6003/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6003">#6003</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Ci</h3> <ul> <li>[CI] Remove triton version for compatibility bug; update req torch &gt;=2.2 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2472783390" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6018" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6018/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6018">#6018</a>) by <a href="https://api.github.com/users/Edenzzzz">Wenxuan Tan</a></li> </ul> <h3>Pre-commit.ci</h3> <ul> <li>[pre-commit.ci] auto fixes from pre-commit.com hooks by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> <li>[pre-commit.ci] auto fixes from pre-commit.com hooks by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> <li>[pre-commit.ci] auto fixes from pre-commit.com hooks by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> <li>[pre-commit.ci] auto fixes from pre-commit.com hooks by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> <li>[pre-commit.ci] auto fixes from pre-commit.com hooks by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> <li>[pre-commit.ci] pre-commit autoupdate (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2461566198" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5995" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5995/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5995">#5995</a>) by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> <li>[pre-commit.ci] auto fixes from pre-commit.com hooks by <a href="https://api.github.com/users/pre-commit-ci%5Bbot%5D">pre-commit-ci[bot]</a></li> </ul> <h3>Colossalchat</h3> <ul> <li>[ColossalChat] Add PP support (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2464965723" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6001" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6001/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6001">#6001</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> </ul> <h3>Misc</h3> <ul> <li>[misc] Use dist logger in plugins (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2469592391" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6011" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6011/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6011">#6011</a>) by <a href="https://api.github.com/users/Edenzzzz">Edenzzzz</a></li> <li>[misc] update compatibility (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2467661320" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/6008" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/6008/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/6008">#6008</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[misc] Bypass the huggingface bug to solve the mask mismatch problem (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2460425404" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5991" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5991/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5991">#5991</a>) by <a href="https://api.github.com/users/Hz188">Haze188</a></li> <li>[misc] remove useless condition by <a href="https://api.github.com/users/Hz188">haze188</a></li> <li>[misc] fix ci failure: change default value to false in moe plugin by <a href="https://api.github.com/users/Hz188">haze188</a></li> <li>[misc] remove incompatible test config by <a href="https://api.github.com/users/Hz188">haze188</a></li> <li>[misc] remove debug/print code by <a href="https://api.github.com/users/Hz188">haze188</a></li> <li>[misc] skip redunant test by <a href="https://api.github.com/users/Hz188">haze188</a></li> <li>[misc] solve booster hang by rename the variable by <a href="https://api.github.com/users/Hz188">haze188</a></li> </ul> <h3>Feature</h3> <ul> <li>[Feature] Zigzag Ring attention (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2405801745" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5905" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5905/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5905">#5905</a>) by <a href="https://api.github.com/users/Edenzzzz">Edenzzzz</a></li> <li>[Feature]: support FP8 communication in DDP, FSDP, Gemini (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2418255521" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5928" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5928/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5928">#5928</a>) by <a href="https://api.github.com/users/BurkeHulk">Hanks</a></li> <li>[Feature] llama shardformer fp8 support (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2427365310" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5938" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5938/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5938">#5938</a>) by <a href="https://api.github.com/users/GuangyaoZhang">Guangyao Zhang</a></li> <li>[Feature] MoE Ulysses Support (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2415143248" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5918" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5918/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5918">#5918</a>) by <a href="https://api.github.com/users/Hz188">Haze188</a></li> </ul> <h3>Chat</h3> <ul> <li>[Chat] fix readme (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2459951535" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5989" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5989/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5989">#5989</a>) by <a href="https://api.github.com/users/YeAnbang">YeAnbang</a></li> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2444211982" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5962" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5962/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5962">#5962</a> from hpcaitech/colossalchat by <a href="https://api.github.com/users/YeAnbang">YeAnbang</a></li> <li>[Chat] Fix lora (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2434515437" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5946" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5946/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5946">#5946</a>) by <a href="https://api.github.com/users/YeAnbang">YeAnbang</a></li> </ul> <h3>Test ci</h3> <ul> <li>[test ci]Feature/fp8 comm (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2455146167" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5981" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5981/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5981">#5981</a>) by <a href="https://api.github.com/users/flybird11111">flybird11111</a></li> </ul> <h3>Docs</h3> <ul> <li>[Docs] clarify launch port by <a href="https://api.github.com/users/Edenzzzz">Edenzzzz</a></li> </ul> <h3>Test</h3> <ul> <li>[test] add zero fp8 test case by <a href="https://api.github.com/users/ver217">ver217</a></li> <li>[test] add check by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[test] fix test: test_zero1_2 by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[test] add mixtral modelling test by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[test] pass mixtral shardformer test by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[test] mixtra pp shard test by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[test] add mixtral transformer test by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[test] add mixtral for sequence classification by <a href="https://api.github.com/users/botbw">hxwang</a></li> </ul> <h3>Lora</h3> <ul> <li>[lora] lora support hybrid parallel plugin (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2439063504" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5956" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5956/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5956">#5956</a>) by <a href="https://api.github.com/users/wangbluo">Wang Binluo</a></li> </ul> <h3>Feat</h3> <ul> <li>[feat] Dist Loader for Eval (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2435174386" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5950" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5950/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5950">#5950</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> </ul> <h3>Chore</h3> <ul> <li>[chore] remove redundant test case, print string &amp; reduce test tokens by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[chore] docstring by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[chore] change moe_pg_mesh to private by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[chore] solve moe ckpt test failure and some other arg pass failure by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[chore] minor fix after rebase by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[chore] minor fix by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[chore] arg pass &amp; remove drop token by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[chore] trivial fix by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[chore] manually revert unintended commit by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[chore] handle non member group by <a href="https://api.github.com/users/botbw">hxwang</a></li> </ul> <h3>Moe</h3> <ul> <li>[moe] solve dp axis issue by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[moe] remove force_overlap_comm flag and add warning instead by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>Revert "[moe] implement submesh initialization" by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[moe] refactor mesh assignment by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[moe] deepseek moe sp support by <a href="https://api.github.com/users/Hz188">haze188</a></li> <li>[moe] remove ops by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[moe] full test for deepseek and mixtral (pp + sp to fix) by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[moe] finalize test (no pp) by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[moe] init moe plugin comm setting with sp by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[moe] clean legacy code by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[moe] test deepseek by <a href="https://api.github.com/users/botbw">hxwang</a></li> <li>[moe] implement tp by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[moe] add mixtral dp grad scaling when not all experts are activated by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[moe] implement submesh initialization by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[moe] implement transit between non moe tp and ep by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[moe] fix plugin by <a href="https://api.github.com/users/botbw">hxwang</a></li> </ul> <h3>Doc</h3> <ul> <li>[doc] add MoeHybridParallelPlugin docstring by <a href="https://api.github.com/users/botbw">botbw</a></li> </ul> <h3>Deepseek</h3> <ul> <li>[deepseek] replace attn (a workaround for bug in transformers) by <a href="https://api.github.com/users/botbw">hxwang</a></li> </ul> <h3>Bug</h3> <ul> <li>[bug] fix: somehow logger hangs the program by <a href="https://api.github.com/users/botbw">botbw</a></li> </ul> <h3>Zero</h3> <ul> <li>[zero] solve hang by <a href="https://api.github.com/users/botbw">botbw</a></li> <li>[zero] solve hang by <a href="https://api.github.com/users/botbw">hxwang</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.3...v0.4.2"><tt>v0.4.3...v0.4.2</tt></a></p> github-actions[bot] tag:github.com,2008:Repository/422274596/v0.4.2 2024-07-31T02:06:47Z Version v0.4.2 Release Today! <h2>What's Changed</h2> <h3>Release</h3> <ul> <li>[release] update version (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2436949515" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5952" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5952/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5952">#5952</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Zero</h3> <ul> <li>[zero] hotfix update master params (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2436872213" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5951" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5951/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5951">#5951</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Feat</h3> <ul> <li>[Feat] Distrifusion Acceleration Support for Diffusion Inference (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2397439059" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5895" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5895/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5895">#5895</a>) by <a href="https://api.github.com/users/LRY89757">Runyu Lu</a></li> </ul> <h3>Shardformer</h3> <ul> <li>[shardformer] hotfix attn mask (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2434824976" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5947" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5947/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5947">#5947</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> <li>[shardformer] hotfix attn mask (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2434397820" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5945" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5945/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5945">#5945</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <h3>Chat</h3> <ul> <li>Merge pull request <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2415701435" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5922" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5922/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5922">#5922</a> from hpcaitech/kto by <a href="https://api.github.com/users/YeAnbang">YeAnbang</a></li> </ul> <h3>Feature</h3> <ul> <li>[Feature] Add a switch to control whether the model checkpoint needs to be saved after each epoch ends (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2430233965" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5941" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5941/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5941">#5941</a>) by <a href="https://api.github.com/users/zhurunhua">zhurunhua</a></li> </ul> <h3>Hotfix</h3> <ul> <li>[Hotfix] Fix ZeRO typo <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2426534777" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5936" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5936/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5936">#5936</a> by <a href="https://api.github.com/users/Edenzzzz">Edenzzzz</a></li> </ul> <h3>Fix bug</h3> <ul> <li>[FIX BUG] convert env param to int in (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2424696335" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5934" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5934/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5934">#5934</a>) by <a href="https://api.github.com/users/flymin">Gao, Ruiyuan</a></li> <li>[FIX BUG] UnboundLocalError: cannot access local variable 'default_conversation' where it is not associated with a value (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2418487707" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5931" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5931/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5931">#5931</a>) by <a href="https://api.github.com/users/zhurunhua">zhurunhua</a></li> </ul> <h3>Colossalchat</h3> <ul> <li>[ColossalChat] Hotfix for ColossalChat (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2410410373" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5910" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5910/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5910">#5910</a>) by <a href="https://api.github.com/users/TongLi3701">Tong Li</a></li> </ul> <h3>Examples</h3> <ul> <li>[Examples] Add lazy init to OPT and GPT examples (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2417614724" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5924" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5924/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5924">#5924</a>) by <a href="https://api.github.com/users/Edenzzzz">Edenzzzz</a></li> </ul> <h3>Plugin</h3> <ul> <li>[plugin] support all-gather overlap for hybrid parallel (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2415144186" data-permission-text="Title is private" data-url="https://github.com/hpcaitech/ColossalAI/issues/5919" data-hovercard-type="pull_request" data-hovercard-url="/hpcaitech/ColossalAI/pull/5919/hovercard" href="https://github.com/hpcaitech/ColossalAI/pull/5919">#5919</a>) by <a href="https://api.github.com/users/ver217">Hongxin Liu</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/hpcaitech/ColossalAI/compare/v0.4.2...v0.4.1"><tt>v0.4.2...v0.4.1</tt></a></p> github-actions[bot]