Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add Python implementation of accelerated OP #67

Merged
merged 10 commits into from
Nov 29, 2022
Merged

feat: add Python implementation of accelerated OP #67

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
0c783b6
chore: update build requirements
XuehaiPan Nov 23, 2022
26dc334
feat: make CXX/CUDA extenstion optional
XuehaiPan Nov 24, 2022
4fa9b86
feat: build pure-Python wheel
XuehaiPan Sep 5, 2022
efb7436
feat(workflows): enable pure-Python tests
XuehaiPan Nov 24, 2022
ecd4a43
docs(CHANGELOG): update CHANGELOG.md
XuehaiPan Nov 24, 2022
2a08697
chore: update setup.py
XuehaiPan Nov 27, 2022
18a0084
chore: use inplace operators
XuehaiPan Nov 29, 2022
8653f7b
chore(workflows): update action versions
XuehaiPan Nov 29, 2022
1840b64
chore: use torch.Tensor.copy_
XuehaiPan Nov 29, 2022
75e8957
chore: split tests
XuehaiPan Nov 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: use inplace operators
  • Loading branch information
@XuehaiPan
XuehaiPan committed Nov 29, 2022
commit 18a0084523dc1fb0ac3a6585f9729d6412431965
6 changes: 4 additions & 2 deletions torchopt/accelerated_op/_src/adam_op.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,11 @@ def backward_updates(
inv_one_minus_pow_b2 = 1.0 / (1.0 - pow(b2, count))

updates_div_new_mu = updates.div(new_mu)
denominator = updates_div_new_mu.mul(one_minus_pow_b1)
denominator = updates_div_new_mu.mul_(one_minus_pow_b1)
dnew_mu_out = dupdates.mul(updates_div_new_mu)
XuehaiPan marked this conversation as resolved.
Show resolved Hide resolved
dnew_nu_out = dupdates.mul(updates).mul_(denominator.square()).mul_(-0.5 * inv_one_minus_pow_b2)
dnew_nu_out = (
dupdates.mul(updates).mul_(denominator.square_()).mul_(-0.5 * inv_one_minus_pow_b2)
)

mask = new_mu == 0
dnew_mu_out[mask] = 0
Expand Down