Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RNNprofiler: fix gates size retrieval logic in _rnn_flops #3921

Merged
merged 6 commits into from
Jul 24, 2023
Merged

RNNprofiler: fix gates size retrieval logic in _rnn_flops #3921

merged 6 commits into from
Jul 24, 2023

Conversation

pinstripe-potoroo
Copy link
Contributor

Hello,

Sorry, this MR fixes a bug I introduced myself by mistaking the shape of w_ih to be (input_size, hidden_size), when in reality it is (N*hidden_size, input_size), where N differs depending on the RNN cell type (1 if RNN, 3 if GRU, 4 if LSTM).

To avoid complexifying the logic, I went back to something very close to original implementation, simply multiplying by 2 for FLOP (instead of MAC) and subtracting what I called gates_size = N * hidden_size to avoid counting the bias twice.

Thanks, and sorry for introducing another bug through my previous fix...

Cheers

@pinstripe-potoroo pinstripe-potoroo requested a review from cli99 as a code owner July 10, 2023 11:50
@pinstripe-potoroo
Copy link
Contributor Author

Small bump on this MR as the FLOP count is quite off for RNN (because of my previous MR). Although I'm probably the only one still using RNNs these days 😅 I know the MR queue is huge, so no pressure.

@pinstripe-potoroo
Copy link
Contributor Author

is there anything I can do to help? would you want me to add a unit test?

@pinstripe-potoroo
Copy link
Contributor Author

small ping on the MR, glad to help if anything is blocking the approval / merging

@tjruwase tjruwase added this pull request to the merge queue Jul 24, 2023
Merged via the queue into microsoft:master with commit f4d18fa Jul 24, 2023
@pinstripe-potoroo
Copy link
Contributor Author

thank you @tjruwase and sorry again for introducing a bug!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants