Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regex string allocation #1943

Merged
merged 4 commits into from
Dec 16, 2019
Merged

Regex string allocation #1943

merged 4 commits into from
Dec 16, 2019

Conversation

ericproulx
Copy link
Contributor

While memory profilling our app, I found these strange memory allocations :

1442  "(?<_"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
468  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

983  ")"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 9  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/mustermann-grape-1.0.0/lib/mustermann/grape.rb:20

974  ">"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41

468  ">)"
468  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53
12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:a|%61)(?:c|%63)(?:c|%63)(?:o|%6f|%6F)(?:u|%75)(?:n|%6e|%6E)(?:t|%74)(?:s|%73)\\/(?<account_id>[^\\/\\?#.]+)\\/(?:m|%6d|%6D)(?:e|%65)(?:s|%73)(?"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:a|%61)(?:p|%70)(?:p|%70)(?:o|%6f|%6F)(?:i|%69)(?:n|%6e|%6E)(?:t|%74)(?:m|%6d|%6D)(?:e|%65)(?:n|%6e|%6E)(?:t|%74)(?:s|%73)\\/(?:a|%61)(?:v|%7"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:c|%63)(?:h|%68)(?:a|%61)(?:n|%6e|%6E)(?:g|%67)(?:e|%65)(?:_|%5f|%5F)(?:r|%72)(?:e|%65)(?:q|%71)(?:u|%75)(?:e|%65)(?:s|%73)(?:t|%74)(?:s|%73"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:c|%63)(?:o|%6f|%6F)(?:n|%6e|%6E)(?:v|%76)(?:e|%65)(?:r|%72)(?:s|%73)(?:a|%61)(?:t|%74)(?:i|%69)(?:o|%6f|%6F)(?:n|%6e|%6E)(?:s|%73)\\/(?<conv"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:a|%61)(?:p|%70)(?:p|%70)(?:o|%6f|%6F)(?:i|%69)(?:n|%6e"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:f|%66)(?:o|%6f|%6F)(?:r|%72)(?:m|%6d|%6D)(?:s|%73)\\/(?"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:m|%6d|%6D)(?:a|%61)(?:n|%6e|%6E)(?:a|%61)(?:g|%67)(?:e"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:t|%74)(?:e|%65)(?:a|%61)(?:m|%6d|%6D)(?:s|%73)(?:(?:\\."
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:h|%68)(?:o|%6f|%6F)(?:s|%73)(?:p|%70)(?:i|%69)(?:t|%74)(?:a|%61)(?:l|%6c|%6C)(?:s|%73)\\/(?<hospital_id>[^\\/\\?#.]+)\\/(?:c|%63)(?:o|%6f|%6F)("
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:s|%73)(?:c|%63)(?:h|%68)(?:e|%65)(?:d|%64)(?:u|%75)(?:l|%6c|%6C)(?:e|%65)\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<i"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

Turns out, its 2 regex defined in router.rb with interpolation. I replaced the 2 lliteral regexs by the Regexp constructor. Therefore, the strings above (memory profiling) are just allocated once and benefits from the frozen_string_literal

@grape-bot
Copy link

grape-bot commented Dec 14, 2019 •
edited
Loading

1 Warning
⚠️ There’re library changes, but not tests. That’s OK as long as you’re refactoring existing code.

Generated by 🚫 danger

dblock
dblock reviewed Dec 14, 2019
View reviewed changes
CHANGELOG.md Outdated
@@ -3,6 +3,7 @@
#### Features

* Your contribution here.
* [#1943](https://github.com/ruby-grape/grape/pull/1943): Regex string allocation - [@ericproulx](https://github.com/ericproulx).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick, this should describe what it does, eg. "Reduce number of regex string allocations"

@dblock
Copy link
Member

dblock commented Dec 14, 2019

Great work. Rebase, I'll merge.

dblock
dblock reviewed Dec 15, 2019
View reviewed changes
CHANGELOG.md Show resolved Hide resolved
dblock
dblock reviewed Dec 15, 2019
View reviewed changes
CHANGELOG.md Outdated
@@ -3,7 +3,8 @@
#### Features

* Your contribution here.
* [#1942](https://github.com/ruby-grape/grape/pull/1942): Optimized retained memory methods - [@ericproulx](https://github.com/ericproulx).
* [#1943](https://github.com/ruby-grape/grape/pull/1943): Reduce number of regex string allocations - [@ericproulx](https://github.com/ericproulx).
[#1942](https://github.com/ruby-grape/grape/pull/1942): Optimized retained memory methods - [@ericproulx](https://github.com/ericproulx).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing * so danger complains.

@dblock dblock merged commit d30b780 into ruby-grape:master Dec 16, 2019
@dblock
Copy link
Member

dblock commented Dec 16, 2019

Merged thank you

basjanssen pushed a commit to basjanssen/grape that referenced this pull request Feb 28, 2020
@ericproulx @basjanssen
Regex string allocation (ruby-grape#1943)
aaccdb1 
Use Regexp instead of // for regex in router.rb for a more efficient string allocation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dblock dblock dblock left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ericproulx @grape-bot @dblock