Skip to content

Regex string allocation - #1943

Merged
dblock merged 4 commits into
ruby-grape:masterfrom
ericproulx:regex_string_allocation
Dec 16, 2019
Merged

dblock merged 4 commits into
ruby-grape:masterfrom
ericproulx:regex_string_allocation

Conversation

@ericproulx

Copy link
Copy Markdown
Contributor

While memory profilling our app, I found these strange memory allocations :

1442  "(?<_"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
468  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

983  ")"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 9  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/mustermann-grape-1.0.0/lib/mustermann/grape.rb:20

974  ">"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41

468  ">)"
468  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53
12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:a|%61)(?:c|%63)(?:c|%63)(?:o|%6f|%6F)(?:u|%75)(?:n|%6e|%6E)(?:t|%74)(?:s|%73)\\/(?<account_id>[^\\/\\?#.]+)\\/(?:m|%6d|%6D)(?:e|%65)(?:s|%73)(?"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:a|%61)(?:p|%70)(?:p|%70)(?:o|%6f|%6F)(?:i|%69)(?:n|%6e|%6E)(?:t|%74)(?:m|%6d|%6D)(?:e|%65)(?:n|%6e|%6E)(?:t|%74)(?:s|%73)\\/(?:a|%61)(?:v|%7"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:c|%63)(?:h|%68)(?:a|%61)(?:n|%6e|%6E)(?:g|%67)(?:e|%65)(?:_|%5f|%5F)(?:r|%72)(?:e|%65)(?:q|%71)(?:u|%75)(?:e|%65)(?:s|%73)(?:t|%74)(?:s|%73"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:c|%63)(?:o|%6f|%6F)(?:n|%6e|%6E)(?:v|%76)(?:e|%65)(?:r|%72)(?:s|%73)(?:a|%61)(?:t|%74)(?:i|%69)(?:o|%6f|%6F)(?:n|%6e|%6E)(?:s|%73)\\/(?<conv"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:a|%61)(?:p|%70)(?:p|%70)(?:o|%6f|%6F)(?:i|%69)(?:n|%6e"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:f|%66)(?:o|%6f|%6F)(?:r|%72)(?:m|%6d|%6D)(?:s|%73)\\/(?"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:m|%6d|%6D)(?:a|%61)(?:n|%6e|%6E)(?:a|%61)(?:g|%67)(?:e"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:t|%74)(?:e|%65)(?:a|%61)(?:m|%6d|%6D)(?:s|%73)(?:(?:\\."
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:h|%68)(?:o|%6f|%6F)(?:s|%73)(?:p|%70)(?:i|%69)(?:t|%74)(?:a|%61)(?:l|%6c|%6C)(?:s|%73)\\/(?<hospital_id>[^\\/\\?#.]+)\\/(?:c|%63)(?:o|%6f|%6F)("
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:s|%73)(?:c|%63)(?:h|%68)(?:e|%65)(?:d|%64)(?:u|%75)(?:l|%6c|%6C)(?:e|%65)\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<i"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

Turns out, its 2 regex defined in router.rb with interpolation. I replaced the 2 lliteral regexs by the Regexp constructor. Therefore, the strings above (memory profiling) are just allocated once and benefits from the frozen_string_literal

@grape-bot

grape-bot commented Dec 14, 2019

Copy link
Copy Markdown
1 Warning
⚠️ There’re library changes, but not tests. That’s OK as long as you’re refactoring existing code.

Generated by 🚫 danger

Comment thread CHANGELOG.md Outdated
#### Features

* Your contribution here.
* [#1943](https://github.com/ruby-grape/grape/pull/1943): Regex string allocation - [@ericproulx](https://github.com/ericproulx).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick, this should describe what it does, eg. "Reduce number of regex string allocations"

@dblock

dblock commented Dec 14, 2019

Copy link
Copy Markdown
Member

Great work. Rebase, I'll merge.

@ericproulx
ericproulx force-pushed the regex_string_allocation branch from a1a3a03 to 85346f2 Compare December 14, 2019 21:07
@ericproulx
ericproulx force-pushed the regex_string_allocation branch from 85346f2 to 489e900 Compare December 14, 2019 21:09
Comment thread CHANGELOG.md
Comment thread CHANGELOG.md Outdated
* Your contribution here.
* [#1942](https://github.com/ruby-grape/grape/pull/1942): Optimized retained memory methods - [@ericproulx](https://github.com/ericproulx).
* [#1943](https://github.com/ruby-grape/grape/pull/1943): Reduce number of regex string allocations - [@ericproulx](https://github.com/ericproulx).
[#1942](https://github.com/ruby-grape/grape/pull/1942): Optimized retained memory methods - [@ericproulx](https://github.com/ericproulx).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing * so danger complains.

@dblock
dblock merged commit d30b780 into ruby-grape:master Dec 16, 2019
@dblock

dblock commented Dec 16, 2019

Copy link
Copy Markdown
Member

Merged thank you

basjanssen pushed a commit to basjanssen/grape that referenced this pull request Feb 28, 2020
Use Regexp instead of // for regex in router.rb for a more efficient string allocation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants