-
Notifications
You must be signed in to change notification settings - Fork 175
Further optimize ripper translator by not using delegate
#3868
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,6 @@ | ||
| # frozen_string_literal: true | ||
| # :markup: markdown | ||
|
|
||
| require "delegate" | ||
|
|
||
| module Prism | ||
| # This class is responsible for lexing the source using prism and then | ||
| # converting those tokens to be compatible with Ripper. In the vast majority | ||
|
|
@@ -201,27 +199,43 @@ def deconstruct_keys(keys) | |
| # When we produce tokens, we produce the same arrays that Ripper does. | ||
| # However, we add a couple of convenience methods onto them to make them a | ||
| # little easier to work with. We delegate all other methods to the array. | ||
| class Token < SimpleDelegator | ||
| # @dynamic initialize, each, [] | ||
| class Token < BasicObject | ||
| # Create a new token object with the given ripper-compatible array. | ||
| def initialize(array) | ||
| @array = array | ||
| end | ||
|
|
||
| # The location of the token in the source. | ||
| def location | ||
| self[0] | ||
| @array[0] | ||
| end | ||
|
|
||
| # The type of the token. | ||
| def event | ||
| self[1] | ||
| @array[1] | ||
| end | ||
|
|
||
| # The slice of the source that this token represents. | ||
| def value | ||
| self[2] | ||
| @array[2] | ||
| end | ||
|
|
||
| # The state of the lexer when this token was produced. | ||
| def state | ||
| self[3] | ||
| @array[3] | ||
| end | ||
|
|
||
| # We want to pretend that this is just an Array. | ||
| def ==(other) # :nodoc: | ||
| @array == other | ||
| end | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For speed it might be relevant to define |
||
|
|
||
| def respond_to_missing?(name, include_private = false) # :nodoc: | ||
| @array.respond_to?(name, include_private) | ||
| end | ||
|
|
||
| def method_missing(name, ...) # :nodoc: | ||
| @array.send(name, ...) | ||
| end | ||
| end | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason to not just subclass Array?
That would avoid an extra indirection and (in theory at least) be more compatible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried this but then ignores the
==from the subclass. Somehow also slower on crubyThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah right, that's subtle, I think it's because of these semantics: https://github.com/truffleruby/truffleruby/blob/a48ecf894f19636f09235f5ac70ea6d9889be255/src/main/ruby/truffleruby/core/array.rb#L129-L132
i.e. if other is not an Array then always use
other == self, but if other is an Array then just compare all elements and don't callother.==Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change turned out to reveal a bug in TruffleRuby, as TruffleRuby wouldn't destructure/splat such "BasicObject arrays" since BasicObject doesn't have
respond_to?.Fixed in truffleruby/truffleruby#4128
Fixing/revealing that bug is good, but perf-wise having to go through that slow path coercion (
rb_convert_type()) doesn't seem great, I think we should use regular arrays like ::Ripper and then maybe have some private constants so we can do things like:Any thoughts on that?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For Token subclasses we'd still need to use those to have the custom
==, but for instances of Token we could use a plain Array instead.It would also mean less allocations which is always helpful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this https://github.com/ruby/prism/pull/3868/changes#r2724651412? Sorry, I didn't see that comment at all. Would defining
to_arydo it?Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would still go through rb_convert_type() but more direct than without:
https://github.com/truffleruby/truffleruby/blob/4180ab960fe223d333e86b6f3263eb1e29d8f942/src/main/ruby/truffleruby/core/type.rb#L301
vs
https://github.com/truffleruby/truffleruby/blob/4180ab960fe223d333e86b6f3263eb1e29d8f942/src/main/ruby/truffleruby/core/type.rb#L303
So yeah that'd be good but even better would be to use real arrays as I'd think e.g. all method lookups in
rb_convert_type()are uncached on CRuby, and likely megamorphic on TruffleRuby.