Skip to content

[ruby] Add Support for ERB files #5447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

AndreiDreyer
Copy link
Contributor

@AndreiDreyer AndreiDreyer commented Apr 23, 2025

  • Added support for ERB files
  • Added templateOutRaw (<%== %>) and templateOutEscape (<%= %>) operator calls
  • Added RETURN for the joern__buffer which holds all of the appended strings from the ERB lowering
  • Added .html.erb files as config files

* Added tests for ERB processing

* update test expectation

* Add isLineFeed func

* removed erb config file test

* Fixed new line issues with ERB files, updated test expectations
… operators. Included explicit return for ERB file processing
@AndreiDreyer AndreiDreyer added the ruby Relates to rubysrc2cpg label Apr 23, 2025
@AndreiDreyer AndreiDreyer self-assigned this Apr 23, 2025
Copy link
Contributor

@DavidBakerEffendi DavidBakerEffendi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@maltek
Copy link
Contributor

maltek commented Apr 24, 2025

I'm still missing a lot of .erb files. E.g. for https://github.com/chatwoot/chatwoot:

ocular> cpg.file.name(".*.html.erb").size
val res18: Int = 40

ocular> cpg.configFile.name(".*.html.erb").size
val res19: Int = 88

The frontend spits out some warnings for the project, e.g. Yield expression outside of method scope: yield(:head) and Type 'String' is considered a 'core' type, not a 'Kernel-contained' type - but not enough to explain 48 missing files.

@AndreiDreyer
Copy link
Contributor Author

@maltek these are the new results on chatwoot with all of the changes made for ERB handling.

joern> cpg.configFile.name(".*.html.erb").l.map(_.name).l.diff(cpg.file.name(".*.html.erb").map(_.name).l)
val res1: List[String] = List()

joern> cpg.file.name(".*.html.erb").l.size
val res2: Int = 88

joern> cpg.configFile.name(".*.html.erb").l.size
val res3: Int = 88

import io.shiftleft.codepropertygraph.generated.nodes.{Block, Call, FieldIdentifier, Identifier, Literal, TypeRef}
import io.shiftleft.semanticcpg.language.*

class ErbTests extends RubyCode2CpgFixture {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you writing tests that include the lowering of a ruby AST into CPG?
The ERB support takes input ERB files and emits ruby code that is then fed into rubyAstGen. For the later part we already have plenty of tests. Here it would be much better readable if you test the first step ( ERB -> ruby code) the added benefit would be much better readability.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ERB -> Ruby code is tested on the ruby_ast_gen project e.g., https://github.com/joernio/ruby_ast_gen/blob/main/spec/ruby_ast_gen_spec.rb#L155

Albeit, one could argue for more tests there perhaps, but we haven't designed the RubyAstGen hook to give us the lowered ERB code as-is, only via AST

@ml86 ml86 self-requested a review May 15, 2025 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ruby Relates to rubysrc2cpg
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants