Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

getVariable is an efficiency hotspot #1639

Closed
DavyLandman opened this issue Aug 25, 2022 · 5 comments · Fixed by #1640
Closed

getVariable is an efficiency hotspot #1639

DavyLandman opened this issue Aug 25, 2022 · 5 comments · Fixed by #1640
Labels
enhancement interpreter parser

Comments

@DavyLandman
Copy link
Member

DavyLandman commented Aug 25, 2022 •
edited
Loading

I've been profiling why it takes 30s+ to generate IDE's in vscode. I've noticed a pattern, it's mostly in the parsergenerator generating the parsers. But I was just curious why, so I dived in a bit.

image
image

Looking at the whole profile, 15% of the time is spend on these getVariable calls, and especially the case where it's just throwing an exception.

rascal/src/org/rascalmpl/interpreter/env/ModuleEnvironment.java

Lines 432 to 435 in fa8b7d2

ModuleEnvironment imported = getImport(modulename);
if (imported == null) {
throw new UndeclaredModule(modulename, name);
}

Looking at the caller code:

rascal/src/org/rascalmpl/interpreter/TraversalEvaluator.java

Lines 411 to 429 in fa8b7d2

assert args != null;
QualifiedName n = Names.toQualifiedName(cons.getType().getName(), cons.getName(), null);
rcons = (IConstructor) eval.call(n, kwParams != null ? kwParams : Collections.<String,IValue>emptyMap(), args);
}
catch (UndeclaredFunction | UndeclaredModule | ArgumentMismatch e) {
// This may happen when visiting data constructors dynamically which are not
// defined in the current scope. For example, when data was serialized and the format
// has changed in the meantime, or when a generic function from a library calls visit.
//
// We issue a warning, because it is indicative of a bug
// and normalizing "rewrite rules" will not trigger at all, but we can gracefully continue
// because we know what the tree looked like before we started visiting.
if (kwParams != null) {
rcons = (IConstructor) eval.getValueFactory().constructor(cons.getConstructorType(), args, kwParams);
}
else {
rcons = (IConstructor) eval.getValueFactory().constructor(cons.getConstructorType(), args);
}
}

It appears to only suppose to happen when the constructors are not defined in the current scope. This leads me to wondering if there might be an import missing in the parser generator that could make this code a lot faster? The warning was removed in this commit: 330b88e

@jurgenvinju can you take a look? or explain why we can't work around it?

Also, it's all happening during import, the actual #GrammarReference is causing no problems. So my interpretation it's about how during imports we handle concrete syntax.
@DavyLandman
Copy link
Member Author

DavyLandman commented Aug 25, 2022 •
edited
Loading

I've also run the parser generator through the profiler:

rascal>src = newGenerate("_test", "TestParser", gr);
FRAMES PROFILE: 347 data points, 10169 ticks, tick = 1 milliSecs
                  Scope   Ticks        %  Source
               unsetRec    7426    73.0%  |std:///Node.rsc|(3277,179,<147,0>,<153,2>)
            sym2newitem     617     6.1%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(18527,7439,<453,0>,<558,1>) 
       generateNewItems     310     3.0%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(12939,2408,<284,0>,<341,1>) 
       computeDontNests     263     2.6%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(10821,1011,<245,0>,<258,1>) 
generateCharClassArrays     226     2.2%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(25968,282,<560,0>,<568,1>)  
                   head     180     1.8%  |std:///List.rsc|(3601,960,<156,0>,<194,34>)
          literals2ints     131     1.3%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(17820,263,<420,0>,<432,1>)  
                rewrite     126     1.2%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(2505,105,<66,4>,<69,8>)     
                     uu     107     1.1%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(26939,81,<599,0>,<599,81>)  

ASTS PROFILE: 347 data points, 10169 ticks, tick = 1 milliSecs
                  Scope   Ticks        %  Source
               unsetRec    4909    48.3%  |std:///Node.rsc|(3421,1,<151,33>,<151,34>)
                  unset    2272    22.3%  |std:///Node.rsc|(3450,1,<152,23>,<152,24>)
               unsetRec     219     2.2%  |std:///Node.rsc|(3277,179,<147,0>,<153,2>)
       computeDontNests     186     1.8%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(11827,1,<257,131>,<257,132>)
                   head     148     1.5%  |std:///List.rsc|(3601,960,<156,0>,<194,34>)
            sym2newitem     133     1.3%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(18707,3,<457,13>,<457,16>)
str: "package _test;\n\nimport java.io.IOException;\nimport java.io.StringReader;\n\nimport io.usethesource.vallang.type.TypeFa

Looks like we should rewrite this unsetRec to be a java function:

rascal/src/org/rascalmpl/library/Node.rsc

Lines 151 to 153 in fa8b7d2

public &T unsetRec(&T x) = visit(x) {
case node n => unset(n)
};

@DavyLandman
Copy link
Member Author

Here is a repo case:

import Grammar;
import ParseTree;
import lang::rascal::grammar::ParserGenerator;
import lang::rascal::grammar::definition::Modules;
import lang::rascal::grammar::definition::Parameters;


import lang::rascal::\syntax::Rascal;

g = grammar(#start[Module]);

s = newGenerate("_test", "Test", g);
:set profiling true
s = newGenerate("_test", "Test", g);
FRAMES PROFILE: 516 data points, 19620 ticks, tick = 1 milliSecs
                   Scope   Ticks        %  Source
                unsetRec   13796    70.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/Node.rsc|(3277,179,<147,0>,<153,2>)
             sym2newitem    1338     6.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18527,7439,<453,0>,<558,1>)       
        generateNewItems     579     3.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12939,2408,<284,0>,<341,1>)       
                 rewrite     412     2.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2505,105,<66,4>,<69,8>)
 generateCharClassArrays     386     2.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(25968,282,<560,0>,<568,1>)        
               doNotNest     347     1.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Priorities.rsc|(845,5496,<28,0>,<133,1>)    
             newGenerate     318     1.6%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(1679,9138,<44,0>,<243,1>)
                    head     274     1.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(3601,960,<156,0>,<194,34>)
                striprec     271     1.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(531,157,<18,0>,<21,2>)
                      uu     218     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26939,81,<599,0>,<599,81>)        

ASTS PROFILE: 516 data points, 19620 ticks, tick = 1 milliSecs
                   Scope   Ticks        %  Source
                unsetRec    9228    47.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/Node.rsc|(3421,1,<151,33>,<151,34>)
                unsetRec    4318    22.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/Node.rsc|(3450,1,<152,23>,<152,24>)
             sym2newitem     316     1.6%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18707,3,<457,13>,<457,16>)        
                 rewrite     305     1.6%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2590,7,<68,30>,<68,37>)
                unsetRec     217     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/Node.rsc|(3277,179,<147,0>,<153,2>)
                    head     207     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(3601,960,<156,0>,<194,34>)

it takes 19s to generate the rascal parser on my machine. of which 70% is spend in unsetRec.

@DavyLandman DavyLandman mentioned this issue Aug 25, 2022
Merged
@DavyLandman
Copy link
Member Author

DavyLandman commented Aug 25, 2022 •
edited
Loading

Okay, rewriting that was "easy" (see: #1640). And made the parser generator 3x quicker.

New profile:

FRAMES PROFILE: 499 data points, 5704 ticks, tick = 1 milliSecs
                        Scope   Ticks        %  Source
                  sym2newitem    1023    17.9%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18527,7439,<453,0>,<558,1>)
             generateNewItems     420     7.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12939,2408,<284,0>,<341,1>)
                    doNotNest     369     6.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Priorities.rsc|(845,5496,<28,0>,<133,1>)
      generateCharClassArrays     364     6.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(25968,282,<560,0>,<568,1>)
                      rewrite     361     6.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2505,105,<66,4>,<69,8>)
                           cl     360     6.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13402,42,<290,2>,<290,44>)
                     striprec     291     5.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(531,157,<18,0>,<21,2>)
                         head     271     4.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(3601,960,<156,0>,<194,34>)
                           uu     248     4.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26939,81,<599,0>,<599,81>)
                  newGenerate     241     4.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(1679,9138,<44,0>,<243,1>)
                literals2ints     222     3.9%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(17820,263,<420,0>,<432,1>)
                       except     184     3.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Priorities.rsc|(7969,503,<175,0>,<182,80>)
                      getType     161     2.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12889,47,<281,0>,<281,47>)
                      getType     142     2.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12765,52,<279,0>,<279,52>)
                        index     129     2.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(5167,403,<221,0>,<239,57>)
                         tail     118     2.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(17714,666,<765,0>,<799,40>)
                         same      87     1.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Priorities.rsc|(10299,71,<226,0>,<226,71>)
                     str2syms      80     1.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Literals.rsc|(754,145,<23,0>,<26,1>)
             computeDontNests      65     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(10821,1011,<245,0>,<258,1>)
                      delabel      62     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(1193,68,<36,0>,<36,68>)
                        strip      61     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(882,35,<25,0>,<25,35>)

ASTS PROFILE: 499 data points, 5705 ticks, tick = 1 milliSecs
                        Scope   Ticks        %  Source
                     unsetRec     326     5.7%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13441,1,<290,41>,<290,42>)
                  sym2newitem     257     4.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18707,3,<457,13>,<457,16>)
                      rewrite     245     4.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2590,7,<68,30>,<68,37>)
                         head     200     3.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(3601,960,<156,0>,<194,34>)
                     striprec     189     3.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(577,5,<18,46>,<18,51>)
             generateNewItems     135     2.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13736,1,<297,22>,<297,23>)
                      getType     125     2.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12933,1,<281,44>,<281,45>)
                         size     120     2.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(5564,3,<239,51>,<239,54>)
      generateCharClassArrays     115     2.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26108,6,<563,31>,<563,37>)
                      rewrite     113     2.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2554,1,<67,13>,<67,14>)
                literals2ints     112     2.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18032,2,<428,26>,<428,28>)
                  sym2newitem      99     1.7%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(21620,3,<503,12>,<503,15>)
                         tail      93     1.6%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(17714,666,<765,0>,<799,40>)
                  sym2newitem      87     1.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18811,2,<462,14>,<462,16>)
                       escape      79     1.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(27013,4,<599,74>,<599,78>)
             generateNewItems      78     1.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13789,1,<297,75>,<297,76>)
                      getType      75     1.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12814,1,<279,49>,<279,50>)
                         head      71     1.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(4559,1,<194,32>,<194,33>)
                           uu      71     1.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26984,1,<599,45>,<599,46>)
             generateNewItems      69     1.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13457,1,<292,9>,<292,10>)
                      getType      67     1.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12765,52,<279,0>,<279,52>)
                  sym2newitem      65     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(23136,3,<523,83>,<523,86>)
      generateCharClassArrays      61     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26073,2,<562,13>,<562,15>)
                         same      58     1.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Priorities.rsc|(10299,71,<226,0>,<226,71>)

@jurgenvinju jurgenvinju changed the title Call overhead for getVariable is high, review imports for the parser generator? getVariable is an efficiency hotspot Aug 25, 2022
@DavyLandman
Copy link
Member Author

Note, the unsetRec implementation had some problems, (due to a bug in vallang, it wouldn't go into tuples).

After fixing this, I ran everything again.

import Grammar;
import ParseTree;
import util::Benchmark;
import lang::rascal::grammar::ParserGenerator;
import lang::rascal::grammar::definition::Modules;
import lang::rascal::grammar::definition::Parameters;


import lang::rascal::\syntax::Rascal;

g = grammar(#start[Module]);

s = newGenerate("_test", "Test", g);
:set profiling true
realTime(() { for (i <- [0..10]) { s = newGenerate("_test", "Test", g); }}) / 1000.0

(first original timings, got a new machine, so can't be compared to the old ones)

FRAMES PROFILE: 748 data points, 60469 ticks, tick = 1 milliSecs
                     Scope   Ticks        %  Source
                  unsetRec   43759    72.4%  |std:///Node.rsc|(3277,179,<147,0>,<153,2>)
               sym2newitem    3495     5.8%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(18527,7439,<453,0>,<558,1>)
          generateNewItems    1710     2.8%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(12939,2408,<284,0>,<341,1>)
   generateCharClassArrays    1153     1.9%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(25968,282,<560,0>,<568,1>)
                   rewrite    1090     1.8%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(2505,105,<66,4>,<69,8>)
                  striprec    1009     1.7%  |std:///lang/rascal/grammar/definition/Symbols.rsc|(531,157,<18,0>,<21,2>)
                 doNotNest     966     1.6%  |std:///lang/rascal/grammar/definition/Priorities.rsc|(845,5496,<28,0>,<133,1>)
               newGenerate     779     1.3%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(1679,9138,<44,0>,<243,1>)
                        uu     767     1.3%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(26939,81,<599,0>,<599,81>)
                      head     767     1.3%  |std:///List.rsc|(3601,960,<156,0>,<194,34>)
             literals2ints     625     1.0%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(17820,263,<420,0>,<432,1>)

ASTS PROFILE: 748 data points, 60469 ticks, tick = 1 milliSecs
                     Scope   Ticks        %  Source
                  unsetRec   29197    48.3%  |std:///Node.rsc|(3421,1,<151,33>,<151,34>)
                     unset   13812    22.8%  |std:///Node.rsc|(3450,1,<152,23>,<152,24>)
                   rewrite     747     1.2%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(2590,7,<68,30>,<68,37>)
               sym2newitem     745     1.2%  |std:///lang/rascal/grammar/ParserGenerator.rsc|(18707,3,<457,13>,<457,16>)
                  striprec     675     1.1%  |std:///lang/rascal/grammar/definition/Symbols.rsc|(577,5,<18,46>,<18,51>)
real: 91.926

after fixing vallang bug:

ASTS PROFILE: 764 data points, 20155 ticks, tick = 1 milliSecs
                        Scope   Ticks        %  Source
                     unsetRec    2582    12.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13441,1,<290,41>,<290,42>)
                      rewrite     904     4.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2590,7,<68,30>,<68,37>)
                     striprec     737     3.7%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(577,5,<18,46>,<18,51>)
                  sym2newitem     713     3.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18707,3,<457,13>,<457,16>)
                         head     604     3.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(3601,960,<156,0>,<194,34>)
                literals2ints     417     2.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18032,2,<428,26>,<428,28>)
             generateNewItems     402     2.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13736,1,<297,22>,<297,23>)
      generateCharClassArrays     384     1.9%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26108,6,<563,31>,<563,37>)
                  sym2newitem     344     1.7%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(21620,3,<503,12>,<503,15>)
                      getType     342     1.7%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12933,1,<281,44>,<281,45>)
                      rewrite     333     1.7%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2554,1,<67,13>,<67,14>)
                         size     313     1.6%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(5564,3,<239,51>,<239,54>)
                         tail     300     1.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(17714,666,<765,0>,<799,40>)
                       escape     299     1.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(27013,4,<599,74>,<599,78>)
             generateNewItems     286     1.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13789,1,<297,75>,<297,76>)
                           uu     276     1.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26984,1,<599,45>,<599,46>)
                  sym2newitem     264     1.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18811,2,<462,14>,<462,16>)
                  sym2newitem     264     1.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(23136,3,<523,83>,<523,86>)
             generateNewItems     256     1.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13457,1,<292,9>,<292,10>)
                      getType     226     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12765,52,<279,0>,<279,52>)
                     striprec     220     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(625,1,<19,38>,<19,39>)
real: 29.653

So it got 3.1x quicker. but it's still in there quite a bit. 12.8% namely.

Now if I inline the code for the rewriting, and avoid rebuilding new IValues, it gets a bit faster:

FRAMES PROFILE: 708 data points, 17007 ticks, tick = 1 milliSecs
                   Scope   Ticks        %  Source
             sym2newitem    2923    17.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18527,7439,<453,0>,<558,1>)
                      cl    1439     8.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13402,42,<290,2>,<290,44>)
                 rewrite    1183     7.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2505,105,<66,4>,<69,8>)
        generateNewItems    1137     6.7%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12939,2408,<284,0>,<341,1>)
               doNotNest    1100     6.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Priorities.rsc|(845,5496,<28,0>,<133,1>)
 generateCharClassArrays    1050     6.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(25968,282,<560,0>,<568,1>)
                striprec     985     5.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(531,157,<18,0>,<21,2>)
                      uu     815     4.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26939,81,<599,0>,<599,81>)
                    head     728     4.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(3601,960,<156,0>,<194,34>)
           literals2ints     683     4.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(17820,263,<420,0>,<432,1>)
             newGenerate     677     4.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(1679,9138,<44,0>,<243,1>)
                 getType     415     2.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12765,52,<279,0>,<279,52>)
                 getType     406     2.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12889,47,<281,0>,<281,47>)
                  except     368     2.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Priorities.rsc|(7969,503,<175,0>,<182,80>)
                   index     349     2.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(5167,403,<221,0>,<239,57>)
                    tail     313     1.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(17714,666,<765,0>,<799,40>)
                str2syms     299     1.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Literals.rsc|(754,145,<23,0>,<26,1>)
                    same     248     1.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Priorities.rsc|(10299,71,<226,0>,<226,71>)
        computeDontNests     203     1.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(10821,1011,<245,0>,<258,1>)
                   strip     198     1.2%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(882,35,<25,0>,<25,35>)

ASTS PROFILE: 708 data points, 17007 ticks, tick = 1 milliSecs
                   Scope   Ticks        %  Source
                unsetRec    1360     8.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13441,1,<290,41>,<290,42>)
                 rewrite     865     5.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2590,7,<68,30>,<68,37>)
                striprec     661     3.9%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(577,5,<18,46>,<18,51>)
             sym2newitem     572     3.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18707,3,<457,13>,<457,16>)
                    head     569     3.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(3601,960,<156,0>,<194,34>)
           literals2ints     406     2.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18032,2,<428,26>,<428,28>)
        generateNewItems     391     2.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13736,1,<297,22>,<297,23>)
 generateCharClassArrays     336     2.0%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(26108,6,<563,31>,<563,37>)
                unsetRec     323     1.9%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12933,1,<281,44>,<281,45>)
                   index     315     1.9%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(5564,3,<239,51>,<239,54>)
                 rewrite     312     1.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(2554,1,<67,13>,<67,14>)
             sym2newitem     298     1.8%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(21620,3,<503,12>,<503,15>)
                  escape     283     1.7%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(27013,4,<599,74>,<599,78>)
             sym2newitem     253     1.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(18811,2,<462,14>,<462,16>)
                    tail     247     1.5%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/List.rsc|(17714,666,<765,0>,<799,40>)
        generateNewItems     243     1.4%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(13457,1,<292,9>,<292,10>)
                 getType     222     1.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12765,52,<279,0>,<279,52>)
             sym2newitem     215     1.3%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(23136,3,<523,83>,<523,86>)
                 getType     187     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/ParserGenerator.rsc|(12814,1,<279,49>,<279,50>)
                striprec     179     1.1%  |file:///C:/Users/Davy/swat.engineering/rascal/rascal/src/org/rascalmpl/library/lang/rascal/grammar/definition/Symbols.rsc|(625,1,<19,38>,<19,39>)
real: 25.585

This reduces it to 8%. And making it 3.6x quicker at the cost of more ugly code.

@jurgenvinju what do you think? switch to the fastest one? or keep the okay one with less code (and depending on a class that nobody else depends on in Vallang.

@DavyLandman DavyLandman reopened this Sep 7, 2022
@DavyLandman DavyLandman mentioned this issue Sep 7, 2022
Merged
@jurgenvinju
Copy link
Member

there is only one way forward. let's merge this and factor the code later

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement interpreter parser
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Made java version of unsetRec usethesource/rascal
2 participants
@DavyLandman @jurgenvinju