Skip to content

Conversation

@kaspersv
Copy link
Contributor

@kaspersv kaspersv commented Mar 2, 2023

The Core team is planning to make some changes to the join orderer that causes a join order regression for the controllerTemplateFile predicate. This PR restructures the predicate slightly to prevent the join order regression.

Old predicate with current join orderer:

Evaluated non-recursive predicate ActionController#32b59475::controllerTemplateFile#2#ff@829c7fc9 in 512ms (size: 382).
Evaluated relational algebra for predicate ActionController#32b59475::controllerTemplateFile#2#ff@829c7fc9 with tuple counts:
        8048121   ~4%    {2} r1 = JOIN locations_default_10#join_rhs WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1
           4258   ~2%    {2} r2 = JOIN r1 WITH DataFlowPublic#e1781e31::ModuleNode::getLocation#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
                     
            253   ~1%    {4} r3 = JOIN r2 WITH ActionController#32b59475::ActionControllerClass#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0, "", InverseAppend(("" ++ "app/controllers/"),"_controller.rb",Lhs.1)
            253   ~0%    {3} r4 = SCAN r3 OUTPUT In.1, "", In.3
                     
            253   ~2%    {4} r5 = JOIN r2 WITH ActionController#32b59475::ActionControllerClass#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0, "^(.*/)app/controllers/(?:.*?)/(?:[^/]*)$", 1
              0   ~0%    {6} r6 = JOIN r5 WITH PRIMITIVE regexpCapture#bbff ON Lhs.0,Lhs.2
              0   ~0%    {6} r7 = SELECT r6 ON In.4 = 1
              0   ~0%    {4} r8 = SCAN r7 OUTPUT In.0, In.1, In.5, InverseAppend((In.5 ++ "app/controllers/"),"_controller.rb",In.0)
              0   ~0%    {3} r9 = SCAN r8 OUTPUT In.1, In.2, In.3
                     
            253   ~0%    {3} r10 = r4 UNION r9
         227700   ~2%    {4} r11 = JOIN r10 WITH Erb#b2b9e6ed::ErbFile#ff CARTESIAN PRODUCT OUTPUT Rhs.0, Lhs.0, Lhs.1, Lhs.2
         227700   ~4%    {5} r12 = JOIN r11 WITH containerparent_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2, Lhs.3, Lhs.0
                     
         227700   ~3%    {6} r13 = JOIN r12 WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.3, Lhs.4, Rhs.1, (Lhs.2 ++ "app/views/" ++ Lhs.3)
            381   ~0%    {6} r14 = SELECT r13 ON In.5 = In.4
            381   ~0%    {2} r15 = SCAN r14 OUTPUT In.0, In.3
                     
         227700   ~1%    {6} r16 = JOIN r12 WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.4, Lhs.1, Lhs.2, Lhs.3, Rhs.1, (Lhs.2 ++ "app/views/layouts/" ++ Lhs.3 ++ "%")
         227700   ~0%    {7} r17 = JOIN r16 WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.3, Lhs.0, Lhs.4, Lhs.5, Rhs.1
              1   ~0%    {7} r18 = SELECT r17 ON In.6 matches In.5
              1   ~0%    {2} r19 = SCAN r18 OUTPUT In.0, In.3
                     
            382   ~0%    {2} r20 = r15 UNION r19
                         return r20

Old predicate with new join orderer:

Evaluated non-recursive predicate ActionController#32b59475::controllerTemplateFile#2#ff@f69d6du7 in 164146ms (size: 382).
Evaluated relational algebra for predicate ActionController#32b59475::controllerTemplateFile#2#ff@f69d6du7 with tuple counts:
               900   ~0%    {1} r1 = SCAN Erb#b2b9e6ed::ErbFile#ff OUTPUT In.0
               900   ~0%    {1} r2 = STREAM DEDUP r1
               900   ~0%    {2} r3 = JOIN r2 WITH containerparent_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.0
               900   ~0%    {2} r4 = JOIN r3 WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1
        7243309800   ~2%    {4} r5 = JOIN r4 WITH locations_default CARTESIAN PRODUCT OUTPUT Rhs.1, Lhs.0, Lhs.1, Rhs.0
        7243308900   ~6%    {4} r6 = JOIN r5 WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Rhs.1
           3832200   ~2%    {4} r7 = JOIN r6 WITH DataFlowPublic#e1781e31::ModuleNode::getLocation#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.2, Lhs.3
                        
            227700   ~0%    {6} r8 = JOIN r7 WITH ActionController#32b59475::ActionControllerClass#f ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.3, Lhs.0, "", InverseAppend(("" ++ "app/controllers/"),"_controller.rb",Lhs.3)
                        
            227700   ~0%    {6} r9 = SCAN r8 OUTPUT In.0, In.1, In.3, "", In.5, ("" ++ "app/views/" ++ In.5)
               381   ~0%    {6} r10 = SELECT r9 ON In.5 = In.1
               381   ~0%    {2} r11 = SCAN r10 OUTPUT In.2, In.0
                        
            227700   ~0%    {6} r12 = JOIN r7 WITH ActionController#32b59475::ActionControllerClass#f ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.3, Lhs.0, "^(.*/)app/controllers/(?:.*?)/(?:[^/]*)$", 1
                        
                 0   ~0%    {8} r13 = JOIN r12 WITH PRIMITIVE regexpCapture#bbff ON Lhs.2,Lhs.4
                 0   ~0%    {8} r14 = SELECT r13 ON In.6 = 1
                 0   ~0%    {6} r15 = SCAN r14 OUTPUT In.0, In.1, In.2, In.3, In.7, InverseAppend((In.7 ++ "app/controllers/"),"_controller.rb",In.2)
                 0   ~0%    {6} r16 = SCAN r15 OUTPUT In.0, In.1, In.3, In.4, In.5, (In.4 ++ "app/views/" ++ In.5)
                 0   ~0%    {6} r17 = SELECT r16 ON In.5 = In.1
                 0   ~0%    {2} r18 = SCAN r17 OUTPUT In.2, In.0
                        
            227700   ~0%    {6} r19 = SCAN r8 OUTPUT In.0, In.1, In.3, "", In.5, ("" ++ "app/views/layouts/" ++ In.5 ++ "%")
                        
                 0   ~0%    {8} r20 = JOIN r12 WITH PRIMITIVE regexpCapture#bbff ON Lhs.2,Lhs.4
                 0   ~0%    {8} r21 = SELECT r20 ON In.6 = 1
                 0   ~0%    {6} r22 = SCAN r21 OUTPUT In.0, In.1, In.2, In.3, In.7, InverseAppend((In.7 ++ "app/controllers/"),"_controller.rb",In.2)
                 0   ~0%    {6} r23 = SCAN r22 OUTPUT In.0, In.1, In.3, In.4, In.5, (In.4 ++ "app/views/layouts/" ++ In.5 ++ "%")
                        
            227700   ~0%    {6} r24 = r19 UNION r23
            227700   ~0%    {7} r25 = JOIN r24 WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.0, Lhs.1, Lhs.2, Lhs.3, Lhs.4, Lhs.5, Rhs.1
                 1   ~0%    {7} r26 = SELECT r25 ON In.6 matches In.5
                 1   ~0%    {2} r27 = SCAN r26 OUTPUT In.2, In.0
                        
                 1   ~0%    {2} r28 = r18 UNION r27
               382   ~0%    {2} r29 = r11 UNION r28
                            return r29

New predicate with old & new join orderer:

Evaluated non-recursive predicate ActionController#32b59475::controllerTemplateFile#2#ff@d51a5300 in 265ms (size: 382).
Evaluated relational algebra for predicate ActionController#32b59475::controllerTemplateFile#2#ff@d51a5300 with tuple counts:
        8048121   ~4%    {2} r1 = JOIN locations_default_10#join_rhs WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Rhs.1
           4258   ~2%    {2} r2 = JOIN r1 WITH DataFlowPublic#e1781e31::ModuleNode::getLocation#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
                     
            253   ~1%    {4} r3 = JOIN r2 WITH ActionController#32b59475::ActionControllerClass#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0, "", InverseAppend(("" ++ "app/controllers/"),"_controller.rb",Lhs.1)
                     
            253   ~0%    {2} r4 = SCAN r3 OUTPUT ("" ++ "app/views/" ++ In.3), In.1
                     
            253   ~2%    {4} r5 = JOIN r2 WITH ActionController#32b59475::ActionControllerClass#f ON FIRST 1 OUTPUT Lhs.1, Lhs.0, "^(.*/)app/controllers/(?:.*?)/(?:[^/]*)$", 1
              0   ~0%    {6} r6 = JOIN r5 WITH PRIMITIVE regexpCapture#bbff ON Lhs.0,Lhs.2
              0   ~0%    {6} r7 = SELECT r6 ON In.4 = 1
              0   ~0%    {4} r8 = SCAN r7 OUTPUT In.0, In.1, In.5, InverseAppend((In.5 ++ "app/controllers/"),"_controller.rb",In.0)
                     
              0   ~0%    {2} r9 = SCAN r8 OUTPUT (In.2 ++ "app/views/" ++ In.3), In.1
                     
            253   ~0%    {2} r10 = r4 UNION r9
             64   ~0%    {2} r11 = JOIN r10 WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1
            385   ~3%    {2} r12 = JOIN r11 WITH containerparent ON FIRST 1 OUTPUT Rhs.1, Lhs.1
            381   ~0%    {2} r13 = JOIN r12 WITH Erb#b2b9e6ed::ErbFile#ff ON FIRST 1 OUTPUT Lhs.1, Lhs.0
                     
            253   ~4%    {4} r14 = SCAN r3 OUTPUT In.1, "", In.3, ("" ++ "app/views/layouts/" ++ In.3 ++ "%")
                     
              0   ~0%    {4} r15 = SCAN r8 OUTPUT In.1, In.2, In.3, (In.2 ++ "app/views/layouts/" ++ In.3 ++ "%")
                     
            253   ~4%    {4} r16 = r14 UNION r15
         227700   ~0%    {5} r17 = JOIN r16 WITH Erb#b2b9e6ed::ErbFile#ff CARTESIAN PRODUCT OUTPUT Rhs.0, Lhs.0, Lhs.1, Lhs.2, Lhs.3
         227700   ~2%    {6} r18 = JOIN r17 WITH FileSystem#df18ed9a::Make#FileSystem#e91ad87f::Input#::Container::getRelativePath#0#dispred#ff ON FIRST 1 OUTPUT Lhs.1, Lhs.2, Lhs.3, Lhs.4, Lhs.0, Rhs.1
              1   ~0%    {6} r19 = SELECT r18 ON In.5 matches In.3
              1   ~0%    {2} r20 = SCAN r19 OUTPUT In.0, In.4
                     
            382   ~0%    {2} r21 = r13 UNION r20
                         return r21

@github-actions github-actions bot added the Ruby label Mar 2, 2023
@kaspersv kaspersv force-pushed the kaspersv/actioncontroller-prevent-bad-join branch 2 times, most recently from c994c3d to 6015142 Compare March 2, 2023 09:01
@kaspersv kaspersv force-pushed the kaspersv/actioncontroller-prevent-bad-join branch from 6015142 to 9cac4bb Compare March 2, 2023 09:02
@kaspersv kaspersv marked this pull request as ready for review March 3, 2023 12:32
@kaspersv kaspersv requested a review from a team as a code owner March 3, 2023 12:32
@kaspersv kaspersv requested review from RasmusWL and removed request for a team March 3, 2023 12:32
@kaspersv kaspersv added the no-change-note-required This PR does not need a change note label Mar 3, 2023
Copy link
Contributor

@alexrford alexrford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@kaspersv kaspersv merged commit fe65fb8 into github:main Mar 3, 2023
@kaspersv kaspersv deleted the kaspersv/actioncontroller-prevent-bad-join branch March 23, 2023 07:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-change-note-required This PR does not need a change note Ruby

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants