Repository navigation
Faster code generation, serialization and library expansion of large SDFGs - #2679
Open
ThrudPrimrose wants to merge 19 commits into
Open
ThrudPrimrose wants to merge 19 commits into
ThrudPrimrose wants to merge 19 commits into
Conversation
…opying it per state (cherry picked from commit d45befe)
…expression Substituting the whole dictionary into every bound walks the expression once per entry; an SDFG specialized over 2878 offset symbols spent 575 ms per subset. Entries are kept when their key names a symbol of the expression or of a value an already kept entry substitutes in, so chained substitutions resolve as before (3.7 ms per subset). (cherry picked from commit 02c40f5)
…ximate (cherry picked from commit e442f53)
expand_library_nodes asked defined_symbols of the same state once per expanded node, and update_cfg_list rebuilt its member check as a list per nested SDFG. (cherry picked from commit 797d492)
(cherry picked from commit 6eb7fff)
…e per nested SDFG It recursed into itself, and every level reset the whole tree again, so a call cost one full rebuild per nested SDFG. The recursion now only points each nested SDFG at its parent, after the single rebuild at the top. (cherry picked from commit 8825be9, code only) (cherry picked from commit cc942e8)
The recursive generator resumed once per nesting level for every region it yielded, and reset_cfg_list runs this walk after most graph edits: 55M generator calls, 315 s of 1008 s (cProfile) on lulesh's map lowering. Same regions, same order; the test pins it to the recursion. (cherry picked from commit 95f5513) (cherry picked from commit 4a9a007)
SubgraphView sorted its members by node_id, a linear scan per member, so a scope subgraph cost members times graph size. For an OrderedDiGraph node_id is the position in nodes(), so filtering nodes() gives the same order in one pass. sw4 rhs4sg canonicalize: SubgraphView.__init__ 28.5 s -> 1.3 s CPU over 22294 views, same pass digest. (cherry picked from commit dcb8bf3) (cherry picked from commit 0f10523)
… it per name
`_ensure_unique_block_name` unioned `_labels` with `{s.label for s in self.nodes()}`
on every call, so naming one block walked every block in the region -- quadratic in
blocks per region, and every `add_state` pays it. The walk added nothing that could
not be recorded when the name arrives: `add_node` now records the label a block
enters with and the `ControlFlowBlock.label` setter records a rename in place, which
are the only two ways a name becomes live in a region (a branch appended to a
`ConditionalBlock` bypasses `add_node`, so `add_branch` records its own). A removal
still leaves its name in the ledger, as before. Membership only -- the set is never
iterated, so nothing order-dependent rides on it.
HPCAgent-Bench cloudsc parse, `to_sdfg(simplify=False)`, 4426 states / 1183 mapped
tasklets / 2271 descriptors, three alternating rounds on a contended box:
before 29.9 / 39.0 / 37.3 CPU s, after 26.1 / 32.3 / 34.1 CPU s -- mean 35.4 -> 30.8,
a 13% cut. Same work: 4484 `_ensure_unique_block_name` calls in both arms, and the
parsed SDFG's JSON is byte-identical with `guid`/`debuginfo` stripped on cloudsc plus
tsvc s3110, polybench 3mm and npbench hdiff.
(cherry picked from commit bae19b7)
(cherry picked from commit 71e9c61)
Serializing an edge looks up both endpoints with node_id, a linear scan, so saving a large SDFG (e.g. the invalid-SDFG dump of h_psi) was quadratic. (cherry picked from commit 9755b9e)
Every container needing a common (post)dominator recomputed the whole SDFG's dominators and their transitive closure; QE h_psi's code generation spent over two hours there. One tree per SDFG answers dominance from DFS intervals. (cherry picked from commit d102614)
…ics, and trim their comments
ThrudPrimrose
added a commit
that referenced
this pull request
Oct 12, 2026
Take the PR's dominator trees, ContextVar node-id index and cfg_list short-circuit over extended's earlier copies of the same work; keep extended's CodegenAnalysisPipeline, states() iteration, the expand_library_nodes predicate and its defined-symbol typing test. Drop the duplicated nsdfg_argument_symbols method and cache field the textual merge produced.
The symbol-side optimizations leave this PR: memoized assignment-RHS names, the per-call interstate type environment in defined_symbols, the library-expansion defined-symbols memo, the nested-SDFG argument-symbol cache and the in-place symbol table in SDFGState.to_json. Kept: dominator trees built once per SDFG for allocation lifetimes, Range.replace handing SymPy only reachable substitutions, the frozen node_id index (now entered by region serialization), the block-name ledger, the explicit-stack region walk, the linear cfg-list merge, call-site debug info, subgraph-view ordering and history-free lowering.
ThrudPrimrose
marked this pull request as ready for review
October 13, 2026 13:58
ThrudPrimrose
added a commit
that referenced
this pull request
Oct 14, 2026
Extended already carries the full codegen, serialization and expansion speedups #2679 splits out for main.
Collaborator
Author
|
I removed the symbol-related optimizations I think the rest are still good to have as optimizations for big Fortran SDFGs. |
ThrudPrimrose
added a commit
that referenced
this pull request
Oct 16, 2026
Files owned by open PRs (#2443, #2677, #2679, #2680, #2690, #2691, #2693, #2696) now equal main plus that PR's own diff; map fusion equals main plus #2696. Removed: DemoteNonStructuralSymbols with tasklet_utils, narrowing, optionals and the utils demote helpers (unused by dace-fortran); ShortLoopUnroll and MakeTransientsPersistent (move to dace-fortran); use_authority serialization; new-libnodes CI (general CI runs those tests).
ThrudPrimrose
added a commit
that referenced
this pull request
Oct 16, 2026
Taking PR #2679's state.py dropped two FaCe fixes: SymbolResolver keeps only the last state per SDFG (one entry per state costs gigabytes on QE h_psi), and remove_node bounds-checks a deserialized start block index.
new_symbols(sdfg, ...) copied the symbols and every array dtype per assigning edge: 2.4 s per call on warpx_field_gather.
ThrudPrimrose
added a commit
that referenced
this pull request
Oct 16, 2026
ThrudPrimrose
added a commit
that referenced
this pull request
Oct 16, 2026
# Conflicts: # dace/codegen/targets/framecode.py # tests/codegen/allocation_lifetime_test.py
ThrudPrimrose
commented
Oct 20, 2026
|
|
||
|
|
||
| def _get_dominator_and_postdominator(sdfg: SDFG, accesses: list[tuple[SDFGState, nodes.AccessNode]]): | ||
| class DominatorTree: |
Collaborator
Author
There was a problem hiding this comment.
This both fixes allocations as well as improves speed by not recomputing the dominance all the time
ThrudPrimrose
commented
Oct 20, 2026
|
|
||
|
|
||
| @contextmanager | ||
| def frozen_node_ids() -> Iterator[None]: |
Collaborator
Author
There was a problem hiding this comment.
Computing node ids is kind of expensive that is why I'm trying to cache them, helps a lot
ThrudPrimrose
commented
Oct 20, 2026
| if dace.Config.get("compiler", "lineinfo") == "inspect": | ||
| caller = inspect.getframeinfo(inspect.stack()[2][0], context=0) | ||
| return dtypes.DebugInfo(caller.lineno, 0, caller.lineno, 0, caller.filename) | ||
| lineno, filename = caller_position(sys._getframe(2)) |
Collaborator
Author
There was a problem hiding this comment.
_getframe is much faster getting the full stack all the time.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Code generation, saving and library expansion of large SDFGs repeat whole-graph walks per element: a linear node_id scan per serialized edge, a dominator closure per allocated container, the full substitution dictionary handed to SymPy per range bound, a block-name ledger rebuild per new block, a deep copy of the SDFG into the history on the first expansion, and a recursive generator per nesting level in all_control_flow_regions. This computes each of those once per span or keeps it incrementally, without changing the generated code; symbol-side caching is left to the symbol registry (#2673).