Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Substantial slowdown during resizer invocation post-grt #4121

Closed
donn opened this issue Oct 11, 2023 · 14 comments · Fixed by #4146
Closed

Substantial slowdown during resizer invocation post-grt #4121

donn opened this issue Oct 11, 2023 · 14 comments · Fixed by #4146
Assignees
@maliberty
Labels
grt Global Routing rsz Resizer

Comments

@donn
Copy link
Contributor

donn commented Oct 11, 2023 •
edited
Loading

Describe the bug

CI printed 2023-10-11T00:27:13.3062451Z [INFO RSZ-0058] Using max wire length 4459um. then timed out

Expected Behavior

Resizer step should pass

Environment

fatal: unable to access 'https://github.com/The-OpenROAD-Project/OpenROAD.git/': Failed to connect to github.com port 443 after 99 ms: Couldn't connect to server

bdc8e94

To Reproduce

https://drive.google.com/file/d/1Q1IQOyUUbxX26my9kEVG9M7-MK7qs4zt/view?usp=sharing

Untar then run ./run.sh

Relevant log output

2023-10-11T00:26:53.8248685Z Reading timing models for corner nom_tt_025C_1v80…
2023-10-11T00:26:53.8249714Z Reading timing library for the 'nom_tt_025C_1v80' corner at 
2023-10-11T00:26:53.8250863Z '/home/runner/.volare/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__tt_0
2023-10-11T00:26:53.8302204Z 25C_1v80.lib'…
2023-10-11T00:26:54.1498139Z Reading timing models for corner nom_ff_n40C_1v95…
2023-10-11T00:26:54.1498723Z Reading timing library for the 'nom_ff_n40C_1v95' corner at 
2023-10-11T00:26:54.1499276Z '/home/runner/.volare/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__ff_n
2023-10-11T00:26:54.1499636Z 40C_1v95.lib'…
2023-10-11T00:26:54.1826957Z [WARNING STA-0122] message limit reached, this message will no longer print
2023-10-11T00:26:55.1298482Z Reading timing models for corner nom_ss_100C_1v60…
2023-10-11T00:26:55.1299091Z Reading timing library for the 'nom_ss_100C_1v60' corner at 
2023-10-11T00:26:55.1299691Z '/home/runner/.volare/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__ss_1
2023-10-11T00:26:55.1300086Z 00C_1v60.lib'…
2023-10-11T00:26:55.4572690Z Reading OpenROAD database at 
2023-10-11T00:26:55.4573577Z '/home/runner/work/openlane2/openlane2/test/designs/aes_core/runs/CI/29-openroad
2023-10-11T00:26:55.4574115Z -globalrouting/aes_core.odb'…
2023-10-11T00:26:55.6611272Z Reading design constraints file at 
2023-10-11T00:26:55.6612464Z '/home/runner/work/openlane2/openlane2/openlane/scripts/base.sdc'…
2023-10-11T00:26:55.6612878Z [INFO] Using clock clk…
2023-10-11T00:26:55.6613202Z [INFO] Setting output delay to: 4
2023-10-11T00:26:55.6613540Z [INFO] Setting input delay to: 4
2023-10-11T00:26:55.6709048Z [INFO] Setting load to: 0.033442
2023-10-11T00:26:55.6716757Z [INFO] Setting clock uncertainty to: 0.25
2023-10-11T00:26:55.6733283Z [INFO] Setting clock transition to: 
2023-10-11T00:26:55.6733650Z 0.1499999999999999944488848768742172978818416595458984375
2023-10-11T00:26:55.6734275Z [INFO] Setting timing derate to: 5%
2023-10-11T00:26:55.8262708Z [INFO] Setting RC values…
2023-10-11T00:26:55.8282441Z [INFO] Setting signal min routing layer to: met1 and clock min routing layer to 
2023-10-11T00:26:55.8282880Z met1.
2023-10-11T00:26:55.8283267Z [INFO] Setting signal max routing layer to: met5 and clock max routing layer to 
2023-10-11T00:26:55.8283652Z met5.
2023-10-11T00:26:55.8335191Z -congestion_iterations 50 -verbose
2023-10-11T00:26:55.8335662Z [INFO GRT-0020] Min routing layer: met1
2023-10-11T00:26:55.8336079Z [INFO GRT-0021] Max routing layer: met5
2023-10-11T00:26:55.8336486Z [INFO GRT-0022] Global adjustment: 30%
2023-10-11T00:26:55.8336882Z [INFO GRT-0023] Grid origin: (0, 0)
2023-10-11T00:26:55.8337286Z [INFO GRT-0043] No OR_DEFAULT vias defined.
2023-10-11T00:26:55.8338182Z [INFO GRT-0088] Layer li1     Track-Pitch = 0.4600  line-2-Via Pitch: 0.3400
2023-10-11T00:26:55.8338755Z [INFO GRT-0088] Layer met1    Track-Pitch = 0.3400  line-2-Via Pitch: 0.3400
2023-10-11T00:26:55.8339328Z [INFO GRT-0088] Layer met2    Track-Pitch = 0.4600  line-2-Via Pitch: 0.3500
2023-10-11T00:26:55.8345760Z [INFO GRT-0088] Layer met3    Track-Pitch = 0.6800  line-2-Via Pitch: 0.6150
2023-10-11T00:26:55.8354182Z [INFO GRT-0088] Layer met4    Track-Pitch = 0.9200  line-2-Via Pitch: 1.0400
2023-10-11T00:26:55.8361781Z [INFO GRT-0088] Layer met5    Track-Pitch = 3.4000  line-2-Via Pitch: 3.1100
2023-10-11T00:26:56.9477247Z [INFO GRT-0019] Found 576 clock nets.
2023-10-11T00:27:00.8664765Z [INFO GRT-0001] Minimum degree: 2
2023-10-11T00:27:00.8665642Z [INFO GRT-0002] Maximum degree: 20
2023-10-11T00:27:00.9449119Z [INFO GRT-0003] Macros: 0
2023-10-11T00:27:00.9593866Z [INFO GRT-0004] Blockages: 38963
2023-10-11T00:27:00.9656198Z 
2023-10-11T00:27:00.9664523Z [INFO GRT-0053] Routing resources analysis:
2023-10-11T00:27:00.9670718Z Routing      Original      Derated      Resource
2023-10-11T00:27:00.9677599Z Layer     Direction    Resources     Resources    Reduction (%)
2023-10-11T00:27:00.9683651Z ---------------------------------------------------------------
2023-10-11T00:27:00.9690656Z li1        Vertical            0             0          0.00%
2023-10-11T00:27:00.9697814Z met1       Horizontal     360523        174346          51.64%
2023-10-11T00:27:00.9704657Z met2       Vertical       271245        179015          34.00%
2023-10-11T00:27:00.9711486Z met3       Horizontal     180195        124532          30.89%
2023-10-11T00:27:00.9718331Z met4       Vertical       108822         69054          36.54%
2023-10-11T00:27:00.9725142Z met5       Horizontal      36039         17292          52.02%
2023-10-11T00:27:00.9731400Z ---------------------------------------------------------------
2023-10-11T00:27:00.9736117Z 
2023-10-11T00:27:01.6847862Z [INFO GRT-0101] Running extra iterations to remove overflow.
2023-10-11T00:27:10.2595511Z [INFO GRT-0197] Via related to pin nodes: 134177
2023-10-11T00:27:10.2596508Z [INFO GRT-0198] Via related Steiner nodes: 5658
2023-10-11T00:27:10.2597221Z [INFO GRT-0199] Via filling finished.
2023-10-11T00:27:10.2704589Z [INFO GRT-0111] Final number of vias: 157807
2023-10-11T00:27:10.2712577Z [INFO GRT-0112] Final usage 3D: 728190
2023-10-11T00:27:10.5532440Z 
2023-10-11T00:27:10.5533632Z [INFO GRT-0096] Final congestion report:
2023-10-11T00:27:10.5534408Z Layer         Resource        Demand        Usage (%)    Max H / Max V / Total 
2023-10-11T00:27:10.5534973Z Overflow
2023-10-11T00:27:10.5547950Z --------------------------------------------------------------------------------
2023-10-11T00:27:10.5548557Z -------
2023-10-11T00:27:10.5558798Z li1                  0             0            0.00%             0 /  0 /  0
2023-10-11T00:27:10.5568349Z met1            174346        101785           58.38%             0 /  0 /  0
2023-10-11T00:27:10.5578414Z met2            179015        106772           59.64%             0 /  0 /  0
2023-10-11T00:27:10.5588019Z met3            124532         22954           18.43%             0 /  0 /  0
2023-10-11T00:27:10.5607661Z met4             69054         23133           33.50%             0 /  0 /  0
2023-10-11T00:27:10.5616451Z met5             17292           125            0.72%             0 /  0 /  0
2023-10-11T00:27:10.5625486Z --------------------------------------------------------------------------------
2023-10-11T00:27:10.5626050Z -------
2023-10-11T00:27:10.5635614Z Total           564239        254769           45.15%             0 /  0 /  0
2023-10-11T00:27:10.5640459Z 
2023-10-11T00:27:10.5647511Z [INFO GRT-0018] Total wirelength: 2351271 um
2023-10-11T00:27:10.5654583Z [INFO GRT-0014] Routed nets: 24278
2023-10-11T00:27:13.3062451Z [INFO RSZ-0058] Using max wire length 4459um.
2023-10-11T03:19:31.5856752Z ##[error]The operation was canceled.

Screenshots

No response

Additional Context

No response
@maliberty maliberty self-assigned this Oct 11, 2023
@maliberty maliberty added the rsz Resizer label Oct 11, 2023
@maliberty
Copy link
Member

Is this something that used to work? We've always considered post-grt rsz to be in development. It is not enabled in ORFS.

@maliberty
Copy link
Member

If I add -verbose I can see it is progressing slowly.

@maliberty maliberty added the waiting on op Information has been requested from the Issue Author label Oct 11, 2023
@donn
Copy link
Contributor Author

donn commented Oct 12, 2023

We've had post-GRT resizing for as long as I remember, and it worked fine as recently as 0a6d0fd

@donn donn mentioned this issue Oct 15, 2023
Merged
@donn
Copy link
Contributor Author

donn commented Oct 17, 2023 •
edited
Loading

Is there any timeframe for an investigation into this? It's been a very useful feature; we didn't know it was in development and we've been using the resizer post-GRT since August 2021 The-OpenROAD-Project/OpenLane#531

@maliberty
Copy link
Member

Its in the queue but I haven't started to look at it yet.

@maliberty
Copy link
Member

The older version does run faster. It is a substantial slowdown rather than a hang from what I see.

The window is two months wide. Do you have any narrower scope of the problem? There have been a lot of a lot changes in that time.

@maliberty maliberty changed the title Hang during resizer invocation Substantial slowdown during resizer invocation post-grt Oct 17, 2023
@maliberty
Copy link
Member

@luis201420 @eder-matheus any guesses on the source of this slowdown?

@maliberty maliberty added grt Global Routing and removed waiting on op Information has been requested from the Issue Author labels Oct 17, 2023
@maliberty
Copy link
Member

git bisect shows:
[0817393] set the critical nets percentage on global route as 5%

@arthurjolo thoughts on the performance implications of this change?

@maliberty
Copy link
Member

I verified that if I change the default critical_nets_percentage_ to 0 at the head of master then it runs much faster.

@arthurjolo
Copy link
Contributor

git bisect shows: [0817393] set the critical nets percentage on global route as 5%

@arthurjolo thoughts on the performance implications of this change?

If the design runs a large number of iteration to remove overflow the run time should have a large increase, since it is updating the slack for most of the nets.

@maliberty
Copy link
Member

@arthurjolo this is happening from inside repair_design

@arthurjolo
Copy link
Contributor

@maliberty I haven't seen much of repair_design, I will take a look into it.

@maliberty
Copy link
Member

repair_design will make some netlist edits and then call for updated parasitics (Resizer::ensureWireParasitic). Since we are post-grt that will call for incremental global routing on the modified nets. In that tight inner loop I don't think we want to any timing updates.

@openroad-robot openroad-robot mentioned this issue Oct 17, 2023
Merged
openroad-robot pushed a commit to The-OpenROAD-Project-staging/OpenROAD that referenced this issue Oct 17, 2023
@maliberty
grt: set critical_nets_percentage to 0 during incremental grt
b54bb9e 
Fixes The-OpenROAD-Project#4121

Signed-off-by: Matt Liberty <mliberty@precisioninno.com>
openroad-robot pushed a commit to The-OpenROAD-Project-staging/OpenROAD that referenced this issue Oct 17, 2023
@maliberty
grt: set critical_nets_percentage to 0 during incremental grt
f148646 
Fixes The-OpenROAD-Project#4121

Signed-off-by: Matt Liberty <mliberty@precisioninno.com>
@donn
Copy link
Contributor Author

donn commented Oct 18, 2023

Thank you so much for the fast turnaround on this- appreciated

donn added a commit to efabless/openlane2 that referenced this issue Oct 25, 2023
@donn
Fix Failing Designs (#216)
c2780d6 
* Update OpenROAD to `0a584d1`
  * Fixes issue where post-GRT resizing run-time and memory consumption got out of hand: see The-OpenROAD-Project/OpenROAD#4121 (comment) for one example
* Disabled latch linting for `salsa20`
* Enable post-GRT resizer for `aes_core`, `spm` and `aes`
* Added hidden debug options `--force-design-dir`, `--force-run-dir`
@arthurjolo arthurjolo mentioned this issue Nov 6, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@maliberty maliberty

Labels
grt Global Routing rsz Resizer
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

grt: set critical_nets_percentage to 0 during incremental grt The-OpenROAD-Project-staging/OpenROAD
3 participants
@maliberty @donn @arthurjolo