PeekPokeAPI: include source location on failed expect() calls.

[kivikakk]

Fixes #4128. (cc @jackkoenig)

Some notes:

• b3fca5e adds the SourceInfo collection and reporting in the exception.
  • I haven't tried stack trace collection. Other than simply not getting to it yet, I don't think I have a rigorous enough setup (or a large enough testcase, which I have a working build going for) to meaningfully assess the performance cost.
  • Minor API design question: do we ever expect testableData.expect (with expected, encode and buildMessage arguments) to be called by a test user directly? If not, I have a slight preference for making the sourceInfo argument non-implicit there, and only having it implicit in the user-facing calls on SimulationData.
• 2d58c7e adds a baseline testcase that expect() actually does anything. This isn't specific to this work.
• In f507170, I try a very simple refactor of the source line collection in ErrorLog, and use it to embellish the exception.
  • It's not very pretty — I just nyonk getErrorLineInFile over to ExceptionHelpers and mark it private[chisel3] so it doesn't become a new public API to maintain. We have to take sourceRoots as an argument.
  • I couldn't actually find a meaningful way to collect source roots at the point of calling expect() (no Builder context, etc., so I'm not sure how to get at the ChiselOptions for a given module, if it's even possible), so we just use none, which defaults to Seq(new File(".")). This works fine for Chisel's own test cases and mine in my testing, but maybe there's more complicated setups out there.
  • I also add some very primitive and somewhat brittle checks to the test case added in the previous commit.

I'm not emotionally wed to any of this, so do feel free to suggest other ways for it to be done, or to rework it yourself if that's less overall work! (Particularly thinking of the refactor here, as it's not the cleanest.)

It does work nicely, though:

[info] UARTSpec:
[info] UART
[info] - should receive a byte *** FAILED ***
[info]   chisel3.simulator.PeekPokeAPI$FailedExpectationException: Failed Expectation: Observed value '1' != 0. Expectation failed: observed value 1 != 0 @[src/test/scala/ee/hrzn/athena/uart/UARTSpec.scala 64:29]
[info]         c.io.rx.valid.expect(false.B)
[info]                             ^
[info]   at chisel3.simulator.PeekPokeAPI$testableData.$anonfun$expect$9(PeekPokeAPI.scala:157)
[info]   at chisel3.simulator.PeekPokeAPI$testableData.$anonfun$expect$9$adapted(PeekPokeAPI.scala:147)
[info]   at svsim.Simulation$Port$$anonfun$check$1.applyOrElse(Simulation.scala:434)
[info]   at svsim.Simulation$Port$$anonfun$check$1.applyOrElse(Simulation.scala:432)
[info]   at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
[info]   at svsim.Simulation$Controller.$anonfun$completeInFlightCommands$1(Simulation.scala:203)
[info]   at svsim.Simulation$Controller.$anonfun$completeInFlightCommands$1$adapted(Simulation.scala:200)
[info]   at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
[info]   at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
[info]   at scala.collection.AbstractIterable.foreach(Iterable.scala:935)
[info]   ...

Contributor Checklist

• Did you add Scaladoc to every public function/method?
• Did you add at least one test demonstrating the PR?
• Did you delete any extraneous printlns/debugging code?
• Did you specify the type of improvement?
• Did you add appropriate documentation in docs/src?
  • The only relevant documentation appears to be the "Migrating from ChiselTest" page, but there's not anything to add here really.
• Did you request a desired merge strategy?
• Did you add text to be included in the Release Notes for this change?

Type of Improvement

• Feature (or new API)
  • -ish. Simulator debugging info improvement.
• Internal or build-related (includes code refactoring/cleanup)

Desired Merge Strategy

• Rebase: You will rebase the PR onto master and it will be merged with a merge commit.

Release Notes

SimulationData.expect calls now record source location and report it in the FailedExpectationException on failure.

Reviewer Checklist (only modified by reviewer)

• Did you add the appropriate labels? (Select the most appropriate one based on the "Type of Improvement")
• Did you mark the proper milestone (Bug fix: 3.6.x, 5.x, or 6.x depending on impact, API modification or big change: 7.0)?
• Did you review?
• Did you check whether all relevant Contributor checkboxes have been checked?
• Did you do one of the following when ready to merge:
  • Squash: You/ the contributor Enable auto-merge (squash), clean up the commit message, and label with Please Merge.
  • Merge: Ensure that contributor has cleaned up their commit history, then merge with Create a merge commit.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: kivikakk / name: Asherah Connor (f4fb9af, 891ed06, f320d82, 77ebff8, 3570d52, 6dc98b7)

[jackkoenig]

This is a fantastic PR, thank you!

I have left a few minor/nitty comments, please address and I will merge!

I haven't tried stack trace collection. Other than simply not getting to it yet, I don't think I have a rigorous enough setup (or a large enough testcase, which I have a working build going for) to meaningfully assess the performance cost.

I don't see any reason to bother, this is great.

I try a very simple refactor of the source line collection in ErrorLog, and use it to embellish the exception. It's not very pretty — I just nyonk getErrorLineInFile over to ExceptionHelpers and mark it private[chisel3] so it doesn't become a new public API to maintain. We have to take sourceRoots as an argument.

I would have done the exact same thing so LGTM

I couldn't actually find a meaningful way to collect source roots at the point of calling expect() (no Builder context, etc., so I'm not sure how to get at the ChiselOptions for a given module, if it's even possible), so we just use none, which defaults to Seq(new File(".")). This works fine for Chisel's own test cases and mine in my testing, but maybe there's more complicated setups out there.

Yeah it would be nice if we could extract that information from the Builder context, but that requires a larger effort of propagating that info from Chisel elaboration to ChiselSim that's way beyond the scope of this PR. This particular issue probably also doesn't come up very often, and in those cases, they will still get the source locator even if they don't get the source line and carat.

[jackkoenig]

Note this is failing formatting which you can run with ./mill __.reformat (see bottom of CONTRIBUTING.md, file is a bit dated but that command at least is correct!).

[kivikakk]

Note this is failing formatting which you can run with ./mill __.reformat (see bottom of CONTRIBUTING.md, file is a bit dated but that command at least is correct!).

Thanks for pointing that out! I just got started with Scala a month ago or so and I've been very slowly exposing myself to the toolchain; Mill is one I've been deferring on (until now!).

All cleaned up — I went and reformatted the individual patches rather than adding a 'reformat' commit at the end — and rebased on main. Thanks for your guidance on this, I really appreciate it!

[jackkoenig]

Thanks for pointing that out! I just got started with Scala a month ago or so and I've been very slowly exposing myself to the toolchain; Mill is one I've been deferring on (until now!).

It doesn't help that we are in the process of slowly switching from SBT to mill so like 80% of things in this repo are still using SBT but a couple of things are using Mill 🙃

[jackkoenig]

Thank you!