Adding the End to End flow #278
Conversation
Jatin-exe
changed the title
|
|
||
| # Create directory structure | ||
| RUN mkdir -p /output | ||
| RUN mkdir -p /workloads /outputs /workspace $RISCV |
There was a problem hiding this comment.
This line seems redundant since each folder already has its own WORKDIR command. As for the output directory, it is only used for host system binds, so there is also no need to create it in the dockerfile
| DOCKER_IMAGE_NAME = "riscv-perf-model:latest" | ||
| DOCKER_TEMP_FOLDER = "/host" | ||
| class Const: | ||
| DOCKER_IMAGE_NAME = "riscv-perf-model:olympia" |
There was a problem hiding this comment.
Using :olympia doesn’t really follow versioning convention. If you want “olympia” in the image name, it would be better placed before the colon. The part after : is typically reserved for version tags.
| - **QEMU**: Generates simple assembly traces using `-d in_asm` output | ||
|
|
||
| QEMU cannot generate STF traces - only basic assembly instruction logs. | ||
| QEMU can generate STF traces via a plugin (libstfmem), while Spike generates STF natively. |
There was a problem hiding this comment.
Spike doesn’t generate STF traces natively, we use a Condor computing fork of it
| from flow.utils.paths import BenchmarkPaths, simpoint_analysis_root | ||
| from flow.utils.util import CommandError, Util | ||
|
|
||
|
|
There was a problem hiding this comment.
I noticed the TraceGenerator class definition was removed. Was there a specific reason for this change? The original idea behind the class was to make it easier to reuse in other scripts if needed.
| single = sub.add_parser("single", help="Generate a single STF trace window") | ||
| single.add_argument("binary", help="Path to workload binary") | ||
| single.add_argument("--emulator", required=True, choices=["spike", "qemu"]) | ||
| single.add_argument("--mode", choices=["macro", "insn_count", "pc_count"], default="macro") |
There was a problem hiding this comment.
Why were the mode subparsers removed? Using subparsers helped validate arguments more easily and also made --help output more readable.
| except CommandError as err: | ||
| Util.warn(f"stf_dump failed: {err}") | ||
| return | ||
| trace_path.with_suffix(".dump").write_text(dump_result.stdout) |
There was a problem hiding this comment.
This stf_dump tool part is repeated in the run_single method
| ) -> subprocess.CompletedProcess: | ||
| docker_cmd = self._docker_prefix(workdir, interactive) + ["bash", "-lc", shlex.join(command)] | ||
| Util.info("Docker exec: " + " ".join(docker_cmd)) | ||
| result = subprocess.run(docker_cmd) |
There was a problem hiding this comment.
Is there a reason the Docker Python library was removed? I found that using it made the code easier to read and maintain.
|
|
||
|
|
||
| # output generated by the end to end flow | ||
| outputs |
There was a problem hiding this comment.
I’m not sure if this change affects anything related to the Olympia setup. To be safer, I’d suggest ignoring only the outputs inside the docker stf trace gen folder instead of applying it more broadly.
| Util.warn("stf_count not available; skipping verification") | ||
| return {"verified": None, "counted": None} | ||
| try: | ||
| result = Util.run_cmd([str(count_tool), str(trace_path)]) |
There was a problem hiding this comment.
Similar to the stf_dump tool. This stf_count should probably be called using the Docker image rather than directly from the host.
Implemented:
To do: