Skip to content

Tracking Issue for porting run-make tests to use Rust #121876

New issue
New issue
Closed
#136581
Closed
Tracking Issue for porting run-make tests to use Rust#121876
#136581
Assignees
jieyouxu
Labels
A-run-makeArea: port run-make Makefiles to rmake.rsA-testsuiteArea: The testsuite used to check the correctness of rustcC-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFCT-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
@jieyouxu

Description

@jieyouxu
jieyouxu
opened on Mar 1, 2024

We want to stop run-make tests from relying on make, and improve the run-make tests so that
they are more accessible to rustc contributors by allowing the tests to be written in Rust (see
#40713 for context). PR #113026 was merged to address this, and now we are able to write run-make
tests in Rust recipes. We would like your help to port over existing run-make tests still using
Makefiles to use Rust recipes instead.

If you would like to work on porting one of the run-make tests, please link to this issue in your
PR and leave a comment to claim the test (or multiple tests). If you are stuck, please don't hesitate to open a thread
on Rust's Zulip.

run-make is the most flexible fallback test kind, but some run-make tests could be migrated to become e.g. ui tests instead. Please check if you can convert the run-make test into other test kinds before porting!

When you try to port a test, also consider:

  • Add some comments on what the test is trying to test.
  • Add some comments on how the test tries to accomplish its goals if the test is non-trivial. This is especially true if you discovered that a test has platform/compiler/architecture/tooling-specific behavior that is a pain to debug.
  • Any related issues? Any relevant links?
  • Is the test still applicable? Has it become outdated? Is it duplicated?
  • Can it be written in other test suites?

Context: Rust recipes?

PR #113026 adds basic infrastructure support to write run-make tests using small Rust programs,
called recipes.

We aim to eliminate the dependency on make and Makefiles for building run-make-style tests.
Makefiles are replaced by recipes (rmake.rs). The PR implements running run-make recipes in
3 steps:

  1. We build the support library run_make_support which the rmake.rs recipes depend on as a tool
    lib.
  2. We build the recipe rmake.rs and link in the support library.
  3. We run the recipe to build and run the tests.

rmake.rs is basically a replacement for Makefile, and allows running arbitrary Rust code. The
support library is built using cargo, and so can depend on external crates if desired.

The infrastructure implemented by the PR is very barebones, and is the minimally required
infrastructure needed to build, run and pass the two example run-make tests ported over to the new
infrastructure:

You likely will find that you would need to improve the API of the support library, and extend
the functionality of the support library.

Common traps and pitfalls, and tips and tricks

  • $PATH uses ; on Windows and : for *nixes. Use std::env::{join,split}_paths to properly
    handle $PATH.

  • Be careful of path separator platform differences. Always prefer PathBuf operations not string
    paths if possible.

  • tests/ are not (currently) formatted by rustfmt.

  • Consult https://github.com/rust-lang/rust/blob/master/tests/run-make/tools.mk for which flags
    and envs are passed to various executables or libraries. May have to triple check on those.

  • You can always request your reviewer to run try jobs to test out your PR on environments you don't locally have access to. Good candidate CI jobs include:

    • aarch64-apple (apple, 64 bits, aarch64/arm64)
    • armhf-gnu (cross-compile)
    • test-various (cross-compile, wasm)
    • x86_64-mingw (x86_64, 64 bits, windows, mingw)
    • x86_64-msvc (x86_64, 64 bits, windows, msvc)
    • x86_64-gnu-llvm-18 (x86_64, 64 bits, windows, gnu, llvm 18)
    • i686-msvc (x86, 32 bits, windows, msvc)

Tests that need porting

Metadata

Metadata

Assignees

Labels

A-run-makeArea: port run-make Makefiles to rmake.rsA-testsuiteArea: The testsuite used to check the correctness of rustcC-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFCT-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions