refactor: Add parse! substitution and enable support for macros with it#1589
Open
refactor: Add parse! substitution and enable support for macros with it#1589
parse! substitution and enable support for macros with it#1589Conversation
randomPoison
reviewed
Feb 5, 2026
Contributor
randomPoison
left a comment
randomPoison
left a comment
There was a problem hiding this comment.
Seems reasonable, but would it be possible to add tests for this? Do we have a way to do tests for rewrite_expr? If you do add tests, I'd suggest having a case for a rewrite that wants to generate a parse! macro in the rewritten output, e.g. rewrite_expr 'test_func($x)' 'parse!(parse!(println!($x)))'. This PR handles that case fine, but would be good to have test coverage for that edge case.
| false, | ||
| None, | ||
| ); | ||
| *e = parser.parse_expr().expect("Failed to parse Expr"); |
Contributor
There was a problem hiding this comment.
These expect calls are going to fail if the contents of parse!() fail to parse, do we want that? The error output in that case is pretty opaque:
Output when expect fails
./target/debug/c2rust-refactor rewrite_expr 'test_func($x)' 'parse!()' -r print -- test_parse.rs 2>&1
error[E0433]: failed to resolve: maybe a missing crate `std`?
|
= help: consider adding `extern crate std` to use the `std` crate
thread 'rustc' panicked at 'Failed to parse Expr: Diagnostic { level: Error { lint: false }, message: [(Str("expected expression, found `<eof>`"), NoStyle)], code: None, span: MultiSpan { primary_spans: [Span { lo: BytePos(0), hi: BytePos(0), ctxt: #0 }], span_labels: [(Span { lo: BytePos(0), hi: BytePos(0), ctxt: #0 }, Str("expected expression"))] }, children: [], suggestions: Ok([]), args: [], sort_span: Span { lo: BytePos(0), hi: BytePos(0), ctxt: #0 }, is_lint: false }', c2rust-refactor/src/matcher/subst.rs:169:42
stack backtrace:
0: 0x73fbccb6cc60 - std::backtrace_rs::backtrace::libunwind::trace::h60bf900414c5dcf2
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
1: 0x73fbccb6cc60 - std::backtrace_rs::backtrace::trace_unsynchronized::habfc3ef92583c158
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x73fbccb6cc60 - std::sys_common::backtrace::_print_fmt::hae196fe4153d0b63
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/sys_common/backtrace.rs:66:5
3: 0x73fbccb6cc60 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h08f9c50f850514f5
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/sys_common/backtrace.rs:45:22
4: 0x73fbccbc862c - core::fmt::write::hcd15d2c673b5a9c1
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/core/src/fmt/mod.rs:1198:17
5: 0x73fbccb5dca5 - std::io::Write::write_fmt::h417be4d25b915a22
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/io/mod.rs:1672:15
6: 0x73fbccb6f941 - std::sys_common::backtrace::_print::h95ab226f37d571aa
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/sys_common/backtrace.rs:48:5
7: 0x73fbccb6f941 - std::sys_common::backtrace::print::hc77fb9ed6b2eeb3d
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/sys_common/backtrace.rs:35:9
8: 0x73fbccb6f941 - std::panicking::default_hook::{{closure}}::hca418bd67097eb22
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panicking.rs:295:22
9: 0x73fbccb6f60e - std::panicking::default_hook::h1df4f10c6ed28c53
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panicking.rs:314:9
10: 0x73fbcf3f07a4 - rustc_driver[6a784e82872844b3]::DEFAULT_HOOK::{closure#0}::{closure#0}
11: 0x73fbccb70176 - std::panicking::rust_panic_with_hook::hf1e7a1cb721a9823
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panicking.rs:702:17
12: 0x73fbccb6ffc7 - std::panicking::begin_panic_handler::{{closure}}::h573a00e95ce9340d
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panicking.rs:588:13
13: 0x73fbccb6d184 - std::sys_common::backtrace::__rust_end_short_backtrace::h597b7803117e52fd
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/sys_common/backtrace.rs:138:18
14: 0x73fbccb6fcf2 - rust_begin_unwind
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panicking.rs:584:5
15: 0x73fbccbc5103 - core::panicking::panic_fmt::h62ccf03c8a8a51b5
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/core/src/panicking.rs:142:14
16: 0x73fbccbc5463 - core::result::unwrap_failed::hff48f82f03d418ae
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/core/src/result.rs:1814:5
17: 0x57a929f32e5b - core::result::Result<T,E>::expect::h65fa60e7fba9ea94
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/core/src/result.rs:1064:23
18: 0x57a92a81147f - <c2rust_refactor::matcher::subst::SubstFolder as rustc_ast::mut_visit::MutVisitor>::visit_expr::h8eedf5173e263640
at /home/random/dev/c2rust-master/c2rust-refactor/src/matcher/subst.rs:169:22
19: 0x57a92b15a81f - <rustc_ast::ptr::P<rustc_ast::ast::Expr> as c2rust_refactor::ast_manip::fold::MutVisit>::visit::h6fcd643a397877cc
at /home/random/dev/c2rust-master/c2rust-refactor/src/ast_manip/fold.rs:74:1
20: 0x57a92a889887 - <rustc_ast::ptr::P<rustc_ast::ast::Expr> as c2rust_refactor::matcher::subst::Subst>::subst::h69614261d9225063
at /home/random/dev/c2rust-master/c2rust-refactor/src/matcher/subst.rs:315:17
21: 0x57a929f5ceae - <c2rust_refactor::transform::rewrite::RewriteExpr as c2rust_refactor::transform::Transform>::transform::{{closure}}::h3f278b0e0040fc3c
at /home/random/dev/c2rust-master/c2rust-refactor/src/transform/rewrite.rs:65:20
22: 0x57a92b30873c - <c2rust_refactor::matcher::ExprPatternFolder<F> as rustc_ast::mut_visit::MutVisitor>::visit_expr::{{closure}}::h641181d11f70738d
at /home/random/dev/c2rust-master/c2rust-refactor/src/matcher/mod.rs:741:25
23: 0x57a92b307ede - <c2rust_refactor::matcher::ExprPatternFolder<F> as rustc_ast::mut_visit::MutVisitor>::visit_expr::h01aa8eab0baf4cbd
at /home/random/dev/c2rust-master/c2rust-refactor/src/matcher/mod.rs:782:22
24: 0x57a92a4429a5 - rustc_ast::mut_visit::noop_visit_local::ha8f50fd49daa0340
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:584:13
25: 0x57a92b2f1e9f - rustc_ast::mut_visit::MutVisitor::visit_local::hbdb4dcbca598ac0d
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:209:9
26: 0x57a92a5f88ec - rustc_ast::mut_visit::noop_flat_map_stmt_kind::had88a14004e2ed9b
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1469:13
27: 0x57a92a4aa595 - rustc_ast::mut_visit::noop_flat_map_stmt::h9bb78ddd1d14dbfa
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1450:30
28: 0x57a92b2f2ed5 - rustc_ast::mut_visit::MutVisitor::flat_map_stmt::h77decd864ac77bff
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:137:9
29: 0x57a92a430382 - rustc_ast::mut_visit::noop_visit_block::{{closure}}::he725380d87fa157d
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1004:36
30: 0x57a92aaeb28f - <alloc::vec::Vec<T> as rustc_data_structures::map_in_place::MapInPlace<T>>::flat_map_in_place::hb23ffd7eb97379bb
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_data_structures/src/map_in_place.rs:34:28
31: 0x57a92a42afd0 - rustc_ast::mut_visit::noop_visit_block::h79d60e7b7ce78af2
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1004:5
32: 0x57a92b2f11af - rustc_ast::mut_visit::MutVisitor::visit_block::h140e3c9cf35212c6
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:133:9
33: 0x57a92a53dce7 - rustc_ast::mut_visit::noop_visit_item_kind::{{closure}}::h260ee374c6637f59
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1026:36
34: 0x57a92a7437a8 - rustc_ast::mut_visit::visit_opt::h5621c89aeb3f1250
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:336:9
35: 0x57a92a504399 - rustc_ast::mut_visit::noop_visit_item_kind::h179ea03cc88972cf
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1026:13
36: 0x57a92b2f52ef - rustc_ast::mut_visit::MutVisitor::visit_item_kind::h34e91188a9701f59
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:109:9
37: 0x57a92a496e76 - rustc_ast::mut_visit::noop_flat_map_item::ha0e77ae514cbce8d
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1162:5
38: 0x57a92b2f2ab9 - rustc_ast::mut_visit::MutVisitor::flat_map_item::h8f9c3cb66c4abef4
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:97:9
39: 0x57a92a43678e - rustc_ast::mut_visit::noop_visit_crate::{{closure}}::h4f9be2a9d48bc4ff
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1147:36
40: 0x57a92a9177c9 - <alloc::vec::Vec<T> as rustc_data_structures::map_in_place::MapInPlace<T>>::flat_map_in_place::h0f89811ca4a2e36e
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_data_structures/src/map_in_place.rs:34:28
41: 0x57a92a432d38 - rustc_ast::mut_visit::noop_visit_crate::h6313aa28376ff29a
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:1147:5
42: 0x57a92b2f13ef - rustc_ast::mut_visit::MutVisitor::visit_crate::h1bb1e1741e00d62d
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_ast/src/mut_visit.rs:77:9
43: 0x57a92b0827bf - <rustc_ast::ast::Crate as c2rust_refactor::ast_manip::fold::MutVisit>::visit::h583a1e83f8d23835
at /home/random/dev/c2rust-master/c2rust-refactor/src/ast_manip/fold.rs:74:1
44: 0x57a92b15bbbb - <rustc_ast::ptr::P<rustc_ast::ast::Expr> as c2rust_refactor::matcher::Pattern<rustc_ast::ptr::P<rustc_ast::ast::Expr>>>::visit::h45e5428cf8769bb1
at /home/random/dev/c2rust-master/c2rust-refactor/src/matcher/mod.rs:763:17
45: 0x57a92b306abc - c2rust_refactor::matcher::mut_visit_match_with::h325af2fa531d6ef4
at /home/random/dev/c2rust-master/c2rust-refactor/src/matcher/mod.rs:960:5
46: 0x57a92b142021 - <c2rust_refactor::transform::rewrite::RewriteExpr as c2rust_refactor::transform::Transform>::transform::h852f86a8cd4e75ff
at /home/random/dev/c2rust-master/c2rust-refactor/src/transform/rewrite.rs:58:9
47: 0x57a929ef5449 - <c2rust_refactor::transform::TransformCommand<T> as c2rust_refactor::command::Command>::run::{{closure}}::h8a78ed6581665dbe
at /home/random/dev/c2rust-master/c2rust-refactor/src/transform/mod.rs:30:17
48: 0x57a92ad82a9f - c2rust_refactor::command::RefactorState::transform_crate::{{closure}}::{{closure}}::h4e31a0d604cf3f59
at /home/random/dev/c2rust-master/c2rust-refactor/src/command.rs:453:25
49: 0x57a92b1402a1 - rustc_interface::passes::QueryContext::enter::{{closure}}::he715e4cf8cc212e2
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_interface/src/passes.rs:787:42
50: 0x57a92a1819ea - rustc_middle::ty::context::tls::enter_context::{{closure}}::h4c05403068ed6c31
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_middle/src/ty/context.rs:1929:50
51: 0x57a92a187f6f - rustc_middle::ty::context::tls::set_tlv::h5132cec29df3109b
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_middle/src/ty/context.rs:1913:9
52: 0x57a92b13635a - rustc_middle::ty::context::tls::enter_context::hdaeef61b1aa14967
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_interface/src/passes.rs:787:9
53: 0x57a92b13635a - rustc_interface::passes::QueryContext::enter::h5a3a83b046de4aab
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_interface/src/passes.rs:787:9
54: 0x57a92ac9535e - c2rust_refactor::command::RefactorState::transform_crate::{{closure}}::h1c546cab15fba3be
at /home/random/dev/c2rust-master/c2rust-refactor/src/command.rs:432:29
55: 0x57a92afd837e - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter::h815dab18296f78ce
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_interface/src/queries.rs:381:19
56: 0x57a92ac7c6b2 - c2rust_refactor::command::RefactorState::transform_crate::h56649c0d46c3ba3d
at /home/random/dev/c2rust-master/c2rust-refactor/src/command.rs:311:9
57: 0x57a929ef33a3 - <c2rust_refactor::transform::TransformCommand<T> as c2rust_refactor::command::Command>::run::hc50b90fbce7aa76d
at /home/random/dev/c2rust-master/c2rust-refactor/src/transform/mod.rs:28:9
58: 0x57a92adb83fd - c2rust_refactor::command::RefactorState::run::h6a9c6774600872a0
at /home/random/dev/c2rust-master/c2rust-refactor/src/command.rs:584:9
59: 0x57a929ecf879 - c2rust_refactor::main_impl::{{closure}}::hff24ae4e5362c2a5
at /home/random/dev/c2rust-master/c2rust-refactor/src/lib.rs:508:31
60: 0x57a92b0f8a0b - c2rust_refactor::driver::run_refactoring::{{closure}}::h1c844f2014699682
at /home/random/dev/c2rust-master/c2rust-refactor/src/driver.rs:319:9
61: 0x57a92b2ef81d - scoped_tls::ScopedKey<T>::set::h712b46649f7971ee
at /cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137:9
62: 0x57a92b0f1d73 - rustc_span::create_session_globals_then::he0afad94c76bfefb
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_span/src/lib.rs:112:5
63: 0x57a92a1459d5 - rustc_interface::util::run_in_thread_pool_with_globals::{{closure}}::h189b00c5a9872fdc
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/compiler/rustc_interface/src/util.rs:159:32
64: 0x57a92a2ba0f0 - std::sys_common::backtrace::__rust_begin_short_backtrace::h282ec3da5eeece93
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/sys_common/backtrace.rs:122:18
65: 0x57a929eaf120 - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h68752fae52119274
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/thread/mod.rs:505:17
66: 0x57a92b3f7aa4 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hfe9ed08c27fda4b0
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/core/src/panic/unwind_safe.rs:271:9
67: 0x57a92b3d9ff6 - std::panicking::try::do_call::h40083190dfca91b8
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panicking.rs:492:40
68: 0x57a92b3eb58b - __rust_try
69: 0x57a92b3d28fd - std::panicking::try::hbca6b9b433583015
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panicking.rs:456:19
70: 0x57a92b39bb44 - std::panic::catch_unwind::h83220794933ba73d
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panic.rs:137:14
71: 0x57a929eadd47 - std::thread::Builder::spawn_unchecked_::{{closure}}::h28d53dfb2344f770
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/thread/mod.rs:504:30
72: 0x57a92a02219f - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb33c802506ad0e7f
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/core/src/ops/function.rs:248:5
73: 0x73fbccb7a143 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hede4885ee64a1fac
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/alloc/src/boxed.rs:1935:9
74: 0x73fbccb7a143 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h06c745de9f474954
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/alloc/src/boxed.rs:1935:9
75: 0x73fbccb7a143 - std::sys::unix::thread::Thread::new::thread_start::h529a493f2fb191b6
at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/sys/unix/thread.rs:108:17
76: 0x73fbcc09caa4 - start_thread
at ./nptl/./nptl/pthread_create.c:447:8
77: 0x73fbcc129c6c - __GI___clone3
at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
78: 0x0 - <unknown>
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md
note: rustc 1.65.0-nightly (d394408fb 2022-08-07) running on x86_64-unknown-linux-gnu
query stack during panic:
end of query stackThough is it even possible to recover gracefully here? These visitor methods don't return Result so I'm not sure what the graceful recovery option would be.
Contributor
Author
There was a problem hiding this comment.
I think we do want to fail here, because it means the user passed in an incorrect substitution and the output Rust code would probably be incorrect anyway. But I agree that the error message should be more useful, I'll see what I can do.
kkysen
changed the title
parse! substitution and enable support for macros with it
Contributor
Author
I added a small test that adds |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds support for rewrites that emit macros, e.g.,
rewrite_expr 'printf($e)' 'parse!(println!($e))'. Without theparse!substitution, c2rust-refactor would try to parseprintln!in bindings not as a macro but as a special matcher directive.