Allow `frame-macros` to work without needing `frame-support`, `frame-system` and such being in scope.

[kianenigma]

I am playing with the idea of having a single crate, called frame, that is only docs + re-exports of the major types that you need when developing with frame, hopefully removing the need to import like 6 crates just have a shell pallet.

One blocker I have so far is that pallet macros currently make an assumption about the existence of frame-system, frame-support, codec, and type-info in Cargo.toml. In principle, I don't think this dependency is strictly needed, and end of the day the pallet macros need to have a valid path to these crates, eg in the simplest case #[pallet::pallet(<path-to-system>, <path-to-support>)], but I am wondering if there is a beter way to handle that?

Before investigating further, I should bring in this frame examples as a branch.

[bkchr]

#[pallet::pallet(<path-to-system>, <path-to-support>)]

Maybe we could add support for this and then let the new crate expose macros with the same name, but internally it just puts the respective macro with the correct paths.

[kianenigma]

I have some half baked version of this here:

diff --git a/frame/support/procedural/src/construct_runtime/mod.rs b/frame/support/procedural/src/construct_runtime/mod.rs
index 958a10e724..ff4458cd65 100644
--- a/frame/support/procedural/src/construct_runtime/mod.rs
+++ b/frame/support/procedural/src/construct_runtime/mod.rs
@@ -198,7 +198,6 @@ fn construct_runtime_intermediary_expansion(
 		expansion = quote::quote!(
 			#frame_support::tt_call! {
 				macro = [{ #pallet_path::tt_default_parts }]
-				frame_support = [{ #frame_support }]
 				~~> #frame_support::match_and_insert! {
 					target = [{ #expansion }]
 					pattern = [{ #pallet_name: #pallet_path #pallet_instance }]
@@ -642,7 +641,6 @@ fn decl_static_assertions(
 		quote! {
 			#scrate::tt_call! {
 				macro = [{ #path::tt_error_token }]
-				frame_support = [{ #scrate }]
 				~~> #scrate::assert_error_encoded_size! {
 					path = [{ #path }]
 					runtime = [{ #runtime }]
diff --git a/frame/support/procedural/src/pallet/expand/error.rs b/frame/support/procedural/src/pallet/expand/error.rs
index 376a6a9f51..3b4e183f6a 100644
--- a/frame/support/procedural/src/pallet/expand/error.rs
+++ b/frame/support/procedural/src/pallet/expand/error.rs
@@ -32,6 +32,7 @@ pub fn expand_error(def: &mut Def) -> proc_macro2::TokenStream {
 	let frame_support = &def.frame_support;
 	let frame_system = &def.frame_system;
 	let config_where_clause = &def.config.where_clause;
+	let tt_return_path = quote::quote!(#frame_support::tt_return);
 
 	let error = if let Some(error) = &def.error {
 		error
@@ -42,9 +43,8 @@ pub fn expand_error(def: &mut Def) -> proc_macro2::TokenStream {
 			macro_rules! #error_token_unique_id {
 				{
 					$caller:tt
-					frame_support = [{ $($frame_support:ident)::* }]
 				} => {
-					$($frame_support::)*tt_return! {
+					#tt_return_path! {
 						$caller
 					}
 				};
@@ -170,9 +170,8 @@ pub fn expand_error(def: &mut Def) -> proc_macro2::TokenStream {
 		macro_rules! #error_token_unique_id {
 			{
 				$caller:tt
-				frame_support = [{ $($frame_support:ident)::* }]
 			} => {
-				$($frame_support::)*tt_return! {
+				#tt_return_path! {
 					$caller
 					error = [{ #error_ident }]
 				}
diff --git a/frame/support/procedural/src/pallet/expand/genesis_config.rs b/frame/support/procedural/src/pallet/expand/genesis_config.rs
index de46afecf3..c79535b89f 100644
--- a/frame/support/procedural/src/pallet/expand/genesis_config.rs
+++ b/frame/support/procedural/src/pallet/expand/genesis_config.rs
@@ -17,6 +17,7 @@
 
 use crate::{pallet::Def, COUNTER};
 use frame_support_procedural_tools::get_doc_literals;
+use quote::ToTokens;
 use syn::{spanned::Spanned, Ident};
 
 ///
@@ -79,7 +80,7 @@ pub fn expand_genesis_config(def: &mut Def) -> proc_macro2::TokenStream {
 	let genesis_config_item =
 		&mut def.item.content.as_mut().expect("Checked by def parser").1[genesis_config.index];
 
-	let serde_crate = format!("{}::serde", frame_support);
+	let serde_crate = format!("{}::serde", frame_support.to_token_stream());
 
 	match genesis_config_item {
 		syn::Item::Enum(syn::ItemEnum { attrs, .. }) |
diff --git a/frame/support/procedural/src/pallet/expand/tt_default_parts.rs b/frame/support/procedural/src/pallet/expand/tt_default_parts.rs
index f36c765f7b..8f5029e863 100644
--- a/frame/support/procedural/src/pallet/expand/tt_default_parts.rs
+++ b/frame/support/procedural/src/pallet/expand/tt_default_parts.rs
@@ -75,6 +75,8 @@ pub fn expand_tt_default_parts(def: &mut Def) -> proc_macro2::TokenStream {
 		.any(|c| matches!(c.composite_keyword, CompositeKeyword::SlashReason(_)))
 		.then_some(quote::quote!(SlashReason,));
 
+	let scrate = &def.frame_support;
+	let tt_return_path = quote::quote!(#scrate::tt_return);
 	quote::quote!(
 		// This macro follows the conventions as laid out by the `tt-call` crate. It does not
 		// accept any arguments and simply returns the pallet parts, separated by commas, then
@@ -90,9 +92,8 @@ pub fn expand_tt_default_parts(def: &mut Def) -> proc_macro2::TokenStream {
 		macro_rules! #default_parts_unique_id {
 			{
 				$caller:tt
-				frame_support = [{ $($frame_support:ident)::* }]
 			} => {
-				$($frame_support)*::tt_return! {
+				#tt_return_path! {
 					$caller
 					tokens = [{
 						::{
diff --git a/frame/support/procedural/src/pallet/parse/composite.rs b/frame/support/procedural/src/pallet/parse/composite.rs
index 2bbfcd2e99..882b174154 100644
--- a/frame/support/procedural/src/pallet/parse/composite.rs
+++ b/frame/support/procedural/src/pallet/parse/composite.rs
@@ -91,7 +91,7 @@ impl CompositeDef {
 	pub fn try_from(
 		attr_span: proc_macro2::Span,
 		index: usize,
-		scrate: &proc_macro2::Ident,
+		scrate: &syn::Path,
 		item: &mut syn::Item,
 	) -> syn::Result<Self> {
 		let item = if let syn::Item::Enum(item) = item {
diff --git a/frame/support/procedural/src/pallet/parse/config.rs b/frame/support/procedural/src/pallet/parse/config.rs
index 6a3693a3ab..07dc3f0317 100644
--- a/frame/support/procedural/src/pallet/parse/config.rs
+++ b/frame/support/procedural/src/pallet/parse/config.rs
@@ -153,24 +153,22 @@ pub struct PalletAttr {
 	typ: PalletAttrType,
 }
 
-pub struct ConfigBoundParse(syn::Ident);
+pub struct ConfigBoundParse(syn::Path);
 
 impl syn::parse::Parse for ConfigBoundParse {
 	fn parse(input: syn::parse::ParseStream) -> syn::Result<Self> {
-		let ident = input.parse::<syn::Ident>()?;
-		input.parse::<syn::Token![::]>()?;
-		input.parse::<keyword::Config>()?;
+		let system_config_path = input.call(syn::Path::parse_mod_style)?;
 
 		if input.peek(syn::token::Lt) {
 			input.parse::<syn::AngleBracketedGenericArguments>()?;
 		}
 
-		Ok(Self(ident))
+		Ok(Self(system_config_path))
 	}
 }
 
-/// Parse for `IsType<<Sef as $ident::Config>::RuntimeEvent>` and retrieve `$ident`
-pub struct IsTypeBoundEventParse(syn::Ident);
+/// Parse for `IsType<<Sef as $ident>::RuntimeEvent>` and retrieve `$ident`
+pub struct IsTypeBoundEventParse(syn::Path);
 
 impl syn::parse::Parse for IsTypeBoundEventParse {
 	fn parse(input: syn::parse::ParseStream) -> syn::Result<Self> {
@@ -179,15 +177,13 @@ impl syn::parse::Parse for IsTypeBoundEventParse {
 		input.parse::<syn::Token![<]>()?;
 		input.parse::<syn::Token![Self]>()?;
 		input.parse::<syn::Token![as]>()?;
-		let ident = input.parse::<syn::Ident>()?;
-		input.parse::<syn::Token![::]>()?;
-		input.parse::<keyword::Config>()?;
+		let system_config_path = input.call(syn::Path::parse_mod_style)?;
 		input.parse::<syn::Token![>]>()?;
 		input.parse::<syn::Token![::]>()?;
 		input.parse::<keyword::RuntimeEvent>()?;
 		input.parse::<syn::Token![>]>()?;
 
-		Ok(Self(ident))
+		Ok(Self(system_config_path))
 	}
 }
 
@@ -225,7 +221,7 @@ impl syn::parse::Parse for FromEventParse {
 /// Check if trait_item is `type RuntimeEvent`, if so checks its bounds are those expected.
 /// (Event type is reserved type)
 fn check_event_type(
-	frame_system: &syn::Ident,
+	frame_system: &syn::Path,
 	trait_item: &syn::TraitItem,
 	trait_has_instance: bool,
 ) -> syn::Result<bool> {
@@ -237,18 +233,22 @@ fn check_event_type(
 					no generics nor where_clause";
 				return Err(syn::Error::new(trait_item.span(), msg))
 			}
-			// Check bound contains IsType and From
 
+			// Check bound contains IsType and From
 			let has_is_type_bound = type_.bounds.iter().any(|s| {
-				syn::parse2::<IsTypeBoundEventParse>(s.to_token_stream())
-					.map_or(false, |b| b.0 == *frame_system)
+				syn::parse2::<IsTypeBoundEventParse>(s.to_token_stream()).map_or(false, |b| {
+					// b.0.segments.iter().take(b.0.segments.len() - 1).collect::<Vec<_>>() ==
+					// 	frame_system.segments.iter().collect::<Vec<_>>()
+					// TODO: I don't think this check is necessary.
+					true
+				})
 			});
 
 			if !has_is_type_bound {
 				let msg = format!(
 					"Invalid `type RuntimeEvent`, associated type `RuntimeEvent` is reserved and must \
 					bound: `IsType<<Self as {}::Config>::RuntimeEvent>`",
-					frame_system,
+					frame_system.to_token_stream(),
 				);
 				return Err(syn::Error::new(type_.span(), msg))
 			}
@@ -299,7 +299,7 @@ pub fn replace_self_by_t(input: proc_macro2::TokenStream) -> proc_macro2::TokenS
 
 impl ConfigDef {
 	pub fn try_from(
-		frame_system: &syn::Ident,
+		frame_system: &syn::Path,
 		attr_span: proc_macro2::Span,
 		index: usize,
 		item: &mut syn::Item,
@@ -397,8 +397,11 @@ impl ConfigDef {
 		let disable_system_supertrait_check = attr.is_some();
 
 		let has_frame_system_supertrait = item.supertraits.iter().any(|s| {
-			syn::parse2::<ConfigBoundParse>(s.to_token_stream())
-				.map_or(false, |b| b.0 == *frame_system)
+			syn::parse2::<ConfigBoundParse>(s.to_token_stream()).map_or(false, |b| {
+				// b.0.segments.iter().take(b.0.segments.len() - 1).collect::<Vec<_>>() ==
+				// 	frame_system.segments.iter().collect::<Vec<_>>()
+				true
+			})
 		});
 
 		if !has_frame_system_supertrait && !disable_system_supertrait_check {
@@ -415,12 +418,13 @@ impl ConfigDef {
 			};
 
 			let msg = format!(
-				"Invalid pallet::trait, expected explicit `{}::Config` as supertrait, \
+				"Invalid pallet::trait, expected explicit `{}` as supertrait, \
 				found {}. \
 				(try `pub trait Config: frame_system::Config {{ ...` or \
 				`pub trait Config<I: 'static>: frame_system::Config {{ ...`). \
 				To disable this check, use `#[pallet::disable_frame_system_supertrait_check]`",
-				frame_system, found,
+				frame_system.to_token_stream(),
+				found,
 			);
 			return Err(syn::Error::new(item.span(), msg))
 		}
diff --git a/frame/support/procedural/src/pallet/parse/mod.rs b/frame/support/procedural/src/pallet/parse/mod.rs
index 4a4602964a..c4669a4291 100644
--- a/frame/support/procedural/src/pallet/parse/mod.rs
+++ b/frame/support/procedural/src/pallet/parse/mod.rs
@@ -60,8 +60,8 @@ pub struct Def {
 	pub extra_constants: Option<extra_constants::ExtraConstantsDef>,
 	pub composites: Vec<composite::CompositeDef>,
 	pub type_values: Vec<type_value::TypeValueDef>,
-	pub frame_system: syn::Ident,
-	pub frame_support: syn::Ident,
+	pub frame_system: syn::Path,
+	pub frame_support: syn::Path,
 	pub dev_mode: bool,
 }
 
@@ -98,7 +98,6 @@ impl Def {
 
 		for (index, item) in items.iter_mut().enumerate() {
 			let pallet_attr: Option<PalletAttr> = helper::take_first_item_pallet_attr(item)?;
-
 			match pallet_attr {
 				Some(PalletAttr::Config(span, with_default)) if config.is_none() =>
 					config = Some(config::ConfigDef::try_from(
@@ -429,6 +428,7 @@ mod keyword {
 
 /// Parse attributes for item in pallet module
 /// syntax must be `pallet::` (e.g. `#[pallet::config]`)
+#[derive(Debug)]
 enum PalletAttr {
 	Config(proc_macro2::Span, bool),
 	Pallet(proc_macro2::Span),
@@ -564,7 +564,7 @@ impl syn::parse::Parse for PalletAttr {
 }
 
 /// The optional weight annotation on a `#[pallet::call]` like `#[pallet::call(weight($type))]`.
-#[derive(Clone)]
+#[derive(Clone, Debug)]
 pub struct InheritedCallWeightAttr {
 	pub typename: syn::Type,
 	pub span: proc_macro2::Span,
diff --git a/frame/support/procedural/src/pallet_error.rs b/frame/support/procedural/src/pallet_error.rs
index 246a5bd4a2..051ad5bcf1 100644
--- a/frame/support/procedural/src/pallet_error.rs
+++ b/frame/support/procedural/src/pallet_error.rs
@@ -111,7 +111,7 @@ pub fn derive_pallet_error(input: proc_macro::TokenStream) -> proc_macro::TokenS
 
 fn generate_field_types(
 	field: &syn::Field,
-	scrate: &syn::Ident,
+	scrate: &syn::Path,
 ) -> syn::Result<Option<proc_macro2::TokenStream>> {
 	let attrs = &field.attrs;
 
@@ -143,7 +143,7 @@ fn generate_field_types(
 
 fn generate_variant_field_types(
 	variant: &syn::Variant,
-	scrate: &syn::Ident,
+	scrate: &syn::Path,
 ) -> syn::Result<Option<Vec<proc_macro2::TokenStream>>> {
 	let attrs = &variant.attrs;
 
diff --git a/frame/support/procedural/src/storage_alias.rs b/frame/support/procedural/src/storage_alias.rs
index b44a7ee997..66c40e46a1 100644
--- a/frame/support/procedural/src/storage_alias.rs
+++ b/frame/support/procedural/src/storage_alias.rs
@@ -224,7 +224,7 @@ impl StorageType {
 	/// Generate the actual type declaration.
 	fn generate_type_declaration(
 		&self,
-		crate_: &Ident,
+		crate_: &syn::Path,
 		storage_instance: &StorageInstance,
 		storage_name: &Ident,
 		storage_generics: Option<&SimpleGenerics>,
@@ -550,7 +550,7 @@ struct StorageInstance {
 
 /// Generate the [`StorageInstance`] for the storage alias.
 fn generate_storage_instance(
-	crate_: &Ident,
+	crate_: &syn::Path,
 	storage_name: &Ident,
 	storage_generics: Option<&SimpleGenerics>,
 	storage_where_clause: Option<&WhereClause>,
diff --git a/frame/support/procedural/src/tt_macro.rs b/frame/support/procedural/src/tt_macro.rs
index 69b5eb3d55..77ef20817f 100644
--- a/frame/support/procedural/src/tt_macro.rs
+++ b/frame/support/procedural/src/tt_macro.rs
@@ -92,7 +92,7 @@ pub fn create_tt_return_macro(input: proc_macro::TokenStream) -> proc_macro::Tok
 		macro_rules! #unique_name {
 			{
 				$caller:tt
-				$(frame_support = [{ $($frame_support:ident)::* }])?
+				$(frame_support = [{ $frame_support:path }])?
 			} => {
 				#frame_support::tt_return! {
 					$caller
diff --git a/frame/support/procedural/tools/src/lib.rs b/frame/support/procedural/tools/src/lib.rs
index 541accc8ab..f8f3fa0669 100644
--- a/frame/support/procedural/tools/src/lib.rs
+++ b/frame/support/procedural/tools/src/lib.rs
@@ -49,21 +49,40 @@ pub fn generate_crate_access(unique_id: &str, def_crate: &str) -> TokenStream {
 /// Generate the crate access for the crate using 2018 syntax.
 ///
 /// for `frame-support` output will for example be `frame_support`.
-pub fn generate_crate_access_2018(def_crate: &str) -> Result<syn::Ident, Error> {
-	match crate_name(def_crate) {
+pub fn generate_crate_access_2018(def_crate: &str) -> Result<syn::Path, Error> {
+	if let Ok(FoundCrate::Name(name)) = crate_name(&"frame") {
+		let path = format!("{}::deps::{}", name, def_crate.to_string().replace("-", "_"));
+		let path = syn::parse_str::<syn::Path>(&path)?;
+		return Ok(path)
+	}
+
+	let ident = match crate_name(def_crate) {
 		Ok(FoundCrate::Itself) => {
 			let name = def_crate.to_string().replace("-", "_");
 			Ok(syn::Ident::new(&name, Span::call_site()))
 		},
 		Ok(FoundCrate::Name(name)) => Ok(Ident::new(&name, Span::call_site())),
 		Err(e) => Err(Error::new(Span::call_site(), e)),
-	}
+	}?;
+
+	Ok(syn::Path::from(ident))
 }
 
 /// Generates the hidden includes that are required to make the macro independent from its scope.
 pub fn generate_hidden_includes(unique_id: &str, def_crate: &str) -> TokenStream {
 	let mod_name = generate_hidden_includes_mod_name(unique_id);
 
+	if let Ok(FoundCrate::Name(name)) = crate_name(&"frame") {
+		let path = format!("{}::deps::{}", name, def_crate.to_string().replace("-", "_"));
+		let path = syn::parse_str::<syn::Path>(&path).unwrap();
+		return quote::quote!(
+			#[doc(hidden)]
+			mod #mod_name {
+				pub use #path as hidden_include;
+			}
+		)
+	}
+
 	match crate_name(def_crate) {
 		Ok(FoundCrate::Itself) => quote!(),
 		Ok(FoundCrate::Name(name)) => {
diff --git a/frame/support/src/lib.rs b/frame/support/src/lib.rs
index de87b59a0b..7d4365e35e 100644
--- a/frame/support/src/lib.rs
+++ b/frame/support/src/lib.rs
@@ -1541,6 +1541,7 @@ pub mod testing_prelude {
 /// Prelude to be used alongside pallet macro, for ease of use.
 pub mod pallet_prelude {
 	pub use crate::{
+		defensive, defensive_assert,
 		dispatch::{
 			DispatchClass, DispatchError, DispatchResult, DispatchResultWithPostInfo, Parameter,
 			Pays,
@@ -1549,11 +1550,14 @@ pub mod pallet_prelude {
 		inherent::{InherentData, InherentIdentifier, ProvideInherent},
 		storage,
 		storage::{
+			bounded_btree_map::BoundedBTreeMap,
+			bounded_btree_set::BoundedBTreeSet,
 			bounded_vec::BoundedVec,
 			types::{
 				CountedStorageMap, Key as NMapKey, OptionQuery, ResultQuery, StorageDoubleMap,
 				StorageMap, StorageNMap, StorageValue, ValueQuery,
 			},
+			weak_bounded_vec::WeakBoundedVec,
 		},
 		traits::{
 			ConstU32, EnsureOrigin, GenesisBuild, Get, GetDefault, GetStorageVersion, Hooks,

I will try and make it work bare minimum in paritytech/substrate#14137, but would love someone else to try and properly backport it to master, add proper tests and so on.

What I am doing is declaring that:

if a frame called frame is in scope, don't look for eg. frame-support and instead use frame::deps::frame_support. Same for system.

This has been done in both the FRAME macros and construct_runtime.

The only last know issues is something related to derive_impl which still has a hard assumption on frame_support, namely here: https://github.com/paritytech/substrate/pull/13454/files#diff-55fa2bbcb98405e03a4311fca1d14b211001133cffa61fb0d4837be69c9e7bebR990

[sam0x17]

Just making a note here that the fix is probably:

1. Make a custom attribute macro that calls import_tokens_attr_internal directly instead of using the macro_magic provided one, and place this inside of a new proc macro crate that can be used in frame_support_procedural (I usually call this core_macros
2. In this custom macro, do the same logic you are doing in generate_crate_access_2018 to resolve the proper path to frame so we can generate a proper mm_override_path (location of macro_magic re-export).
3. Once you have mm_override_path, the rest of the macro body should just call import_tokens_attr_internal passing the quoted tokens for mm_override_path as attr and the enclosing macro's tokens as tokens. We should also do syn::parse2::<Nothing>()? on the enclosing macro's attrs just to make sure no one passes anything to it, since we are overriding whatever gets passed anyway
4. This should give us something like #[frame_import_tokens_attr] which we can then use instead of #[import_tokens_attr] in frame_support_procedural (or wherever)
5. Now we will get the usual stuff #[import_tokens_attr] would do but using your same crate resolution logic you are using elsewhere to locate the macro_magic re-export 👍🏻

An equivalent, but less efficient solution would be to write an attribute that expands to #[import_tokens_attr(mm_path)] where mm_path is based on your override logic. Luckily we don't have to do it this way as I expose all the macro_magic internals in the macro_magic_core crate, so we can just call the internal implementation directly.

[gui1117]

2. In this custom macro, do the same logic you are doing in generate_crate_access_2018 to resolve the proper path to frame so we can generate a proper mm_override_path (location of macro_magic re-export).

Isn't it too early? when #[frame_import_tokens_attr] is generting code, it doesn't know yet if derive_impl will be called from a crate importing frame or a crate importing frame_support.

So I ended up creating 2 macro, one for each path:
I have a branch here: https://github.com/thiolliere/substrate/tree/pr14137
https://github.com/paritytech/substrate/compare/kiz-frame-api...thiolliere:substrate:pr14137?expand=1
it compiles the example

[gui1117]

for macro_magic usage I think we should allow to pass along the path to macro_magic. so the call to DeriveImpl can lookup for the crate frame or frame_support or macro_magic in the Cargo.toml and give the correct pass for all the recursive macro calls along the way.

I have made more comment about how we could improve macro magic by copying the design of construct_runtime sam0x17/macro_magic#3

[sam0x17]

This is really good you have solved the central problem @thiolliere, will probably do a 0.4.x release when we get this all together