Tracking issue for concat_idents

[aturon]

Tracks stabilization for the concat_idents macro.

Update(fmease): Please see #124225 (comment) (an alternative to this feature).

[retep998]

concat_idents is so fundamentally useless at the moment because macros cannot be used in ident positions.

[ghost]

Yeah it is pretty much useless right now (see #12249 #13294).

[nrc]

Relevant blog for future plans: http://www.ncameron.org/blog/untitledconcat_idents-and-macros-in-ident-position/

[eddyb]

A solution that has been thrown around on IRC (but never ended up anywhere else AFAIK):
Have a macro invocation form for "early expansion", i.e. the following would work without macros in ident position:

macro_rules! wrap {
    ($name:ident) => { struct concat_idents!!(Wrapped, $name)($name); }
}

The "early expansion" macro_name!!(args) syntax was originally $*macro_name!(args) and is a subject of bikeshed.

If we move towards having all macros produce token streams that are parsed on-demand, such "early expansion" could be used in many more locations without adding macro support for each one.
Generic parameters, ADT fields, function signatures and match arms come to mind - there are so many recursive push-down hacks macros can end up with, just to construct lists of things and whatnot, that would simply not be necessary with "early expansion".

The only disadvantage is that concat_idents!! and friends would only work inside the RHS of macro_rules!, but I don't really see why you would need to use such a capability outside of a macro.

[skade]

https://github.com/mikkyang/rust-blas uses it for quick notation of FFI functions that follow a certain scheme: https://github.com/mikkyang/rust-blas/blob/master/src/prefix.rs https://github.com/mikkyang/rust-blas/blob/master/src/matrix/ll.rs

[skade]

Well, given that mikkyang/rust-blas#12 is a nicer solution within the current language, I'd also like to raise my hand for "useless".

[nrc]

I think we should never stabilise the current version - as others have noted it is useless.

[aidanhs]

Since this is the tracking issue, other who stumble across it might be interested in https://github.com/SkylerLipthay/interpolate_idents which works on nightly rust.

[retep998]

Unfortunately that solution will always depend on nightly Rust (until the day that syntax extensions become stable and pigs fly).

[skade]

Given that the consensus seems to be "useless" and theres nicer solutions (e.g. [1], maybe we can remove this feature?

[1] https://github.com/mikkyang/rust-blas/pull/12/files

[eddyb]

See rust-lang/rfcs#1628.

[aidanhs]

@retep998 yes, just noting it as a stopgap.

@skade it's the implementation rather than the idea that's useless. If concat_idents is removed rather than fixed, something else needs to fill its place (your solution doesn't work for all use-cases). I like the look of the rfc @eddyb linked, will follow that.

[skade]

@aidanhs that still means that we should drop the feature to make sure no one uses it.

I agree that my solution doesn't cover all edge-cases, but all uses that I've currently seen in the wild. Also, but this is outside of my competence to decide, I think this is a perfect case for the use of a code generator.

[aidanhs]

@skade afaict nobody can use it, so that's not a big concern...but equally there's no reason to keep it around. My main motivation for leaving it was as a reminder that people do want the functionality, but I'm content to follow the RFC and have changed my position to "I don't mind either way".

Regarding use cases, here's the motivating example that brought me to this issue in the first place. Codegen's awesome for things like phf, but it feels a bit like overkill when I just want to generate some repetitive methods to extract tuples from enum variants. I guess it's down to personal preference.

[ExpHP]

An amusing fact about the current implementation of concat_idents! is that it will accept an empty argument list, making it possible to construct the empty string as an identifier:

error[E0425]: cannot find value `` in this scope
 --> src/main.rs:5:5
  |
5 |     concat_idents!();
  |     ^^^^^^^^^^^^^^^^^ not found in this scope

(good luck actually doing anything with it)

[petrochenkov]

construct the empty string as an identifier

Empty string is used as a reserved identifier with special meaning in several places in the compiler, so this can potentially cause issues.

[jonhoo]

The linked RFC has been closed as postponed. Should concat_idents! now be removed? Is there any chance of seeing something in its place that supports generating identifiers?

[durka]

The RFC should be reopened. It never should have been closed, as no better solution was proposed.

[skade]

@durka I don't agree. There's ample reason to expect a new one when moving towards macros 2.0. RFCs are not necessarily closed to be replaced immediately, they are closed to stop following that line of discussion. Future RFCs might refer to it.

[alexreg]

I'm with @durka on this.

[mimoo]

Is there a way to use this but with a type ty? Perhaps by extracting the ident from ty? But I can't find a way to do that in a macro_rules.

[ExpHP]

Once a macro_rules! macro parses a token as anything other than tt or ident it becomes completely opaque. If you need to do something more complicated like get the Foo from Foo<A, B> then you should consider writing a proc macro and using the syn and quote crates.

(or if it's a macro for internal use, you can always settle for some subset of type syntax that's easier to parse, e.g. $name:ident<$($param:ty),*>)

[joshtriplett]

If we were to keep/stabilize this, it would need to work in the fashion people expect it to work (in place of an identifier).

[reitermarkus]

I'd like for this to also support concatenating with integer literals, e.g.

const MY_CONST1: u32 = 1;
const MY_CONST2: u32 = 2;

concat_idents!(MY_CONST, 2)

[AlexisTM]

This is especially useful for embedded when we have many names we do not have control over. (register names) where most of the difference is a number.

[dvdsk]

It would be nice if we could use concat_idents! would optionally accept self. as first argument. This would allow accessing member variables and functions.

[reitermarkus]

@dvdsk, I think this is already covered by #29599 (comment). If the macro works in ident positions, I assume you can simply use e.g. self.concat_idents!($field, 0).

[dvdsk]

@dvdsk, I think this is already covered by #29599 (comment). If the macro works in ident positions, I assume you can simply use e.g. self.concat_idents!($field, 0).

Ah I thought 'ident positions' referres to things like fn concat_idents!(macro_args)(function args) { ... }. Of course self can also be followed by an ident so its also an ident position.

Reading the rust reference the dot symbol (.) can currently only be followed by Field access or Tuple index. Not by a macro invocation. I think macro invocations need to be preceded by whitespace, I am no expect on Rust syntax however.

On a separate note self.macro might be confusing, one might think that syntax means calling a member macro? I would propose concat_idents!(self, test) resolving to self.test.

Finally self is not an ident but a keyword. So strictly speaking concat_idents should not cover it. So maybe we should drop concat_idents! and add a concat macro or look at #111930.

[tgross35]

Three comments:

1. What are the problems with stabilizing this as-is and later adding the expected features? This does not seem to conflict anywhere.
2. Are there problems with special casing this macro to allow only its use in the identifier position? Even only within other macros, so that the relationship between macro_rules! and concat_idents! is about the same as paste! and paste's [<...>] syntax. Obviously this would not be trivial to implement...
3. This should be able to unstringify string literals, concat_idents!(foo, "bar") -> foobar (as paste does), for macros where it is logical for a user to specify a string but an identifier needs to be created from it

[sylv256]

From the docs:

Also, as a general rule, macros are only allowed in item, statement or expression position. That means while you may use this macro for referring to existing variables, functions or modules etc, you cannot define a new one with it.

Is there any particular rationale for disallowing identifier concatenation in the definition of items? I really hate using paste.

[SimonSapin]

It’s not that they’re excluded explicitly. It’s the other way around: parts of the language that expect an item, statement, or expression each accept various kinds of syntax, one of which being a macro invocation. In parts of the language that expect an identifier, the syntax is more restrictive (only accepting a plain or raw identifier token).

Accepting a macro invocation everywhere an identifier is accepted would be tricky at best because macro invocations themselves start with an identifier. paste adds dedicated [<…>] syntax specifically to avoid these ambiguities. Maybe accepting a macro invocation in a few more specific places like the name of an fn item is possible, I don’t know. I suspect even coming up with a concrete and plausible proposal would take some work.

[DemiMarie]

What about having concat_idents be a reserved word in Rust 2024? That would eliminate the ambiguity, because concat_idents! would no longer be a macro. Instead, it would be dedicated syntax that merely appeared to be a macro.

Of course, this might cause parsing problems. In particular, I doubt it would be compatible with an LALR(1) or LR(1) grammar.

[petrochenkov]

At this point it would probably be better to implement ident concatenation as a "metavariable expression", like ${concat_idents(a, b, c)} or something.

Then in a code produced by macro_rules and parsed by Rust parser it will already be represented as a single identifier, so it will work in any position.

[c410-f3r]

I think it would be nice to have something like ${concat_idents(a, ${concat_idents(b, c)}), i.e., a mechanism to eagerly evaluate a metavariable expression.

[tgross35]

I'm sure there is a good answer but I don't see it here - why do we need concat_idents as a standalone macro in the first place? In theory it should be possible to have macro bodies have a syntax that concatenates things, like foo_##bar in C macros or [<foo_ $bar>] in paste. I guess this is in line with the metavar idea.

Also, it is extremely helpful to be able to convert between upper, lower, snake, and camel cases identifiers in paste. I sure wouldn't mind having this in std somehow if there were a good syntax...

[safinaskar]

🚀 I found a way to evaluate concat_idents (and concat and few other built-in macros) before evaluating other macro, which takes concat_idents as an argument! I. e. I found a way to evaluate a!(concat_idents!(...)) such way, that concat_idents evaluates before a. Answer is crate https://crates.io/crates/with_builtin_macros !!! In other words, with_builtin_macros is paste, but not only for concat_idents, but also for concat and some other macros.

And in other words, with_builtin_macros allows one to achieve eager evaluation of macros in limited way.

Also, with_builtin_macros allows one to use concat_idents when defining new identifier.

Also, https://crates.io/crates/with_builtin_macros allows one to use concat_idents in stable Rust.

// (This code was not tested, may contain typos)

fn concat_idents!(a, b) () {} // Doesn't work

with_builtin_macros::with_eager_expansions! {
  fn #{ concat_idents!(a, b) } () {} // Works! Even on stable!
}

macro_rules! this_macro_accepts_ident {
  ($a:ident) => {}
}

this_macro_accepts_ident!(concat_idents!(a, b)); // Doesn't work, because "this_macro_accepts_ident" evaluates before "concat_idents"

with_builtin_macros::with_eager_expansions! {
  this_macro_accepts_ident!(#{ concat_idents!(a, b) }); // Works! Even on stable!
}