Lighter syntax for stateless networks?

[gdalle]

When the neural network I'm working with has no evolving state, I find it quite cumbersome to always call model(x, ps, st). This becomes especially painful when the st needs to be passed down as an argument, even though it is essentially useless.
How hard would it be to have a default model(x, ps) for these cases? Or a very cheap function empty_state(model) to avoid passing down st in lower level calls?

[avik-pal]

There are a few solutions to this (believe me I want it too. It will be a significant QoL feature). But there are quite a few shortcomings of each:

1. Add Lux.@layerdef for easy custom model creation #46 -- Use code transformations so that users can define models without state and we automatically generate the required code. This is not very trivial to implement and there are still edge cases.
2. How about (l::AbstractLayer)(x, ps, st) = l(x, ps), NamedTuple()? In this case, the user might have defined l(x, st) but since we don't know parameters from states by their types this call is inherently ambiguous. Could we deal with this using better documentation? Possibly, but I have tried extremely hard not to make implicit assumptions about user code (except 3-4 interface requirements), but this would violate that.
3. Update abstract types to contain information like AbstractExplicitLayer{paramterless, stateless}. Then we can unambiguously implement point 2. In the long term, I see this as the most stable solution to the problem and will honestly simplify a lot of code, but it is also not on my current priority list to implement. (That being said I am happy to review and merge a PR implementing this)

[gdalle]

I agree that option 3 seems like the most natural and achievable! Not sure I can submit a PR about this right now, but I'll still take a look

[gdalle]

Just to be sure, the code to modify would only be core.jl and the layers folder?