eager evaluation of full syntax tree information

[benmkw]

I'd like to propose that it would be useful to have an API which makes all information of a rust file easily available in one go.

My use case is to generate a json file from each rust file which has type/ usage information about each struct/ function definition and definition and docs information about each usage (motivation: https://drewdevault.com/2019/07/08/Announcing-annotations-for-sourcehut.html).

I looked at how cbindgen does that because that seemed similar enough and they indeed have a similar problem (see the warning) and already also looked into ra mozilla/cbindgen#214 (comment) .

As a comparison for other languages I would highlight c ("clang -Xclang -ast-dump=json sokol_gfx.h" https://twitter.com/FlohOfWoe/status/1290299480881782795) and go (https://github.com/golang/example/tree/master/gotypes and https://pkg.go.dev/golang.org/x/tools/go/packages).

I asked about this on zulip and was pointed to the syntax_highlighting.rs code so maybe the pattern used there could be turned into an API that is useful for consumers.

[lnicola]

The rust-analyzer analysis-stats implementation might be useful here, but you could also look into the new rustdoc JSON backend, which might be easier to get started with.

[benmkw]

The rustdoc JSON backend looks interesting although it does not cover the doc use case as it only contains information for declarations and not bodies as far as I can see, but it might be interesting for cbindgen actually as it seems roughly equivalent to clang -Xclang -ast-dump=json.

Regarding the unactionable label I'm not sure what specific details are required to make this actionable other than the example of the go package and the example use case.
Another one would be generating a call-graph which would be pretty trivial with that information and helpful for docs as well.

[bjorn3]

Another one would be generating a call-graph which would be pretty trivial with that information and helpful for docs as well.

See https://github.com/robinmoussu/cargo-callgraph/

My use case is to generate a json file from each rust file which has type/ usage information about each struct/ function definition and definition and docs information about each usage (motivation: https://drewdevault.com/2019/07/08/Announcing-annotations-for-sourcehut.html).

I think it would make sense to build a tool using the rust-analyzer api that directly outputs the exact format that sourcehut expects and possibly even directly uploads it rather than have it consume a rust-analyzer specific json format that likely won't be stable either and may be slower due to needing to be more generic and thus requiring more info to be generated. In addition a json format is likely to miss information as new features are added to rust-analyzer.

[benmkw]

See https://github.com/robinmoussu/cargo-callgraph/

Thanks I looked at some a while ago and concluded they were not really usable yet e.g.

Currently, I highly advice against testing cargo-callgraph on any crate that has more than 500 SLOC since the output will be very hard to visualize, and dot is extremely slow with bigger graph.

but on the other hand thats easy enough to fix eventually and not related to the issue I opened.

I think it would make sense to build a tool using the rust-analyzer api

I totally agree, I just tried 3 times I think and it seemed not very easy to use for me (compared to what it looked like in go, maybe because the scope of the project is slightly different)

[Veykril]

Sounds like the SCIP/LSIF output is what is desired here which already exists