Processing Reflection-Related Strings in Go

[Arker123]

These strings appear to be related to RTTI / reflection. two things suggest this:

1. their contents, which are Go type names, and
2. working backwards, to see where they're used, are type descriptors (rtype and friends).

it might be possible to reconstruct these type descriptors, such as structures and their fields, used in a Go program, and then we'd be able to link the string to the type/structure/metadata.

the upside: more complete information
downside: more work

the existing approach is simple and probably works fine. maybe we can update the documentation here to clarify what strings these likely are (reflection-related).

Originally posted by @williballenthin in #792 (comment)

[stevemk14ebr]

Most (but not all!) rtype (or runtime type) structures are linked in the moduledata typelinks list. I would recommend not attempting to parse these structures, it is quite version specific as their layout has changed multiple times. In the case of obfuscators it is often very hard to determine the runtime version which makes parsing these version specific structures even harder.

As we already have a tool capable of parsing these back to both Go and C style structures: https://github.com/mandiant/GoReSym/blob/722143ebb199428cdf0a3dcfcdf29b621aa204fa/objfile/objfile.go#L1015 I would instead suggest that floss avoid this and delegate labeling strings of this type to that tool. There may be some sort of symbiosis we could leverage here, perhaps it makes sense to allow floss to invoke GoReSym? Or Embed it as a library? Are extensions to GoReSym necessary here to give floss the data it requires?

[williballenthin]

given that these strings are already null-terminated, or at least separated by non-printable characters, the net benefit of processing these structures is simply to annotate the FLOSS results better. the naive strings.exe algorithm already extracts these strings, it just isn't able to say "oh this is a reflect-related string, not from program source code". it would be nice to have this, but we should balance the value with the effort/maintenance burden.

[stevemk14ebr]

Agree it would be nice to provide this to a user, I suggest integrating full type parsing into floss would be a burden for multiple reasons. Would the integration of GoReSym in some fashion be in scope/reasonable?

This would contain the complexity to that tool.

[williballenthin]

yeah relying on GoReSym would be a great way to contain the complexity. i'm not sure how easy it would be to package GoReSym and invoke and read the results, but that's probably worthwhile to explore. providing GoReSym python bindings might be best; worse case would be sub process and read json from stdout.