drsyms sideline symbol server

[derekbruening]

From [email protected] on May 06, 2011 11:42:05

split from issue #44 see all the comments in that issue regarding sideline support
this issue covers a sideline implementation on both linux and windows

sideline is needed in cases where symbols don't fit into the app's process
(see https://code.google.com/p/dynamorio/issues/detail?id=44#c11 )

my drsyms API assumes sideline will use the same synchronous interface as
online, and uses shared memory, with a shmid passed to drsym_init(), so the
sideline server must be launched ahead of time.
we can revisit that decision but if we want to provide an asynchronous
model I would suggest adding another set of interfaces for that and keeping
the synchronous ones working with sideline.

here are some notes from earlier discussions including proposals of having
the sym server queries have tunable timeouts:

Shared memory is definitely the best choice for Linux and Windows. For
Windows the keys are strings. See api/samples/stats.c for code using
syscall/ntdll layer (for DR), and tools/DRgui/DynamoRIOView.cpp for code
using Win32 API (for symbol server).

I would vote for what you listed under Optimization:

• Have a simple platform-independent symbol table format that ignores
  scopes and other complexities (unless you have a use case that requires
  scopes?)

• The client symbol query API is also very simple (unless we have use cases
  that need more):

  • look up symbol name from address
  • look up address from symbol name
  • look up func+file+line# from address

  The client query API is abstracted away from the underlying symbol table
  format, which is not exposed, so that we can change the format later and
  add scopes, etc. if we need them.

• On a module load, DR requests that the symbol server provide a symbol
  table for that module. That request is asynchronous by default, and if
  the client makes a query before the information is ready the query simply
  fails. We provide a runtime option that makes DR block until the symbol
  info is ready on a module load for clients that need that and don't want
  to delay/retry their queries.

Then we don't have any client queries going to the symbol server: DR
answers them synchronously from its platform-independent symbol tables, and
only DR's entire-module queries go to the symbol server.

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=446

[derekbruening]

From [email protected] on May 06, 2011 08:50:23

from my original commit notes:

• Sideline vs online: drsyms.h presents the same interface for querying
  symbols locally or remotely. An initialization function specifies the
  server to use, if any. When using sideline, both the client and server
  have a copy of the drsysm library, with the client's drsyms forwarding
  requests to the server, whose drsyms performs actual lookups.

  I'm abandoning these ideas:

  • support a mixture of online and sideline:
    give client control on a per-query basis => extra param to each query?
    or up-front params specifying size threshold of images or something
    and only large queries are routed to sideline?
  • in past discussions we wanted to give the client the choice of having
    queries fail instead of blocking and/or having asynchronous queries,
    and having drsyms load data for a whole module at once on module load
    and cache it. I'm leaning toward ignoring that here and just having
    potentially-high-latency queries when using sideline.

[derekbruening]

From [email protected] on June 15, 2011 11:05:05

xref issue #498 on exposing whatever IPC method is used here through the API