client support for persistent and process-shared caches

[derekbruening]

From [email protected] on February 24, 2009 10:51:55

this was PR 305329

We should explore pcache support now before the 0.9.6 release to determine
whether we can add it later while preserving backward compatibility.
I think we can:

• need client id => we already have it
  need to store in pcache, and make part of pcache namespace for
  simultaneous pcaches.
  for multiple clients, put all ids in namespace, so won't load unless
  exact same combination is present on subsequent run.

  also add event callback when loading pcache => client can decide whether
  to load based on whether already has desired instru in pcache. thus,
  want client version # as well (else, client has to bump id#?). could
  specify separately when do persist nudge and so not change the dr_init
  signature, or make an event callback if DR needs to ask for it. for
  multiple clients, if any one of them rejects it, don't use it.

  problem: the client id is really just for the current instance of DR and is
  specified by the user; it's so the client can request its options, etc.
  it's not a persistent tool identifier.

  so how will a client know whether its instrumentation is present in a
  pcache? do we want two types of identifiers: one that's in the
  namespace, for simultaneous pcaches, and another that's in the pcache
  header that the client can check for versioning? what if for pcaches we
  added two event callbacks: get_client_id() and get_client_version().
  they're ints, or maybe 64-bit ints. we use the 1st in the pcache
  namespace, and store the 2nd in the pcache header for retrieval when we
  load. a client that doesn't supply them isn't allowed to persist. when
  DR creates a pcache it will call these for each client so it can label
  the pcache with which tools were active when it was created.
  problem: confusing having this new notion of id vs the client_id_t used
  in dr_init.

  => solution: use client library identifying hash (similar to what we use
  for app modules for pcaches and hot patches: checksum + timestamp + size)
  in the namespace. if the client is rebuilt, pcaches will have to be
  re-generated: that shouldn't be a big deal, and can avoid bugs where the
  client dev thinks a change shouldn't affect persistence but really it does.

• DR_EMIT_DO_NOT_PERSIST flag so can control on individual fragment
  basis => addition only

• query whether ilist/fragment is persistable => addition only

• control over where pcaches are stored and who they are shared with;
  also over when to persist
  => drdeploy/global options, plus maybe API call to freeze/persist

• relocation:

  • app relocation: client non-meta code: if set translation fields right,
    our own relocation support should cover it
  • client meta code relocation: require client to use position-independent
    code? else, how point client at its code later to fix up? make it
    provide a relocation table? in any case should all be additions.

We'll want to finish some internal pieces first:

• PR 214016/9581, PR 214084/9649: relocation support
• PR 270739: [x64] need vendor pcache flag for lcall,ljmp rex.w differences
• PR 214155/9720: offline publisher + two-level hierarchy of sharing
• PR 215036/10601, PR 215277/10842: two-phase, lazy md5
• perf: PR 215260/10825: measure -no_persist_map_rw_separate prefetch impact
• perf: PR 210308/5836: don't want private bbs not being in ibl to
  cause perf problems!
• PR 206574/2096 (xref PR 215247/10812 winlogon synchall issues)

We also have some performance issues (xref PR 326610):

• PR 213262: support -no_indirect_stubs w/ -coarse_units, to avoid losing perf
• PR 326815: [perf] -coarse_units slow on gcc and gap

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=40

[derekbruening]

From [email protected] on February 08, 2012 13:54:40

in r1243 :

issue #40: add client support for persistent caches

• added 10 new events, registered via:
  • dr_register_persist_ro()
  • dr_register_persist_rx()
  • dr_register_persist_rw()
  • dr_register_persist_patch()
• added new bb event return flag DR_EMIT_PERSISTABLE.
• client can control which bbs are persisted, add additional read-only and
  writable persisted data, add additional generated code, and patch
  persisted bbs
• the patching interface is still under discussion: this commit puts
  in place one version, but it may be changed in the future.
• client can use events to validate persisted files
• DR validates files to ensure the same set of clients is enabled,
  done by full path. decided against checking client lib checksums.
• had to delay loading pcaches from the normal find_executable_vm_areas()
  to a second pass after instrument_init() (we want
  find_executable_vm_areas() before instrument_init() so we can have the
  module list in place for client init-time queries)
• had to re-order the pcache length calculations (though it's probably
  cleaner this way) and split the padding up in order to pass the file
  offset to the persist size event
• added documentation in release.dox on the feature, intro.dox
  on the runtime options, and a big section in bt.dox on usage
• added new test pcache.dll.c run twice
• added support for test name not equal to exe for client tests
• added event tests to events.dll.cpp and events_cpp.dll.cpp

A more sophisticated test will be added later, and an even bigger one once
Dr. Memory uses this interface. I tried to anticipate everything Dr. Memory
would need, so the interface should not need to change, other than
the patching API details as mentioned above.