i#3995 multi-window: Add repeated tracing window feature

Adds new options -trace_for_instrs and -retrace_every_instrs to
drcachesim for periodic trace bursts of an unmodified application.
Implements them by adapting the existing drbbdup cases for switching
between -trace_after_instrs and full tracing.

Adds documentation on the new options.

Adds instru_t::get_instr_count to count instuctions while tracing, to
know when a tracing burst window is finished.  Uses a local counter
only added to the global every 10K instructions to avoid
synchronization costs.

Adds a new marker with the ordinal of the trace window.  This marker
is added to each buffer header.  This, combined with a new check for
the window having changed to ensure a buffer dump at the end of each
block, limits the possible window drift to one block's worth of data.

Augments raw2trace to avoid delaying a branch across a window change.

Augments the view tool to mark window changes and delay timestamp
output to group with the proper window (it is difficult to actually
reorder timestamp and window entries).

Augments the basic_counts tool to track and display per-window global
statistics.

Augments the invariant_checker tool to not complain on a control-flow
gap across a window.  Adds a test of this: but disables it for Windows
temporarily due to more emulation interopability issues which #5390
covers.

Adds a simple online test and a simple offline test that just confirm
multiple windows are hit on simple_app.  Adds an assembly test with
precise values for the windows.

Issue: #3995, #5390

api/docs/release.dox

@@ -132,6 +132,8 @@ Further non-compatibility-affecting changes include:
  - Added new fields analyze_case_ex and instrument_instr_ex to #drbbdup_options_t.
  - Added drbbdup support to drwrap via #DRWRAP_INVERT_CONTROL, drwrap_invoke_insert(),
    and drwrap_invoke_insert_cleanup_only().
+ - Added -trace_for_instrs and -retrace_every_instrs options to drcachesim
+   for periodic trace bustrs of an unmodified application.
 
 The changes between version 9.0.1 and 9.0.0 include the following compatibility
 changes:
@@ -449,8 +451,6 @@ Further non-compatibility-affecting changes include:
    dr_get_process_id() in some contexts.
  - The private loader's malloc redirection now guarantees double-pointer-sized
    alignment, to match what system-provided allocators use.
- - Added a new DR extension, namely "drbbdup", which enables different case
-   instrumentation of the same basic block by duplicating code.
 
 **************************************************
 <hr>

clients/drcachesim/common/options.cpp

@@ -1,5 +1,5 @@
 /* **********************************************************
- * Copyright (c) 2015-2021 Google, Inc.  All rights reserved.
+ * Copyright (c) 2015-2022 Google, Inc.  All rights reserved.
  * **********************************************************/
 
 /*
@@ -240,15 +240,30 @@ droption_t<bytesize_t> op_trace_after_instrs(
     DROPTION_SCOPE_CLIENT, "trace_after_instrs", 0,
     "Do not start tracing until N instructions",
     "If non-zero, this causes tracing to be suppressed until this many dynamic "
-    "instruction "
-    "executions are observed.  At that point, regular tracing is put into place. "
+    "instruction executions are observed from the start of the application. "
+    "At that point, regular tracing is put into place.  "
     "The threshold should be considered approximate, especially for larger values. "
-    "Switching to regular tracing takes some amount of time during which other "
-    "threads than the one that triggered the switch can continue to execute, "
-    "resulting in a larger count of executed instructions before tracing actually "
-    "starts than this given threshold. "
-    "Use -max_trace_size or -max_global_trace_refs to set a limit on the subsequent "
-    "trace length.");
+    "Use -trace_for_instrs or -max_trace_size to set a limit on the subsequent trace "
+    "length.  Use -retrace_every_instrs to trace repeatedly.");
+
+droption_t<bytesize_t> op_trace_for_instrs(
+    DROPTION_SCOPE_CLIENT, "trace_for_instrs", 0,
+    "After tracing N instructions, stop tracing, but continue executing.",
+    "If non-zero, this stops recording a trace after the specified number of "
+    "instructions are traced.  Unlike -exit_after_tracing, which kills the "
+    "application (and counts data as well as instructions), the application "
+    "continues executing.  This can be combined with -retrace_every_instrs. "
+    "The actual trace period may vary slightly from this number due to optimizations "
+    "that reduce the overhead of instruction counting.");
+
+droption_t<bytesize_t> op_retrace_every_instrs(
+    DROPTION_SCOPE_CLIENT, "retrace_every_instrs", 0,
+    "Trace for -trace_for_instrs, execute this many, and repeat.",
+    "This option augments -trace_for_instrs.  After tracing concludes, this option "
+    "causes non-traced instructions to be counted and after the number specified by "
+    "this option, tracing start up again for the -trace_for_instrs duration.  This "
+    "process repeats itself.  This can be combined with -trace_after_instrs for an "
+    "initial period of non-tracing.");
 
 droption_t<bytesize_t> op_exit_after_tracing(
     DROPTION_SCOPE_CLIENT, "exit_after_tracing", 0,

clients/drcachesim/common/options.h

@@ -89,6 +89,8 @@ extern droption_t<bool> op_cpu_scheduling;
 extern droption_t<bytesize_t> op_max_trace_size;
 extern droption_t<bytesize_t> op_max_global_trace_refs;
 extern droption_t<bytesize_t> op_trace_after_instrs;
+extern droption_t<bytesize_t> op_trace_for_instrs;
+extern droption_t<bytesize_t> op_retrace_every_instrs;
 extern droption_t<bytesize_t> op_exit_after_tracing;
 extern droption_t<bool> op_online_instr_types;
 extern droption_t<std::string> op_replace_policy;

clients/drcachesim/common/trace_entry.h

@@ -1,5 +1,5 @@
 /* **********************************************************
- * Copyright (c) 2015-2021 Google, Inc.  All rights reserved.
+ * Copyright (c) 2015-2022 Google, Inc.  All rights reserved.
  * **********************************************************/
 
 /*
@@ -326,6 +326,14 @@ typedef enum {
      */
     TRACE_MARKER_TYPE_RSEQ_ABORT,
 
+    /**
+     * Identifies in the marker value the ordinal of a window during a multi-window
+     * tracing run (see the options -trace_for_instrs and -retrace_every_instrs).
+     * When a marker with an ordinal value different from the last-seen marker
+     * appears, a time gap may exist between this new marker and subsequent entries.
+     */
+    TRACE_MARKER_TYPE_WINDOW_ID,
+
     // ...
     // These values are reserved for future built-in marker types.
     // ...

clients/drcachesim/drcachesim.dox.in

@@ -161,6 +161,8 @@ Some of the more important markers are:
 
 - #TRACE_MARKER_TYPE_FUNC_ID, #TRACE_MARKER_TYPE_FUNC_RETADDR, #TRACE_MARKER_TYPE_FUNC_ARG, #TRACE_MARKER_TYPE_FUNC_RETVAL - These markers are used to capture information about function calls.  Which functions to capture must be explicitly selected at tracing time.  Typical candiates are heap allocation and freeing functions.  See \ref sec_drcachesim_funcs.
 
+- #TRACE_MARKER_TYPE_WINDOW_ID - The marker value contains the ordinal of the trace burst window when a multi-window trace was collected (see \ref sec_drcachesim_partial).
+
 The full set of markers is listed under the enum #trace_marker_type_t.
 
 ****************************************************************************
@@ -954,6 +956,15 @@ dynamic instruction executions.  This can be used to skip initialization
 and arrive at the desired starting point.  The trace's length can be
 limited in several ways:
 
+- The \p -trace_for_instrs option stops tracing after the specified number
+  of dynamic instrutions.
+- The \p -retrace_every_instrs option augments -p -trace_for_instrs by
+  executing its specified instruction count without tracing and then
+  re-enabling tracing for \p -trace_for_instrs again, resulting in
+  tracing windows repeated at regular intervals throughout the execution.
+  A single final trace is created at the end, with #TRACE_MARKER_TYPE_WINDOW_ID
+  markers (see \ref sec_drcachesim_format_other) identifying the trace window
+  transitions.
 - The \p -max_global_trace_refs option causes the recording of trace
   data to cease once the specified threshold is exceeded by the sum of
   all trace references across all threads.  One trace reference entry

clients/drcachesim/tests/allasm-repstr-basic-counts.templatex

@@ -1,37 +1,45 @@
 Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
 ---- <application exited with code 0> ----
 Basic counts tool results:
 Total counts:
-          19 total \(fetched\) instructions
-          19 total unique \(fetched\) instructions
-/* One for each movs after the first one. */
+          95 total \(fetched\) instructions
+          23 total unique \(fetched\) instructions
            4 total non-fetched instructions
            0 total prefetches
            5 total data loads
            5 total data stores
            0 total icache flushes
            0 total dcache flushes
            1 total threads
-     .* total scheduling markers
-     .* total transfer markers
-     .* total function id markers
-     .* total function return address markers
-     .* total function argument markers
-     .* total function return value markers
-     .* total other markers
-Thread .* counts:
-          19 \(fetched\) instructions
-          19 unique \(fetched\) instructions
+          24 total scheduling markers
+           0 total transfer markers
+           0 total function id markers
+           0 total function return address markers
+           0 total function argument markers
+           0 total function return value markers
+           3 total other markers
+Thread [0-9]* counts:
+          95 \(fetched\) instructions
+          23 unique \(fetched\) instructions
            4 non-fetched instructions
            0 prefetches
            5 data loads
            5 data stores
            0 icache flushes
            0 dcache flushes
-     .* scheduling markers
-     .* transfer markers
-     .* function id markers
-     .* function return address markers
-     .* function argument markers
-     .* function return value markers
-     .* other markers
+          24 scheduling markers
+           0 transfer markers
+           0 function id markers
+           0 function return address markers
+           0 function argument markers
+           0 function return value markers
+           3 other markers

clients/drcachesim/tests/allasm_repstr.asm

@@ -1,5 +1,5 @@
  /* **********************************************************
- * Copyright (c) 2021 Google, Inc.  All rights reserved.
+ * Copyright (c) 2021-2022 Google, Inc.  All rights reserved.
  * **********************************************************/
 
 /*
@@ -53,12 +53,17 @@ _start:
         cld
         rep      movsb
 
-        // Print end message.
+        // Print a message in a loop for testing tracing windows.
+        mov      ebx, 10          // Loop count.
+repeat:
         mov      rdi, 2           // stderr
         lea      rsi, hello_str
         mov      rdx, 13          // sizeof(hello_str)
         mov      eax, 1           // SYS_write
         syscall
+        dec      ebx
+        cmp      ebx, 0
+        jnz      repeat
 
         // Exit.
         mov      rdi, 0           // exit code

clients/drcachesim/tests/offline-allasm-repstr-basic-counts.templatex

@@ -1,36 +1,44 @@
 Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
+Adios world!
 Basic counts tool results:
 Total counts:
-          19 total \(fetched\) instructions
-          19 total unique \(fetched\) instructions
-/* One for each movs after the first one. */
+          95 total \(fetched\) instructions
+          23 total unique \(fetched\) instructions
            4 total non-fetched instructions
            0 total prefetches
            5 total data loads
            5 total data stores
            0 total icache flushes
            0 total dcache flushes
            1 total threads
-     .* total scheduling markers
-     .* total transfer markers
-     .* total function id markers
-     .* total function return address markers
-     .* total function argument markers
-     .* total function return value markers
-     .* total other markers
-Thread .* counts:
-          19 \(fetched\) instructions
-          19 unique \(fetched\) instructions
+          24 total scheduling markers
+           0 total transfer markers
+           0 total function id markers
+           0 total function return address markers
+           0 total function argument markers
+           0 total function return value markers
+           3 total other markers
+Thread [0-9]* counts:
+          95 \(fetched\) instructions
+          23 unique \(fetched\) instructions
            4 non-fetched instructions
            0 prefetches
            5 data loads
            5 data stores
            0 icache flushes
            0 dcache flushes
-     .* scheduling markers
-     .* transfer markers
-     .* function id markers
-     .* function return address markers
-     .* function argument markers
-     .* function return value markers
-     .* other markers
+          24 scheduling markers
+           0 transfer markers
+           0 function id markers
+           0 function return address markers
+           0 function argument markers
+           0 function return value markers
+           3 other markers