i#6635 core filter, part 6: Add core-sharded record filter output #6704
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Multiple changes to allow the record filter to operate in core-sharded fashion: Makes the pc2encoding table per-input, as one input can migrate across multiple core shards and thus one core can see a later instruction without ever having seen its encoding. To handle synchronization, there is no C++11 std:: rwlock, so we use mutexes -- but we limit their use to per-context-switch for the added global lock, and we assume there is no contention for the per-input lock as only one shard operates on one input at any one time. Sets the memref counter reader to core_sharded_ to avoid asserts. Appends footer records to ending-in-idle-record cores. Adds an error check ensuring a single workload, as multiple will require expanding the keys used in some tables. Renames the output files to include "core.<shard_index>" and not the tid. This is surprisingly complex, as an input filename is needed to determine the output filename compression type: yet not all shards are guaranteed to have an input at the start. A condition variable and mutex are used to coordinate this among shards. Adds support for started-idle cores by synthesizing headers in record_filter; #6703 covers having the scheduler do this for all analyzers. Adds the version as another field available up front from the scheduler, and adds an idle-tid sentinel needed to be distinct from INVALID_THREAD_ID. Adds two end-to-end tests, one with a single-threaded app scheduled onto 4 cores to test start-idle cores and one to test multiple threads. Adds a macro to share code with the existing end-to-end test. Updates the unit test mock classes. Issue: #6635, #6703
abhinav92003
requested changes
Mar 12, 2024
…re-sharded-record-filter
|
Failure is #4167 "Invalid trace entry type thread_exit before a bundle" |
abhinav92003
approved these changes
Mar 13, 2024
derekbruening
added a commit
that referenced
this pull request
Mar 14, 2024
…cidentally left in place in PR #6704
derekbruening
added a commit
that referenced
this pull request
Mar 15, 2024
Improves record_filter_t subclass support for initially-idle cores by refactoring get_output_basename() out of initialize_shard_output(), allowing a subclass to share the complex initial setup while still using its own output scheme. Also moves the setup variable to protected for access to output_ext_ in subclasses. Removes code that was refactored into initialize_shard_output() but accidentally left in place in PR #6704. Tested internally. Issue: #6635
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Multiple changes to allow the record filter to operate in core-sharded fashion:
Makes the pc2encoding table per-input, as one input can migrate across multiple core shards and thus one core can see a later instruction without ever having seen its encoding. To handle synchronization, there is no C++11 std:: rwlock, so we use mutexes -- but we limit their use to per-context-switch for the added global lock, and we assume there is no contention for the per-input lock as only one shard operates on one input at any one time.
Sets the memref counter reader to core_sharded_ to avoid asserts.
Appends footer records to ending-in-idle-record cores.
Adds an error check ensuring a single workload, as multiple will require expanding the keys used in some tables.
Renames the output files to include "core.<shard_index>" and not the tid. This is surprisingly complex, as an input filename is needed to determine the output filename compression type: yet not all shards are guaranteed to have an input at the start. A condition variable and mutex are used to coordinate this among shards.
Adds support for started-idle cores by synthesizing headers in record_filter; #6703 covers having the scheduler do this for all analyzers. Adds the version as another field available up front from the scheduler, and adds an idle-tid sentinel needed to be distinct from INVALID_THREAD_ID.
Adds two end-to-end tests, one with a single-threaded app scheduled onto 4 cores to test start-idle cores and one to test multiple threads. Adds a macro to share code with the existing end-to-end test.
Updates the unit test mock classes.
Issue: #6635, #6703