Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

i#4318 xarch memtrace: Support gzipped .raw files #4332

Merged
merged 9 commits into from
Jun 24, 2020

Conversation

derekbruening
Copy link
Contributor

Adds a new gzip wrapper around std::istream to support compressing
.raw files. This lets us easily shrink checked-in test files, and
aids other relocate-then-process situations with raw files.

Updates the altbin test by compressing its .raw files.

Issue: #4318

Adds a new option -alt_module_dir which specifies an alternate
directory in which to look for libraries in a modules.log offline
drcachesim file.  Hooks the option up to the opcode_mix and view tools
and to raw2trace (both its standalone launcher and drcachesim frontend
integration).

This will be useful in general, but is needed for adding tests with
checked-in raw files and binaries, where the checked-in modules.log
has a hardcoded path that will never match all test environments.

Adds a test that runs both raw2trace and opcode_mix on a new
checked-in aarch64 raw trace (this test will evolve for future
cross-arch testing, which is the reason for using aarch64).  The
binaries are stripped, with the two libraries "fake" binaries:
libmemtrace.so is not executed, and libdynamorio.so is only used for
dynamorio_sigreturn, so these are synthetic binaries (created from asm
files) to keep them smaller.  The tests deliberately fail on x86 and
match an error message that will improve when architecture tags are
added.

Issue: #4318
Adds architecture tags to raw traces, propagated to final traces in a
new marker type.  Adds reader support for the new type.

Adds a check for unhandled architectures to raw2trace.

Adds checks for unhandled architectures to the opcode_mix and view
tools.  To reach that point, relaxes the ELF header checks to allow
libraries of the same bitwidth but different architectures (i#1345
prevents allowing all libraries), so we can map in the other-arch
libraries.

Updates the existing aarch64 raw trace files to contain architecture
tags, and updates the x86 failure message.

Cross-architecture handling of these same files was tested manually
with a prototype which will be committed in the future.

Issue: #4318, #1345
Adds a new gzip wrapper around std::istream to support compressing
.raw files.  This lets us easily shrink checked-in test files, and
aids other relocate-then-process situations with raw files.

Updates the altbin test by compressing its .raw files.

Issue: #4318
@derekbruening derekbruening requested a review from s-kanev June 23, 2020 01:03
@derekbruening
Copy link
Contributor Author

So once again Travis isn't triggering on a chained PR (compare vs base PR #4331 instead of master): I'm still not seeing what in the Travis config is causing this. I'll re-base to master once #4331 (and its base #4330) is in and ensure Travis is green prior to merge.

@derekbruening
Copy link
Contributor Author

Wow: so that made two show up (avoiding the dup is what the .travis.yml exclusion is there for). Makes it seem like there's some Github PR trigger problem? And any new commit to the new PR's branch will properly trigger Travis?

@derekbruening
Copy link
Contributor Author

Nope: theory was wrong. Reinstating the exclusion resulted in zero Travis runs. Hmm.

clients/drcachesim/common/gzip_istream.h Outdated Show resolved Hide resolved
Base automatically changed from i4318-arch-tags to master June 24, 2020 03:46
@derekbruening derekbruening merged commit 8da4e78 into master Jun 24, 2020
@derekbruening derekbruening deleted the i4318-gzip-raw-files branch June 24, 2020 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants