Rework syscall invocation for proper behavior under typeguard #1672
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, using Typeguard with Manticore would break several emulated syscalls. With this commit, it does not. Some background information: When a syscall is made from an emulated binary, Manticore uses the syscall number to look up the appropriate Python method that models that syscall, and then uses Python introspection to massage arguments to that syscall model function as deemed appropriate. Previously, this mechanism used the deprecated `inspect.getfullargspec` to determine the number of arguments to the model function, and whether or not it takes varargs. However, `inspect.getfullargspec` doesn't look through wrapper functions; it looks only at exactly the function object it is given. This is an issue, however, when trying to inspect _decorated_ functions. (The use of a decorator in Python introduces a _new_ function object that wraps the decorated item.) How did that break Manticore when using Typeguard? It turns out that When using Typeguard via the `--typeguard-packages=manticore` option to `pytest`, the Typeguard plugin implicitly adds a `@typeguard.check_types` decorator on _every_ function & method in the `manticore` package. In this way, each syscall implementation function in Manticore ends up with a wrapper around it, and the syscall invocation mechanism based on `inspect.getfullargspec` would somewhat quietly cause syscalls to break. Now, instead of using `inspect.getfullargspec`, Manticore's syscall invocation mechansim uses the non-deprecated `inspect.signature` API to get the needed information. This API _does_ look through wrapper functions. Additionally, it allows us to get rid of some conditional logic, about whether `self` appears in a function's parameter list or not.
ekilmer
reviewed
Apr 17, 2020
Wondering: Do we still need |
added 2 commits
April 21, 2020 11:18
We only were using `wrapt` in a single place -- to implement the `unimplemented` syscall decorator. That dependency was added in #1384, so that the old syscall mechanism could work with the decorator. Now, with the rework of Manticore's syscall mechanism, this is no longer necessary, and a "regular" Python decorator implemented using `functools.wraps` should work just fine.
@ehennenfent I looked at this a bit closer, and I don't think it's necessary any more! So I've rephrased the single use of |
ehennenfent
reviewed
Apr 21, 2020
added 3 commits
April 21, 2020 16:57
Additionally, add some extra tests for the decorator, to get better coverage from where it is used.
(found with `mypy --check-untyped-defs` and some grep)
|
|
ekilmer
requested changes
Apr 22, 2020
ekilmer
added a commit
that referenced
this pull request
Apr 23, 2020
* master: (28 commits) Rework syscall invocation for proper behavior under typeguard (#1672) printable_bytes (#1671) Fix several incorrect type hints (#1668) Add type hints to several parts of Manticore (#1667) Fix type confusion in manual CPU tests; delete dead testing code (#1669) Rework WASM Imports and Fix Typos (#1666) Work on "Model is not Available" errors in tests (#1659) Fix TypeGuard Errors in WASM Module (#1601) Remove unused arg ans separate output from workspace (#1651) Add a badge to README.md for LGTM (#1647) Fix 2 problems in Linux `sys_open` support & add type hints (#1657) Fix LGTM Errors (#1656) Delay WASM Branch Condition Concretization (#1641) Add feature for SymbolicSocket through sys_accept (#1618) Add a bunch of type hints & fix a few issues with Linux platform emulation (#1645) Bump to CheckoutV2 (#1654) Add support for `sys_arm_fadvise64_64` (#1648) Add __slots__ to expressions (#1635) Swap remaining uses of `Z3Solver()` to use the singleton interface (#1649) CI: have pytest report 100 slowest tests (#1646) ...
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, using Typeguard with Manticore would break several emulated syscalls. With this commit, it does not.
Some background information:
When a syscall is made from an emulated binary, Manticore uses the syscall number to look up the appropriate Python method that models that syscall, and then uses Python introspection to massage arguments to that syscall model function as deemed appropriate.
Previously, this mechanism used the deprecated
inspect.getfullargspecto determine the number of arguments to the model function, and whether or not it takes varargs.However,
inspect.getfullargspecdoesn't look through wrapper functions; it looks only at exactly the function object it is given. This is an issue, however, when trying to inspect decorated functions. (The use of a decorator in Python introduces a new function object that wraps the decorated item.)How did that break Manticore when using Typeguard? It turns out that when using Typeguard via the
--typeguard-packages=manticoreoption topytest, the Typeguard plugin implicitly adds a@typeguard.check_typesdecorator on every function & method in themanticorepackage.In this way, each syscall implementation function in Manticore ends up with a wrapper around it, and the syscall invocation mechanism based on
inspect.getfullargspecwould somewhat quietly cause syscalls to break.Now, instead of using
inspect.getfullargspec, Manticore's syscall invocation mechansim uses the non-deprecatedinspect.signatureAPI to get the needed information. This API does look through wrapper functions. Additionally, it allows us to get rid of some conditional logic, about whetherselfappears in a function's parameter list or not.