-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proc: use stack machine to evaluate expressions #3508
Conversation
5725659
to
473f003
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Initial review done.
This is amazing work!
Just out of curiosity have you ran benchmarks on the existing implementation and the new one? |
No but I don't expect it to matter much. Last time I did benchmark this code it was completely dominated by system calls. |
This commit splits expression evaluation into two parts. The first part (in pkg/proc/evalop/evalcompile.go) "compiles" as ast.Expr into a list of instructions (defined in pkg/proc/evalop/ops.go) for a stack machine (defined by `proc.(*evalStack)`). The second part is a stack machine (implemented by `proc.(*EvalScope).eval` and `proc.(*EvalScope).evalOne`) that has two modes of operation: in the main mode it executes inteructions from the list (by calling `evalOne`), in the second mode it executes the call injection protocol by calling `funcCallStep` repeatedly until it either the protocol finishes, needs more input from the stack machine (to set call arguments) or fails. This approach has several benefits: - it is now possible to remove the goroutine we use to evaluate expression and the channel used to communicate with the Continue loop. - every time we resume the target to execute the call injection protocol we need to update several local variables to match the changed state of the target, this is now done at the top level of the evaluation loop instead of being hidden inside a recurisive evaluator - using runtime.Pin to pin addresses returned by an injected call would allow us to use a more natural evaluation order for function calls, which would solve some bugs go-delve#3310, allow users to inspect values returned by a call injection go-delve#1599 and allow implementing some other features go-delve#1465. Doing this with the recursive evaluator, while keeping backwards compatibility with versions of Go that do not have runtime.Pin is very hard. However after this change we can simply conditionally change how compileFunctionCall works and add some opcodes.
For some reason I have pushed to my evalrefactor branch but github has decided not to pick up my changes in this PR. I'm not sure what to do, for now I'll wait and see if it fixes itself. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few more small things. This overall looks great though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/go-delve/delve](https://togithub.com/go-delve/delve) | `v1.21.2` -> `v1.22.0` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fgo-delve%2fdelve/v1.22.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2fgo-delve%2fdelve/v1.22.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2fgo-delve%2fdelve/v1.21.2/v1.22.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fgo-delve%2fdelve/v1.21.2/v1.22.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>go-delve/delve (github.com/go-delve/delve)</summary> ### [`v1.22.0`](https://togithub.com/go-delve/delve/releases/tag/v1.22.0) [Compare Source](https://togithub.com/go-delve/delve/compare/v1.21.2...v1.22.0) #### What's Changed - all: remove obsolete build tags by [@​alexandear](https://togithub.com/alexandear) in [https://github.com/go-delve/delve/pull/3513](https://togithub.com/go-delve/delve/pull/3513) - service/dap: add the concrete type to interface type by [@​suzmue](https://togithub.com/suzmue) in [https://github.com/go-delve/delve/pull/3510](https://togithub.com/go-delve/delve/pull/3510) - service/dap: fix bugs in stdout/stderr handling by [@​hyangah](https://togithub.com/hyangah) in [https://github.com/go-delve/delve/pull/3522](https://togithub.com/go-delve/delve/pull/3522) - pkg/terminal: support vscode in edit command by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3524](https://togithub.com/go-delve/delve/pull/3524) - Use a valid timezone in tested binary by [@​upils](https://togithub.com/upils) in [https://github.com/go-delve/delve/pull/3527](https://togithub.com/go-delve/delve/pull/3527) - proc: implement follow exec mode on Windows by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3507](https://togithub.com/go-delve/delve/pull/3507) - service/test: disable TestClientServer_chanGoroutines with rr backend by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3492](https://togithub.com/go-delve/delve/pull/3492) - proc/native: use cgo instead of C for freebsd by [@​4a6f656c](https://togithub.com/4a6f656c) in [https://github.com/go-delve/delve/pull/3529](https://togithub.com/go-delve/delve/pull/3529) - proc: use DW_AT_trampoline to detect auto-generated code by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3528](https://togithub.com/go-delve/delve/pull/3528) - proc: use stack machine to evaluate expressions by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3508](https://togithub.com/go-delve/delve/pull/3508) - proc: fix comment typos by [@​alexandear](https://togithub.com/alexandear) in [https://github.com/go-delve/delve/pull/3531](https://togithub.com/go-delve/delve/pull/3531) - proc: add min and max builtins by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3530](https://togithub.com/go-delve/delve/pull/3530) - CI: update staticcheck version by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3533](https://togithub.com/go-delve/delve/pull/3533) - proc: remove expr evaluation goroutine from EvalExpressionWithCalls by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3532](https://togithub.com/go-delve/delve/pull/3532) - service/api: use bitfield for prettyprint flags by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3536](https://togithub.com/go-delve/delve/pull/3536) - proc: allow evaluator to reference previous frames by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3534](https://togithub.com/go-delve/delve/pull/3534) - \*: Add explicit code of conduct by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3540](https://togithub.com/go-delve/delve/pull/3540) - proc,service/dap,proc/gdbserial: fixes for debugserver --unmask-signals by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3541](https://togithub.com/go-delve/delve/pull/3541) - \*: release 1.21.2 (for master) by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3546](https://togithub.com/go-delve/delve/pull/3546) - proc/native: wherever we check for exited we should also check detached by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3547](https://togithub.com/go-delve/delve/pull/3547) - pkg/proc: add inline function support for stripped binaries by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3549](https://togithub.com/go-delve/delve/pull/3549) - cmd: fix a bunch of linter warnings from GoLand by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3550](https://togithub.com/go-delve/delve/pull/3550) - proc: add regression test for issue [#​3548](https://togithub.com/go-delve/delve/issues/3548) by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3553](https://togithub.com/go-delve/delve/pull/3553) - \*: update dependencies by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3552](https://togithub.com/go-delve/delve/pull/3552) - service: fix a bunch of linter warnings from GoLand by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3551](https://togithub.com/go-delve/delve/pull/3551) - \*: Correct spelling mistakes by [@​alexandear](https://togithub.com/alexandear) in [https://github.com/go-delve/delve/pull/3555](https://togithub.com/go-delve/delve/pull/3555) - pkg,service: Remove redundant build constraints by [@​alexandear](https://togithub.com/alexandear) in [https://github.com/go-delve/delve/pull/3556](https://togithub.com/go-delve/delve/pull/3556) - Shorten variable types by [@​stefanhaller](https://togithub.com/stefanhaller) in [https://github.com/go-delve/delve/pull/3535](https://togithub.com/go-delve/delve/pull/3535) - TeamCity: reupgrade linux/386 builder to Go 1.21 by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3560](https://togithub.com/go-delve/delve/pull/3560) - pkg/proc: improve support unwinding from sigpanic by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3559](https://togithub.com/go-delve/delve/pull/3559) - pkg/proc: unskip passing tests and reorganize by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3561](https://togithub.com/go-delve/delve/pull/3561) - chore: use strings.Contains instead by [@​testwill](https://togithub.com/testwill) in [https://github.com/go-delve/delve/pull/3562](https://togithub.com/go-delve/delve/pull/3562) - pkg,service: remove unnecessary convertions by [@​alexandear](https://togithub.com/alexandear) in [https://github.com/go-delve/delve/pull/3564](https://togithub.com/go-delve/delve/pull/3564) - \*: remove checks for TRAVIS env variable by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3568](https://togithub.com/go-delve/delve/pull/3568) - terminal: clear substitute path rules cache when config is used by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3567](https://togithub.com/go-delve/delve/pull/3567) - service: fix typo in variable name by [@​alexandear](https://togithub.com/alexandear) in [https://github.com/go-delve/delve/pull/3575](https://togithub.com/go-delve/delve/pull/3575) - TeamCity: remove windows/arm64 builders from chain by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3572](https://togithub.com/go-delve/delve/pull/3572) - proc: simplify and generalize runtime.mallocgc workaround by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3571](https://togithub.com/go-delve/delve/pull/3571) - proc/gdbserial: refactor parsing of key-value pairs from gdb protocol by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3574](https://togithub.com/go-delve/delve/pull/3574) - Extract tip builds into a separate sub-project by [@​artspb](https://togithub.com/artspb) in [https://github.com/go-delve/delve/pull/3578](https://togithub.com/go-delve/delve/pull/3578) - pkg,service/dap: use switch instead of if-else-if by [@​alexandear](https://togithub.com/alexandear) in [https://github.com/go-delve/delve/pull/3576](https://togithub.com/go-delve/delve/pull/3576) - CI: update teamcity settings by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3579](https://togithub.com/go-delve/delve/pull/3579) - pkg/proc: use gore to obtain info from stripped binaries by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3577](https://togithub.com/go-delve/delve/pull/3577) - add missing import in TeamCity DSL by [@​artspb](https://togithub.com/artspb) in [https://github.com/go-delve/delve/pull/3580](https://togithub.com/go-delve/delve/pull/3580) - update TeamCity DSL version to 2023.05 and remove tip configurations from Aggregator by [@​artspb](https://togithub.com/artspb) in [https://github.com/go-delve/delve/pull/3581](https://togithub.com/go-delve/delve/pull/3581) - proc: fix TestIssue1101 flake by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3585](https://togithub.com/go-delve/delve/pull/3585) - proc: skip trapthread for harcoded breakpoints after manual stop by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3582](https://togithub.com/go-delve/delve/pull/3582) - tests: fix tests in go1.22 by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3583](https://togithub.com/go-delve/delve/pull/3583) - all: run go mod tidy by [@​prattmic](https://togithub.com/prattmic) in [https://github.com/go-delve/delve/pull/3589](https://togithub.com/go-delve/delve/pull/3589) - add all branches but PRs to filter by [@​artspb](https://togithub.com/artspb) in [https://github.com/go-delve/delve/pull/3590](https://togithub.com/go-delve/delve/pull/3590) - service/dap: fix close on closed channel panic by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3573](https://togithub.com/go-delve/delve/pull/3573) - Show pprof labels in thread names by [@​stefanhaller](https://togithub.com/stefanhaller) in [https://github.com/go-delve/delve/pull/3501](https://togithub.com/go-delve/delve/pull/3501) - \*: Use forked goretk/gore module by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3597](https://togithub.com/go-delve/delve/pull/3597) - proc: make some type casts less counterintuitive by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3596](https://togithub.com/go-delve/delve/pull/3596) - teamcity,version: add 1.22 to supported versions and CI matrix by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3601](https://togithub.com/go-delve/delve/pull/3601) - proc: fix ppc64 arch name check by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3608](https://togithub.com/go-delve/delve/pull/3608) - goversion: include pre-releases in VersionAfterOrEqual check by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3607](https://togithub.com/go-delve/delve/pull/3607) - service/dap: fix close on closed channel by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3609](https://togithub.com/go-delve/delve/pull/3609) - proc: fix TestPackageRenames on go1.22 on linux/386 by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3610](https://togithub.com/go-delve/delve/pull/3610) - \*: Update gore dep for 1.22 by [@​derekparker](https://togithub.com/derekparker) in [https://github.com/go-delve/delve/pull/3611](https://togithub.com/go-delve/delve/pull/3611) - \*: release version 1.22.0 by [@​aarzilli](https://togithub.com/aarzilli) in [https://github.com/go-delve/delve/pull/3606](https://togithub.com/go-delve/delve/pull/3606) #### New Contributors - [@​upils](https://togithub.com/upils) made their first contribution in [https://github.com/go-delve/delve/pull/3527](https://togithub.com/go-delve/delve/pull/3527) - [@​testwill](https://togithub.com/testwill) made their first contribution in [https://github.com/go-delve/delve/pull/3562](https://togithub.com/go-delve/delve/pull/3562) - [@​prattmic](https://togithub.com/prattmic) made their first contribution in [https://github.com/go-delve/delve/pull/3589](https://togithub.com/go-delve/delve/pull/3589) **Full Changelog**: go-delve/delve@v1.21.1...v1.22.0 **Curated Changelog**: https://github.com/go-delve/delve/blob/master/CHANGELOG.md#1220-2023-12-29 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/parca-dev/parca-agent). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjEuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyMS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
This commit splits expression evaluation into two parts. The first part (in
pkg/proc/evalop/evalcompile.go) "compiles" as ast.Expr into a list of
instructions (defined in pkg/proc/evalop/ops.go) for a stack machine
(defined by
proc.(*evalStack)
).The second part is a stack machine (implemented by
proc.(*EvalScope).eval
and
proc.(*EvalScope).evalOne
) that has two modes of operation: in themain mode it executes inteructions from the list (by calling
evalOne
), inthe second mode it executes the call injection protocol by calling
funcCallStep
repeatedly until it either the protocol finishes, needs moreinput from the stack machine (to set call arguments) or fails.
This approach has several benefits:
it is now possible to remove the goroutine we use to evaluate expression
and the channel used to communicate with the Continue loop.
every time we resume the target to execute the call injection protocol we
need to update several local variables to match the changed state of the
target, this is now done at the top level of the evaluation loop instead of
being hidden inside a recurisive evaluator
using runtime.Pin to pin addresses returned by an injected call would
allow us to use a more natural evaluation order for function calls, which
would solve some bugs function call injection has problems when the function call involves passing variables stored in registers #3310, allow users to inspect values returned by a
call injection Function call results cannot be printed via direct address expression syntax #1599 and allow implementing some other features Support map evaluation by composite literal keys #1465. Doing
this with the recursive evaluator, while keeping backwards compatibility
with versions of Go that do not have runtime.Pin is very hard. However after
this change we can simply conditionally change how compileFunctionCall works
and add some opcodes.