-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CT-2049] Fire an event at the end of every command #6878
Comments
Super agree with this. I suggest that we might want to have a CommandCompleted event that contains the data common to all commands, such as timing and success, and refer to an additional event that would contain more of the specific Result dump. We could do a "oneof" of all of the various Result objects, but the way they work in python protobufs is kind of kludgey and slow. We could even include the name of the Result event in the CommandCompleted event, if that would help. |
Discussed during BLG: This event shouldn't include the full result that would be returned from the command if invoked programmatically, à la: results, success = dbt.invoke(['<command>']) Rationale:
This event's
(Plus standard metadata of course - |
Will need to connect with the Execution Team for where we should fire this event. Possibly here? |
Per BLG -- 2/13/23 we'd probably want to do this in the preflight decorator as a generator. If the language team picks this ticket up feel free to ask for a pair from the Execution team, we'd be happy to show how/where that lives. |
For [CT-2049](#6878) we concluded that we wanted a new event type, [CommandCompleted](#6878 (comment)) with [four (4) values](#6878 (comment)): which command was run, whether the command succeeded, the timestamp that the command finished, and how long the command took. This commit adds the new event proto defition, the auto generated proto_types, and the instantiatable even type.
The [preflight decorator](https://github.com/dbt-labs/dbt-core/blob/4186f99b742b47e0e95aca4f604cc09e5c67a449/core/dbt/cli/requires.py#L19) runs at the start of every CLI invocation. Thus is a perfect candidate for emitting the CommandCompleted event. This is noted in the [dicussion on CT-2049](#6878 (comment)).
For [CT-2049](#6878) we concluded that we wanted a new event type, [CommandCompleted](#6878 (comment)) with [four (4) values](#6878 (comment)): which command was run, whether the command succeeded, the timestamp that the command finished, and how long the command took. This commit adds the new event proto defition, the auto generated proto_types, and the instantiatable even type.
The [preflight decorator](https://github.com/dbt-labs/dbt-core/blob/4186f99b742b47e0e95aca4f604cc09e5c67a449/core/dbt/cli/requires.py#L19) runs at the start of every CLI invocation. Thus is a perfect candidate for emitting the CommandCompleted event. This is noted in the [dicussion on CT-2049](#6878 (comment)).
* add protobuf message/class for new CommandCompleted event For [CT-2049](#6878) we concluded that we wanted a new event type, [CommandCompleted](#6878 (comment)) with [four (4) values](#6878 (comment)): which command was run, whether the command succeeded, the timestamp that the command finished, and how long the command took. This commit adds the new event proto defition, the auto generated proto_types, and the instantiatable even type. * begin emitting CommandCompleted event in the preflight decorator The [preflight decorator](https://github.com/dbt-labs/dbt-core/blob/4186f99b742b47e0e95aca4f604cc09e5c67a449/core/dbt/cli/requires.py#L19) runs at the start of every CLI invocation. Thus is a perfect candidate for emitting the CommandCompleted event. This is noted in the [dicussion on CT-2049](#6878 (comment)). * add CommandCompleted event to event unit tests * Add: changelog entry * fire CommandCompleted event reguardless of upstream exceptions Previously, if `--fail-fast` was specified and an issue was run into or an unhandled issue became an exception, the CommandCompleted event would not get fired because at this point in the stack we'd be in exception thrown handling mode. If an exception does reach this point, we want to still fire the event and also continue to propogate the exception. Hence the bare `raise` exists to reraise the caught exception * Update CommandCompleted event to be a `Debug` level event We don't actually "always" need this event to be logged. Thus we've updated it to `Debug` level. [Discussion Context](#7180 (comment))
On the
feature/click-cli
branch, each command explicitly returns a tuple ofsuccess: bool, results: Any
. (The structure ofresults
differs based on the command.)We should additionally fire an event that can be used programmatically to determine:
Ideally, this event would be fired always, even in cases where the command was interrupted (
^C
) or cancelled itself (e.g.--fail-fast
).The text was updated successfully, but these errors were encountered: