-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Metrics for processing #57
Labels
Comments
Eventually it would be nice to also have size of image in pixels and size of image in bytes, so we can evaluate times relative to this, and evaluate them on eachother to evaluate compression rates. |
Might not be able to do comparisons between different events (computed, cached) in NewRelic. So probably better to have single
|
5 tasks
jonnor
added a commit
that referenced
this issue
Jul 20, 2016
This covers the basics needed to see total time, including that spent waiting in the queue. References #57
This was referenced Jul 27, 2016
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Right now we don't have any metrics apart from the ones provided by MsgFlo, which only tracks in/outports and durations. Because we don't use
error
ports, this does not even capture error rates.Info could look like this.
Job end2end time becomes completed-requested. Error rate is ratio of events with error == null and error != null. And we should be able to create statistics segmented on graph/runtime/client.
Ideally we'd also add the following, to be able to determine where time is spent.
So there state machine is.
-> requested (-> queued) -> started -> downloaded -> processed (-> cached) -> completed
. At each step there is also a possibility of going toerror
state.Event
ImgfloImageComputed
would be emitted by workers when they have completed processing an image. Cache hits would be a separate eventImgfloCacheHit
, emitted where we check cache (frontend and in worker before processing).Cache hit/miss ratio then becomes ratio of these two events.
This could be sent using NewRelic by default, though probably the interface used in the code should be metrics-provider agnostic.
The text was updated successfully, but these errors were encountered: