Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add timestamps to evals #5881

Merged
merged 10 commits into from
Aug 7, 2019
Merged

add timestamps to evals #5881

merged 10 commits into from
Aug 7, 2019

Conversation

jazzyfresh
Copy link
Contributor

@jazzyfresh jazzyfresh commented Jun 24, 2019

Overview

Added timestamps to evaluations upon creation/modification.

Behavior

Screenshot from 2019-08-06 09-21-27
Screenshot from 2019-08-06 09-21-46

Implementation

The key here was to add timestamps to evals in the correct place, which would be where the eval is first registered as part of a raft transaction.

  • EvalUpdateRequest:
    • nomad/job_endpoint.go#BatchDeregister
    • nomad/job_endpoint.go#Deregister
    • nomad/job_endpoint.go#Dispatch
    • nomad/job_endpoint.go#Evaluate
    • nomad/job_endpoint.go#Register
    • nomad/job_endpoint.go#Plan
    • leader.go#reapDupBlockedEvaluations
    • leader.go#reapFailedEvaluations
    • periodic.go#DispatchJob
    • worker.go#CreateEval
    • worker.go#ReblockEval
    • worker.go#UpdateEval
  • AllocUpdateRequest:
    • node_endpoint.go#UpdateAlloc
    • node_endpoint.go#createNodeEvals
  • AllocStopRequest:
    • nomad/alloc_endpoint.go#Stop
  • state_store.go#nestedUpsertEval
  • deployment_watcher.go#getEval
  • AllocUpdateDesiredTransition
    • nomad/drainer/drainer.go#drainAllocs
  • ApplyPlanResultsRequestType
    • nomad/plan_apply.go#applyPlan
  • structs.go#NextRollingEval
  • structs.go#CreateBlockedEval
  • structs.go#CreateFailedFollowUpEval
  • mock.Eval

TODO

  • add pretty-printed timestamps to the CLI eval output
  • CreateTime and ModifyTime are equal if unmodified
  • naive lamport clock for sensible modify times

nomad/leader.go Outdated Show resolved Hide resolved
nomad/leader.go Outdated Show resolved Hide resolved
nomad/leader.go Outdated Show resolved Hide resolved
nomad/periodic.go Outdated Show resolved Hide resolved
nomad/worker.go Outdated Show resolved Hide resolved
nomad/worker.go Outdated Show resolved Hide resolved
@preetapan preetapan added 0.9.4 and removed 0.9.4 labels Jul 15, 2019
@jazzyfresh
Copy link
Contributor Author

jazzyfresh commented Jul 25, 2019

So the tests on my branch pass (travis-ci/push), but not the ones merged w/ master (travis-ci/pr). I thought it was a transient issue on master, so I'm rerunning and looking into it if its not.

UPDATE: yes, it was transient. build is passing now.

nomad/alloc_endpoint.go Outdated Show resolved Hide resolved
@jazzyfresh
Copy link
Contributor Author

Working on the description, which will include a walkthrough of the method call tracing and where is the correct place to set eval timestamps

@jazzyfresh jazzyfresh requested review from cgbaker and langmartin July 25, 2019 19:48
@jazzyfresh jazzyfresh marked this pull request as ready for review July 25, 2019 19:48
Copy link
Contributor

@cgbaker cgbaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like ModifyTime should initially be equal to CreateTime. If subsequentt calls to time.Now() mean they're not, this (imho) violates the principle of least surprise.

nomad/alloc_endpoint.go Outdated Show resolved Hide resolved
nomad/leader.go Outdated Show resolved Hide resolved
nomad/leader.go Outdated Show resolved Hide resolved
nomad/leader.go Outdated Show resolved Hide resolved
nomad/structs/structs.go Outdated Show resolved Hide resolved
Copy link
Member

@schmichael schmichael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay! This will be very helpful.

@@ -235,6 +235,7 @@ func (a *Alloc) Stop(args *structs.AllocStopRequest, reply *structs.AllocStopRes
return fmt.Errorf(structs.ErrUnknownAllocationPrefix)
}

now := time.Now().UTC().UnixNano()
eval := &structs.Evaluation{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Someday a helper func for making evals might be nice to ensure we always set the time properly:

func NewEval(namespace, type string) *Evaluation {
    // get time.now, generate uuid, and set basic fields here
}

Not a blocker for this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. This was discussed but I opted not to since a big refactor on all eval creations seemed risky and outside of the scope of this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too bad Go doesn't have constructors! 🐣

nomad/structs/structs.go Outdated Show resolved Hide resolved
// Format the evaluation data
basic := []string{
fmt.Sprintf("ID|%s", limit(eval.ID, length)),
fmt.Sprintf("Create Time|%s", formattedCreateTime),
fmt.Sprintf("Modify Time|%s", formattedModifyTime),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointing out that for old evals this will print an empty string, which may be ok but odd UX.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed it's not the greatest UX. This behavior is consistent with nomad alloc status, although it's unlikely at this point that there are old allocs w/o a CreateTime. Going to think about it, will most likely leave as is but add a note in the docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving it as is sounds good to me if there's prior art already that does this.

Copy link
Contributor

@preetapan preetapan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few more minor comments, otherwise LGTM :shipit:

@jazzyfresh jazzyfresh merged commit 1bdb111 into master Aug 7, 2019
@jazzyfresh jazzyfresh deleted the f-eval-timestamp branch August 7, 2019 16:50
@github-actions
Copy link

github-actions bot commented Feb 5, 2023

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants