api: add related evals to eval details #12305

lgfa29 · 2022-03-16T01:47:21Z

The related query param is used to indicate that the request should
return a list of related (next, previous, and blocked) evaluations.

Closes #11858

lgfa29 · 2022-03-16T01:50:57Z

nomad/state/state_store.go

+			continue
+		}
+
+		eval, err := s.EvalByID(ws, current)


This call will add the eval watch channel to the watchset, which means we can have multiple channels in ws. I don't know if there are any performance implications in this approach.

A alternative would be to set the watchset using the job ID prefix. This would use a single watch channel, but may fire in situations where an eval for another with the same prefix changes.

lgfa29 · 2022-03-16T02:00:16Z

nomad/structs/structs.go

+type EvaluationStub struct {
+	ID                string
+	Namespace         string
+	Priority          int
+	Type              string
+	TriggeredBy       string
+	JobID             string
+	NodeID            string
+	DeploymentID      string
+	Status            string
+	StatusDescription string
+	WaitUntil         time.Time
+	NextEval          string
+	PreviousEval      string
+	BlockedEval       string
+	CreateIndex       uint64
+	ModifyIndex       uint64
+	CreateTime        int64
+	ModifyTime        int64
+}


This list is fairly arbitrary and will be mostly dictated by the UI. Status and StatusDescription are probably the most important right now, but since this is only returned in an opt-in request I added most of the fields.

Using an Evalution directly in RelatedEvals caused a recursive issue while decoding the struct.

just making a notes of the stuff left out:

- JobModifyIndex - NodeModifyIndex - Wait - FailedTGAllocs - ClassEligibility - EscapedComputedClass - QuotaLimitReached - AnnotatePlan - QueuedAllocations - SnapshotIndex

lgfa29 · 2022-03-16T02:02:45Z

nomad/structs/structs.go

@@ -10586,6 +10587,10 @@ type Evaluation struct {
 	// to constraints or lacking resources.
 	BlockedEval string

+	// RelatedEvals is a list of all the evaluations that are related (next,
+	// previous, or blocked) to this one. It may be nil if not requested.
+	RelatedEvals []*EvaluationStub


Originally this was a map, but a list would be a little easier for the UI to consume.

I ran into a similar kerfuffle when working on fuzzy search, we should probably have some kind of methodology for agreeing on a schema before starting implementation work

I think the biggest problem for me is that I don't fully understand the UI data layer 😅

I wrote some questionable JS code to validate this endpoint and a list seemed to align better with how data is loaded into the UI. But I will validate this with someone that knows what they are doing.

shoenig

LGTM; just some thoughts

shoenig · 2022-03-16T14:17:20Z

nomad/state/state_store.go

+		todo = append(todo, eval.RelatedIDs()...)
+		relatedEvals = append(relatedEvals, eval.Stub())
+		done[eval.ID] = true
+	}


Currently the only way out of this loop is to scan all related evals (or experience an error). Is there a way to limit the scanning we do, like on the size of the accumulated relatedEvals or even just a hard stop at like 20?

Hum...good question. I think the problem with setting an upper bound is that the response then becomes incomplete, and the idea of this flag is to give all the data needed quickly. But guarding against potential infinite loops, or slow requests sounds like a good idea. I don't know what a good number would be, so perhaps we can have a time limit, like 1s?

Maybe the most correct thing to do would be to plumb the Request.Context all the way down here. (You could make the same argument for all calls into the state store, it's just that this one feels less strongly bounded than the others)

Ah good point. Is this the channel I should be listening for connection close? https://pkg.go.dev/github.com/hashicorp/yamux#Session.CloseChan

I think we want the http request context

https://pkg.go.dev/net/http#Request.Context

stemming from the request handler http.Request

https://github.com/hashicorp/nomad/blob/main/command/agent/eval_endpoint.go#L10

Ah ok. Unfortunately that's currently not possible 😬
#7119

Whelp so much for that! It's probably fine as-is anyway, I mean how many related evals could there be anyway ... 🤔

fwiw, another endpoint returns all the evaluations for a job in one go, so this is strictly less than that... can't be too bad, right?

Ah yeah, good point. One of the differences here is that we have a (potentially) infinite loop if the todo list never gets to zero. But I think we're safe because we always pop and check if eval has been processed already.

shoenig · 2022-03-16T14:18:46Z

nomad/structs/structs.go

@@ -10586,6 +10587,10 @@ type Evaluation struct {
 	// to constraints or lacking resources.
 	BlockedEval string

+	// RelatedEvals is a list of all the evaluations that are related (next,
+	// previous, or blocked) to this one. It may be nil if not requested.
+	RelatedEvals []*EvaluationStub


I ran into a similar kerfuffle when working on fuzzy search, we should probably have some kind of methodology for agreeing on a schema before starting implementation work

The `related` query param is used to indicate that the request should return a list of related (next, previous, and blocked) evaluations. Co-authored-by: Jasmine Dahilig <[email protected]>

github-actions · 2022-10-25T02:45:12Z

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

lgfa29 added this to the 1.3.0 milestone Mar 16, 2022

lgfa29 commented Mar 16, 2022

View reviewed changes

lgfa29 force-pushed the f-api-related-evals branch from cd041a1 to fe52573 Compare March 16, 2022 01:52

vercel bot deployed to Preview – nomad-storybook-and-ui March 16, 2022 01:52 View deployment

vercel bot deployed to Preview – nomad March 16, 2022 01:52 View deployment

lgfa29 force-pushed the f-api-related-evals branch from fe52573 to 9d48187 Compare March 16, 2022 01:57

vercel bot deployed to Preview – nomad March 16, 2022 01:57 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui March 16, 2022 01:57 View deployment

lgfa29 commented Mar 16, 2022

View reviewed changes

lgfa29 added a commit that referenced this pull request Mar 16, 2022

changelog: add entry for #12305

a835636

vercel bot temporarily deployed to Preview – nomad March 16, 2022 02:01 Inactive

vercel bot deployed to Preview – nomad-storybook-and-ui March 16, 2022 02:01 View deployment

lgfa29 requested review from jazzyfresh, schmichael and shoenig March 16, 2022 02:01

lgfa29 commented Mar 16, 2022

View reviewed changes

shoenig approved these changes Mar 16, 2022

View reviewed changes

jazzyfresh approved these changes Mar 17, 2022

View reviewed changes

lgfa29 and others added 3 commits March 17, 2022 11:13

api: add related evals to eval details

9073525

The `related` query param is used to indicate that the request should return a list of related (next, previous, and blocked) evaluations. Co-authored-by: Jasmine Dahilig <[email protected]>

docs: add related query param to eval details endpoint

60f864e

changelog: add entry for #12305

c8a690f

lgfa29 force-pushed the f-api-related-evals branch from a835636 to c8a690f Compare March 17, 2022 17:07

vercel bot temporarily deployed to Preview – nomad March 17, 2022 17:07 Inactive

vercel bot deployed to Preview – nomad-storybook-and-ui March 17, 2022 17:07 View deployment

lgfa29 merged commit 81687c1 into main Mar 17, 2022

lgfa29 deleted the f-api-related-evals branch March 17, 2022 17:56

github-actions bot locked as resolved and limited conversation to collaborators Oct 25, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api: add related evals to eval details #12305

api: add related evals to eval details #12305

lgfa29 commented Mar 16, 2022

lgfa29 Mar 16, 2022

lgfa29 Mar 16, 2022

jazzyfresh Mar 17, 2022

lgfa29 Mar 16, 2022

shoenig Mar 16, 2022

lgfa29 Mar 16, 2022

shoenig left a comment

shoenig Mar 16, 2022

lgfa29 Mar 16, 2022

shoenig Mar 16, 2022

lgfa29 Mar 16, 2022

shoenig Mar 16, 2022

lgfa29 Mar 16, 2022

shoenig Mar 16, 2022

jazzyfresh Mar 16, 2022

lgfa29 Mar 16, 2022

shoenig Mar 16, 2022

github-actions bot commented Oct 25, 2022

api: add related evals to eval details #12305

api: add related evals to eval details #12305

Conversation

lgfa29 commented Mar 16, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shoenig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 25, 2022