api: implement fuzzy search API #10184

shoenig · 2021-03-16T00:52:32Z

This PR introduces the /v1/search/fuzzy API endpoint, used for fuzzy
searching objects in Nomad. The fuzzy search endpoint routes requests
to the Nomad Server leader, which implements the Search.FuzzySearch RPC
method.

Requests to the fuzzy search API are based on the api.FuzzySearchRequest
object, e.g.

{
  "Text": "ed",
  "Context": "all"
}

Responses from the fuzzy search API are based on the api.FuzzySearchResponse
object, e.g.

{
  "Index": 27,
  "KnownLeader": true,
  "LastContact": 0,
  "Matches": {
    "tasks": [
      {
        "ID": "redis",
        "Scope": [
          "default",
          "example",
          "cache"
        ]
      }
    ],
    "evals": [],
    "deployment": [],
    "volumes": [],
    "scaling_policy": [],
    "images": [
      {
        "ID": "redis:3.2",
        "Scope": [
          "default",
          "example",
          "cache",
          "redis"
        ]
      }
    ]
  },
  "Truncations": {
    "volumes": false,
    "scaling_policy": false,
    "evals": false,
    "deployment": false
  }
}

The API is configurable using the server.search stanza, e.g.

server {
  search {
    fuzzy_enabled   = true
    limit_query     = 200
    limit_results   = 1000
    min_term_length = 5
  }
}

These values can be increased or decreased, so as to provide more
search results or to reduce load on the Nomad Server. The fuzzy search
API can be disabled entirely by setting fuzzy_enabled to false.

nomad/search_endpoint.go

shoenig · 2021-03-16T15:25:17Z

I'll cover API docs and e2e tests in a followup PR. First I want to make sure @DingoEatingFuzz / @backspace are happy with the response format, of which a larger example is below.

edit: see API docs examples

tgross

This looks great @shoenig. The search/sorting algorithm is really clear and understandable. I've left some usability remarks and a few other odds and ends.

tgross · 2021-03-16T15:42:31Z

website/content/docs/configuration/search.mdx

+
+- `fuzzy_enabled` `(bool: true)` - Specifies whether the fuzzy search API is
+  enabled. If not enabled, requests to the fuzzy search API endpoint will return
+  an error response.


We should have a link here to the API docs for this new endpoint as well.

tgross · 2021-03-16T15:44:43Z

nomad/structs/search.go

@@ -0,0 +1,129 @@
+package structs


❤️ for splitting anything out of structs.go

tgross · 2021-03-16T15:50:54Z

nomad/search_endpoint.go

+	return s.srv.blockingRPC(&opts)
+}
+
+func expandContext(context structs.Context) []structs.Context {


Is the plan to factor this out of nomad/search_endpoint_ent.go as well?

allContexts is defined per the _oss and _ent files; this wrapper Just Works with either one

tgross · 2021-03-16T15:55:57Z

command/agent/search_endpoint_test.go


 		prefix := alloc.ID[:len(alloc.ID)-2]
 		data := structs.SearchRequest{Prefix: prefix, Context: structs.Allocs}
 		req, err := http.NewRequest("POST", "/v1/search", encodeReq(data))
-		assert.Nil(err)
+		require.NoError(t, err)


Thanks for taking the time to tidy all these existing tests up, btw.

tgross · 2021-03-16T18:42:05Z

nomad/search_endpoint.go

+		return fmt.Errorf("fuzzy search is not enabled")
+	}
+
+	// check the query term meets minimum length


The ACL check should be before argument validation so that we return ACL errors first. Probably before config validation too?

Yup, I agree. One thing though - to make namespace=* work, this up front ACL check is basically bypassed, instead defering to iterators wrapped with filters, doing the same ACL checks removing objects not allowed by the given token. So this is really more of a "at least try to give them an error if they specify a namespace they can't access" optimization.

tgross · 2021-03-16T18:50:57Z

nomad/search_endpoint.go

+	accumulateSet := func(limited bool, set map[structs.Context][]fuzzyMatch) {
+		for ctx, matches := range set {
+			for _, match := range matches {
+				if len(unsorted[ctx]) < limitResults {


This is going to sound nitpicky, but the results limit config talks about a maximum number of results, whereas this is getting us the maximum number of results per context. Should we update this so that we're tracking a count of how many results overall, or just change the docs? (Either seems reasonable.)

good catch; clarified in docs

tgross · 2021-03-16T18:53:19Z

command/agent/config.go

+		if b.Search.LimitQuery > 0 {
+			result.Search.LimitQuery = b.Search.LimitQuery
+		}
+		if b.Search.LimitResults > 0 {


Should we validate a relationship between LimitQuery and LimitResults, even if just to warn in the logs that it's unexpected to have LimitResults > LimitQuery?

It could actually be the case where the number of results exceeds LimitQuery - because of the way the jobs type is expanded into its subtypes (groups, tasks, services, images, commands, classes). There could be only 1 job registered, but contains more of any of those subtypes than LimitResults.

DingoEatingFuzz · 2021-03-17T18:17:01Z

Hey @shoenig, I annotated the mock response with my thoughts in this gist: https://gist.github.com/DingoEatingFuzz/27ac49ea722c4df5011f58b6207fa3ab

The tl;dr is we need to make sure that all the responses have all the requisite info to build links from.

DingoEatingFuzz · 2021-03-17T18:24:54Z

I poked at the code a bit but had trouble finding answers to my two questions so I'll just ask them:

Will this support cross-namespace searches (i.e., namespace=*)?
Will results include multi-part matches (e.g., bisl still matches command bin/sleep)?

backspace · 2021-03-22T21:07:23Z

re cross-namespace searches, it’s known to be desired 🤞

shoenig · 2021-04-12T14:47:33Z

Hey @DingoEatingFuzz and @backspace I tweaked the response objects to better identify the matching object. I've got API docs in there now with more query examples.

Will this support cross-namespace searches (i.e., namespace=*)?

It should, yes. With ACLs enabled, if namespace=*, a token with limited <kind>:read on one or more specified namespaces will only return results for the <kind> and namespaces in which the token is valid.

Will results include multi-part matches (e.g., bisl still matches command bin/sleep)?

Nope, at least not yet. The substring matches we're doing now are already going to use a questionable amount of resources, so this change includes server config to disable and/or limit queries. Once we get a better understanding of the performance impact maybe we can open up the matching patterns.

tgross · 2021-04-12T15:01:05Z

@shoenig I pulled this branch down locally and I'm not getting the results I'd expect to see with this endpoint. But maybe I'm missing something in the docs:

$ nomad job status
ID       Type     Priority  Status   Submit Date
example  service  50        running  2021-04-12T10:50:29-04:00

$ curl -vs -XPOST -d '{"Text": "ex", "Context": "jobs"}' "localhost:4646/v1/search/fuzzy" | jq .
...
> POST /v1/search/fuzzy HTTP/1.1
> Host: localhost:4646
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Length: 33
> Content-Type: application/x-www-form-urlencoded
>
} [33 bytes data]
* upload completely sent off: 33 out of 33 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< Vary: Accept-Encoding
< X-Nomad-Index: 17
< X-Nomad-Knownleader: true
< X-Nomad-Lastcontact: 0
< Date: Mon, 12 Apr 2021 14:53:48 GMT
< Content-Length: 89
<
{ [89 bytes data]
* Connection #0 to host localhost left intact
{
  "Index": 17,
  "KnownLeader": true,
  "LastContact": 0,
  "Matches": {},
  "Truncations": {
    "jobs": false
  }
}

$ curl -vs -XPOST -d '{"Text": "ex", "Context": "all"}' "localhost:4646/v1/search/fuzzy" | jq .
...
> POST /v1/search/fuzzy HTTP/1.1
> Host: localhost:4646
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Length: 32
> Content-Type: application/x-www-form-urlencoded
>
} [32 bytes data]
* upload completely sent off: 32 out of 32 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< Vary: Accept-Encoding
< X-Nomad-Index: 18
< X-Nomad-Knownleader: true
< X-Nomad-Lastcontact: 0
< Date: Mon, 12 Apr 2021 14:55:44 GMT
< Content-Length: 284
<
{ [284 bytes data]
* Connection #0 to host localhost left intact
{
  "Index": 18,
  "KnownLeader": true,
  "LastContact": 0,
  "Matches": {
    "scaling_policy": [],
    "evals": [],
    "deployment": [],
    "volumes": []
  },
  "Truncations": {
    "nodes": false,
    "plugins": false,
    "namespaces": false,
    "deployment": false,
    "volumes": false,
    "allocs": false,
    "jobs": false,
    "evals": false,
    "scaling_policy": false
  }
}

backspace · 2021-04-14T21:48:03Z

hmm is it possible your queries returned empty results, @tgross? I tried it and got some:

$ curl -XPOST -d '{"Text": "pi", "Context": "all"}' "localhost:4646/v1/search/fuzzy" | jq .

# …

{
  "Index": 30,
  "KnownLeader": true,
  "LastContact": 0,
  "Matches": {
    "evals": [],
    "deployment": [],
    "allocs": [
      {
        "ID": "ping🥳.webs🥳[0]",
        "Scope": [
          "default",
          "4ad14bd5-119b-5eaf-f36e-01d5a63515e0"
        ]
      },
      {
        "ID": "ping🥳.group with short-run task[0]",
        "Scope": [
          "default",
          "c62bff59-8583-391f-1f93-bc4037e0c403"
        ]
      },
      {
        "ID": "ping🥳.group with short-run task[0]",
        "Scope": [
          "default",
          "f0153309-6b01-38d4-6f78-0e769bbfc76e"
        ]
      }
    ],
    "commands": [
      {
        "ID": "ping",
        "Scope": [
          "default",
          "ping🥳",
          "webs🥳",
          "frontend🥳"
        ]
      },
      {
        "ID": "xxping",
        "Scope": [
          "default",
          "ping🥳",
          "group with short-run task",
          "short-run task"
        ]
      }
    ],
    "volumes": [],
    "scaling_policy": []
  },
  "Truncations": {
    "volumes": false,
    "allocs": false,
    "jobs": false,
    "plugins": false,
    "namespaces": false,
    "scaling_policy": false,
    "evals": false,
    "deployment": false,
    "nodes": false
  }
}

tgross · 2021-04-15T12:35:33Z

hmm is it possible your queries returned empty results, @tgross? I tried it and got some:

I'd run the example job from nomad job init first. But I just tried it again with the exact same job and got good results (see below). Doesn't seem to be flappy in any way either. So... I dunno, it's possible I tested against the wrong build or something the first time? That's embarrassing, sorry @shoenig 😊

$ curl -vs -XPOST -d '{"Text": "ex", "Context": "all"}' "localhost:4646/v1/search/fuzzy" | jq .
*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 4646 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 4646 (#0)
> POST /v1/search/fuzzy HTTP/1.1
> Host: localhost:4646
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Length: 32
> Content-Type: application/x-www-form-urlencoded
>
} [32 bytes data]
* upload completely sent off: 32 out of 32 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< Vary: Accept-Encoding
< X-Nomad-Index: 18
< X-Nomad-Knownleader: true
< X-Nomad-Lastcontact: 0
< Date: Thu, 15 Apr 2021 12:32:04 GMT
< Content-Length: 426
<
{ [426 bytes data]
* Connection #0 to host localhost left intact
{
  "Index": 18,
  "KnownLeader": true,
  "LastContact": 0,
  "Matches": {
    "volumes": [],
    "scaling_policy": [],
    "allocs": [
      {
        "ID": "example.cache[0]",
        "Scope": [
          "default",
          "8b8794e6-d2ae-870c-e017-a7b1de642ff7"
        ]
      }
    ],
    "jobs": [
      {
        "ID": "example",
        "Scope": [
          "default"
        ]
      }
    ],
    "evals": [],
    "deployment": []
  },
  "Truncations": {
    "volumes": false,
    "allocs": false,
    "nodes": false,
    "plugins": false,
    "namespaces": false,
    "evals": false,
    "scaling_policy": false,
    "jobs": false,
    "deployment": false
  }
}

backspace · 2021-04-15T15:35:18Z

Hey, I was doing some preliminary experimentation with using this in the UI and it looks like CORS isn’t available on /v1/search/fuzzy, that’ll be something we need 🤓💞

ETA specifically the pre-flight OPTIONS request 405ed:

backspace · 2021-04-15T16:56:31Z

Maybe I’m wrong? Maybe I’m missing something in local development 🤔 I’ll look more into this and let you know, sorry.

backspace · 2021-04-15T17:47:09Z

yes, I was wrong, disregard 😳

backspace · 2021-04-15T21:41:40Z

I don’t know that this needs to be blocking, but now that I’m trying this, case-insensitivity would be nice to have. Compy is named Tethys and searching for te doesn’t return a node, but Te does.

Probably not possible to include now but a future improvement could be returning the substring indices matched for bolding in the UI.

backspace · 2021-04-15T21:57:14Z

Something for jobs that’s required for the UI is knowing the name alongside the ID, would it be possible to add that in the scopes list?

Let me know if there’s a better place for these messages 🤔

shoenig · 2021-04-16T16:48:58Z

case-insensitivity would be nice to have

I think we can slip this in without much of a performance impact; I'll add it.

would it be possible to add that in the scopes list

Yup can definitely add Name. Currently the search is performed on Name, and then ID is used in the ID part of the return struct; which is confusing 🙂 . I'm thinking we should change it so that the search is performed on Name, the Name is used in the ID field, and the job ID is appended to the scope. That would be most consistent with how all the other context types are structured.

Let me know if there’s a better place for these messages

Here is great!

This PR introduces the /v1/search/fuzzy API endpoint, used for fuzzy searching objects in Nomad. The fuzzy search endpoint routes requests to the Nomad Server leader, which implements the Search.FuzzySearch RPC method. Requests to the fuzzy search API are based on the api.FuzzySearchRequest object, e.g. { "Text": "ed", "Context": "all" } Responses from the fuzzy search API are based on the api.FuzzySearchResponse object, e.g. { "Index": 27, "KnownLeader": true, "LastContact": 0, "Matches": { "tasks": [ { "ID": "redis", "Scope": [ "default", "example", "cache" ] } ], "evals": [], "deployment": [], "volumes": [], "scaling_policy": [], "images": [ { "ID": "redis:3.2", "Scope": [ "default", "example", "cache", "redis" ] } ] }, "Truncations": { "volumes": false, "scaling_policy": false, "evals": false, "deployment": false } } The API is tunable using the new server.search stanza, e.g. server { search { fuzzy_enabled = true limit_query = 200 limit_results = 1000 min_term_length = 5 } } These values can be increased or decreased, so as to provide more search results or to reduce load on the Nomad Server. The fuzzy search API can be disabled entirely by setting `fuzzy_enabled` to `false`.

shoenig · 2021-04-16T23:08:20Z

Those 2 additions should work now @backspace

$ curl -s -XPOST localhost:4646/v1/search/fuzzy -d '{"Context":"jobs", "Text": "yex"}' | jq .
{
  "Index": 17,
  "KnownLeader": true,
  "LastContact": 0,
  "Matches": {
    "jobs": [
      {
        "ID": "MyExampleJob",
        "Scope": [
          "default",
          "MyExampleJob"
        ]
      }
    ]
  },
  "Truncations": {
    "jobs": false
  }
}

In this case the JobID and JobName are the same but it should be correct

backspace · 2021-04-19T18:17:05Z

Total success, thanks!

tgross

LGTM. Let's ship it!

github-actions · 2022-11-24T02:21:59Z

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

shoenig force-pushed the f-fuzzy-search branch from e584a62 to da0a8ec Compare March 16, 2021 00:53

vercel bot deployed to Preview – nomad-storybook-and-ui March 16, 2021 00:53 View deployment

vercel bot deployed to Preview – nomad March 16, 2021 00:53 View deployment

drewbailey reviewed Mar 16, 2021

View reviewed changes

nomad/search_endpoint.go Show resolved Hide resolved

vercel bot deployed to Preview – nomad-storybook-and-ui March 16, 2021 14:47 View deployment

vercel bot temporarily deployed to Preview – nomad March 16, 2021 14:47 Inactive

shoenig force-pushed the f-fuzzy-search branch from 003eb84 to 15f190a Compare March 16, 2021 14:50

vercel bot temporarily deployed to Preview – nomad March 16, 2021 14:50 Inactive

vercel bot deployed to Preview – nomad-storybook-and-ui March 16, 2021 14:51 View deployment

shoenig marked this pull request as ready for review March 16, 2021 15:27

shoenig requested a review from tgross March 16, 2021 15:27

tgross requested changes Mar 16, 2021

View reviewed changes

shoenig force-pushed the f-fuzzy-search branch from 15f190a to 0c6f8f3 Compare April 5, 2021 18:55

vercel bot temporarily deployed to Preview – nomad April 5, 2021 18:55 Inactive

vercel bot deployed to Preview – nomad-storybook-and-ui April 5, 2021 18:55 View deployment

vercel bot temporarily deployed to Preview – nomad April 5, 2021 22:40 Inactive

vercel bot deployed to Preview – nomad-storybook-and-ui April 5, 2021 22:40 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui April 6, 2021 17:04 View deployment

vercel bot temporarily deployed to Preview – nomad April 6, 2021 17:04 Inactive

vercel bot deployed to Preview – nomad-storybook-and-ui April 6, 2021 17:10 View deployment

vercel bot temporarily deployed to Preview – nomad April 6, 2021 17:10 Inactive

shoenig force-pushed the f-fuzzy-search branch from 06a6437 to 0dff5cb Compare April 7, 2021 19:29

vercel bot deployed to Preview – nomad April 7, 2021 19:29 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui April 7, 2021 19:29 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui April 8, 2021 14:46 View deployment

vercel bot deployed to Preview – nomad April 8, 2021 14:46 View deployment

shoenig force-pushed the f-fuzzy-search branch from 8fc7668 to 6893aa9 Compare April 9, 2021 15:03

vercel bot deployed to Preview – nomad-storybook-and-ui April 9, 2021 15:03 View deployment

vercel bot deployed to Preview – nomad April 9, 2021 15:03 View deployment

shoenig requested a review from tgross April 12, 2021 14:39

shoenig added 2 commits April 16, 2021 16:36

api: make fuzzy searching case-agnostic

ab92667

shoenig force-pushed the f-fuzzy-search branch from 6893aa9 to ab92667 Compare April 16, 2021 22:56

vercel bot deployed to Preview – nomad-storybook-and-ui April 16, 2021 22:56 View deployment

vercel bot temporarily deployed to Preview – nomad April 16, 2021 22:56 Inactive

api: fuzzy search results include job name with id in scope

068fd43

vercel bot deployed to Preview – nomad-storybook-and-ui April 16, 2021 23:03 View deployment

vercel bot deployed to Preview – nomad April 16, 2021 23:03 View deployment

tgross approved these changes Apr 20, 2021

View reviewed changes

shoenig merged commit ec73d7e into main Apr 20, 2021

shoenig deleted the f-fuzzy-search branch April 20, 2021 15:06

apollo13 mentioned this pull request Aug 10, 2021

Searchbox in nomad web ui only queries for jobs in the current namespace #10101

Closed

github-actions bot locked as resolved and limited conversation to collaborators Nov 24, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api: implement fuzzy search API #10184

api: implement fuzzy search API #10184

shoenig commented Mar 16, 2021

shoenig commented Mar 16, 2021 •

edited

Loading

tgross left a comment

tgross Mar 16, 2021

shoenig Apr 9, 2021

tgross Mar 16, 2021

tgross Mar 16, 2021

shoenig Apr 9, 2021

tgross Mar 16, 2021

tgross Mar 16, 2021

shoenig Apr 9, 2021

tgross Mar 16, 2021

shoenig Apr 9, 2021

tgross Mar 16, 2021

shoenig Apr 9, 2021

DingoEatingFuzz commented Mar 17, 2021

DingoEatingFuzz commented Mar 17, 2021

backspace commented Mar 22, 2021

shoenig commented Apr 12, 2021

tgross commented Apr 12, 2021

backspace commented Apr 14, 2021

tgross commented Apr 15, 2021

backspace commented Apr 15, 2021 •

edited

Loading

backspace commented Apr 15, 2021

backspace commented Apr 15, 2021

backspace commented Apr 15, 2021

backspace commented Apr 15, 2021

shoenig commented Apr 16, 2021

shoenig commented Apr 16, 2021

backspace commented Apr 19, 2021

tgross left a comment

github-actions bot commented Nov 24, 2022

api: implement fuzzy search API #10184

api: implement fuzzy search API #10184

Conversation

shoenig commented Mar 16, 2021

shoenig commented Mar 16, 2021 • edited Loading

tgross left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DingoEatingFuzz commented Mar 17, 2021

DingoEatingFuzz commented Mar 17, 2021

backspace commented Mar 22, 2021

shoenig commented Apr 12, 2021

tgross commented Apr 12, 2021

backspace commented Apr 14, 2021

tgross commented Apr 15, 2021

backspace commented Apr 15, 2021 • edited Loading

backspace commented Apr 15, 2021

backspace commented Apr 15, 2021

backspace commented Apr 15, 2021

backspace commented Apr 15, 2021

shoenig commented Apr 16, 2021

shoenig commented Apr 16, 2021

backspace commented Apr 19, 2021

tgross left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 24, 2022

shoenig commented Mar 16, 2021 •

edited

Loading

backspace commented Apr 15, 2021 •

edited

Loading