Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] The Search API (in Version 3.0.0) is not working without setting an Index #470

Closed
dom192 opened this issue Feb 13, 2024 · 10 comments · Fixed by #471
Closed

[BUG] The Search API (in Version 3.0.0) is not working without setting an Index #470

dom192 opened this issue Feb 13, 2024 · 10 comments · Fixed by #471
Labels
bug Something isn't working

Comments

@dom192
Copy link

dom192 commented Feb 13, 2024

What is the bug?

It seems, that it is not possible to call the Search function of Client:

// Search executes a /_search request with the optional SearchReq
func (c Client) Search(ctx context.Context, req *SearchReq) (*SearchResp, error)

without setting Indices in the SearchReq.

How can one reproduce the bug?

Call the Search function without specifying Indices

	content := strings.NewReader(
	`	{
		"query": {
		  "match_all": {}
		}
	  }`)

	resp, err := client.Search(
		ctx,
		&opensearchapi.SearchReq{
			Indices: []string{},
			Body:    content,
		},
	)

What is the expected behavior?

The expected result should be no error and a valid opensearch result, but instead I get an error:
error msg: failed to json unmarshal body: invalid character '<' looking for beginning of value
The root cause of the problem is getting a http status code 400 Bad Request with http body:
"<html>\r\n<head><title>400 Bad Request</title></head>\r\n<body>\r\n<center><h1>400 Bad Request</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n"
Which is not handled right. The code tries to json.Unmarshal the html and generates the misleading error message.

What is your host/environment?

Ubuntu 22.04.3 LTS
API version 3 (github.com/opensearch-project/opensearch-go/v3/opensearchapi)

Do you have any screenshots?

no

Do you have any additional context?

I debugged the issue and found the root cause I think. The problem is the GetRequest function:
image
In case of no Indices is set, path is set to _search

path = "_search"

this causes the Bad Request response. To fix this I appended a "/" to _search:

path = "/_search"

after that the everything worked like expected :)
@dom192 dom192 added bug Something isn't working untriaged labels Feb 13, 2024
@Jakob3xD
Copy link
Collaborator

Tanks for reporting this issue.
Yes, the path should be prefixed with / .

Can you please tell what you client looks like? Are you using AWS Signing or something like that, as I can not reproduce it with our integration tests and I want to adjust them so it does not happen again.

@Jakob3xD Jakob3xD mentioned this issue Feb 14, 2024
Merged
@dom192
Copy link
Author

dom192 commented Feb 15, 2024

Tanks for reporting this issue. Yes, the path should be prefixed with / .

Can you please tell what you client looks like? Are you using AWS Signing or something like that, as I can not reproduce it with our integration tests and I want to adjust them so it does not happen again.

Thanks for the fast response. I tried it out in a local test setup:
local Kubernetes clusters with kind using basic auth

	return v3api.NewClient(
		v3api.Config{
			Client: osV3.Config{
				Username:  config.OS_USER,
				Password:  config.OS_PASSWORD,
				Addresses: []string{config.OS_URL},

				Transport: &http.Transport{
					TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
				},
			},
		})

@Jakob3xD
Copy link
Collaborator

Hm, weird. I can't seem to reproduce the issue with the integration test. I did the same requests and tried to modify the client address but it does not affect it. If I understand the URL lib correctly it should not matter if the leading slash is missing. I tried looking into this for a few hours but I don't want to dig through the net/http lib to much.

It there any chance you can find where this issue is exactly coming from so we can test it as I am not able to reproduce it.

@dom192
Copy link
Author

dom192 commented Feb 16, 2024 •
edited
Loading

The error happens on the server side. are you testing against a real endpoint? When the "/" is missing the server responds with statuscode 400.

@Jakob3xD
Copy link
Collaborator

The error happens on the server side. are you testing against a real endpoint? When the "/" is missing the server responds with statuscode 400.

I am testing against a real opensearch container. Running make cluster.build cluster.start and make test-integ.
It looks like you have a nginx in between your setup. Can you see how the requests looks like when it reaches the nginx by looking at the nginx logs?

I am still not getting where exactly the missing / is causing the 400 as it is present in the URL when sending the requests. If it was missing the request would not work at all as your-opensearch.com_search is not a valid domain.

@dom192
Copy link
Author

dom192 commented Feb 22, 2024

You are probably right with the nginx, but I have no access to it to see the logs. Will ask for access and come back to you soon.

@wbeckler
Copy link

Maybe you can get the requests using the Logger functionality in the client?

@dom192
Copy link
Author

dom192 commented Mar 12, 2024

Thanks for the hint with the logger within the client. I tried it out and this is my result.
Result without my fix (path = "_search"):

2024-03-12T09:55:37Z POST https://myurl/_search [status:400 request:183ms] 
< <html>
< <head><title>400 Bad Request</title></head>
< <body>
< <center><h1>400 Bad Request</h1></center>
< <hr><center>nginx</center>
< </body>
< </html>

Result with my fix (path = "/_search"):

2024-03-12T09:50:32Z POST https://myurl/_search [status:200 request:1.082s]
< {"took":45,"timed_out":false,"_shards":{"total":659,"successful":659,"skipped":658,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}}

The strange thing is, that the POST call printed out by the logger looks equal!

@dom192
Copy link
Author

dom192 commented Mar 13, 2024

Hi again,
today a checked with a colleague who has more rights than me ;)
We figured out , that a direct access to the opensearch service with port 9200 works fine in both ways. In the case with a NGINX Ingress between it is only working with path = "/_search". I found also in other opensearch api function that there is a leading "/". Would be great if you could fix that :)

@dblock
Copy link
Member

dblock commented Mar 13, 2024

@dom192 want to try to make a fix yourself? start by writing a unit test that shows the problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix search request missing a slash when no indices are given Jakob3xD/opensearch-go
4 participants
@dblock @wbeckler @dom192 @Jakob3xD