[FEATURE] More relevant approach to solving the non-responsive server problem

[zethuman]

Description

Since the solution in this PR was quick and merged as temporary, another more relevant and scalable solution was discussed. In this pull request, this approach was implemented. That is, in order to prevent other clients from having such problems, they can set up a healthcheck (following our example or as they wish) and embed a couple of lines to check the status of the container, if the container did not pass the healthcheck, then it makes no sense to conduct tests.

Issues Resolved

Closes [#296].

If all is well, I'd like to open issue in all clients and tweak this part. Waiting for approval...

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[Jakob3xD]

I didn´t want to hijack your PR, therefore I create a separate branch in my fork.
main...Jakob3xD:opensearch-go:feature/actions/healthcheck
I suggest checking the cluster connection with golang, as it is a golang project. Moreover I don't think it's a good idea to include actions from private user repos. To restart the broken container, I changed the healthcheck to kill it self and docker will automatically restart it.

[zethuman]

@Jakob3xD yes, it's certainly all great, we also did that. Since not only we have this problem, but all clients have it, so the solution should be simple and understandable, and most importantly, native for everyone. This is what we discussed here with @dblock.
Also, why do you say it's a private repo? This action is already published in the marketplace

[Jakob3xD]

Also, why do you say it's a private repo?

What I meant is, that it is under your user namespace and you are the owner and only maintainer of it. There is no community behind, that could continue working on it or change it if needed, as only you have access to change it.
So adding it to all clients would add an dependency to you.
Maybe dblock can add a repo under the opensearch scope where more users can contribute.

Maybe instead of a complex action we could just use the current loop used for the unreleased integration test.
for attempt in {1..20}; do sleep 5; if curl -s localhost:9200; then echo '=====> ready'; break; fi; echo '=====> waiting...'; done
With my self killing health check this could also work.

Beside that. I think it would be nice if lib tests just verify the cluster connection by them self if they want to, so they just need the container.

[dblock]

@Jakob3xD Make a PR with your proposal (if you haven't already) and let's compare to see what's simpler?

[zethuman]

@dblock do you mind if we will use the action that I have on my private repository? if this is important, then we can fork it under the opensearch

[dblock]

@dblock do you mind if we will use the action that I have on my private repository? if this is important, then we can fork it under the opensearch

I personally don't think it's a problem at all, but you should make a release.

[dblock]

Picking #316 over this one. Thank you so much for working through this @zethuman and @Jakob3xD.