master not discovered yet, this node has not previously joined a bootstrapped #68
Comments
ypogeios
added
bug
ypogeios
changed the title
|
Hi @mprimeaux I am also facing this same issue in helm charts. I have tried followed YOUR comment in PR #54 below is the way I have configured in values.yaml, by setting the major version to "7" instead of "" and set the values of cluster initial master nodes & discovery seed hosts But I am getting the below response in logs, master-0 pod logs master-1 pod logs, master-2 pod logs, Below is the response I got when I exec into the pod, Please correct me if I am doing anything wrong. Attached the values.yaml for reference Thanks, |
|
@GaneshbabuRamamoorthy Hi there. I am reviewing your |
1 task
|
Hi @mprimeaux Yes I tried comment out the imageTag and redeployed. Below is the responses I got from log pods, master-0 pod logs master-1 pod logs master-2 pod logs, Attaching the helm package which I tried with values.yaml Please let me know if I am doing anything wrong. Thanks, |
|
Hi @mprimeaux I have tested the same in my other k8s environment where it was working fine without any issues. I was able to see the cluster health status. This is my k8s nodes details where its working fine and without any issues and when I tested the same in other k8s environment with IPv6 cluster I am getting SSL error and below is the k8s node details, and below is the SSL error I am getting in the IPv6 k8s environment pod logs, master-1 IPv6 is resolving but followup connection with master-0 is failing bcs of the Ipv4 address (127.0.0.1) and this is the same behaviour for master-2 when its trying to talk to master-0 also I tried netstat inside the master-0 pod below is the response and its trying to connect IPv4 Is this errors were due to the k8s cluster setup with IPv6? Is opensearch supported with IPv6? please share your thoughts. Thanks, |
Not sure about upstream Opensearch, but this chart doesn't. You'll need to use dual stack or IPv4 for now. |
|
Thanks, @smlx. I was just about to reply with the dual stack option. FWIW, we currently just use single stack (IPv4). |
|
Thanks for your response. So I tried deploying opendistro helm chart in the same IPv6 k8s cluster and I am getting the below responses in pod logs pod status were showing as running below is the logs, data pod logs master pod logs, client pod logs checked the cluster status inside the master pod and below is the response I got, Attaching the opendistro-es helm which I tried in my IPv6 environment and the same was working in IPv4 k8s cluster. Does opendistro elasticsearch helm chart not supported IPv6? Please share your thoughts. Thanks, |
No, it currently does not. |
|
the |
|
This appears to be an issue with auto-discovery in OpenSearch. Not sure if it is addressed in the latest release of OpenSearch but not an issue with the helm chart IMO. |
|
Something to do with #141 ? |
What does this mean? We are using opendistro 7.10.2 and seeing this issue. @ashish1099 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Deployment Error
On a Kubernetes Cluster Version 1.21.3 i tried to deploy OpenSearch with Helm.
All pods deploy.
When i run
kubectl exec -it opensearch-cluster-master-0 -- /bin/bashi get at first the message: Defaulted container "opensearch" out of: opensearch, fsgroup-volume (init)
After that im in the Pod and i run
curl -XGET https://localhost:9200 -u 'admin:admin' --insecurewhich gives following output: OpenSearch Security not initialized.[opensearch@opensearch-cluster-master-0 ~]$
The logs of Pods after command
kubectl logs opensearch-cluster-master-0[2021-09-27T12:11:26,745][WARN ][o.o.c.c.ClusterFormationFailureHelper] [opensearch-cluster-master-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and [cluster.initial_master_nodes] is empty on this node: have discovered [{opensearch-cluster-master-0}{ghk7yaqUTmu3zU4qS3HEEQ}{SGHYqXbrRMS4EsLfPdNDkQ}{172.16.73.110}{172.16.73.110:9300}{dimr}, {opensearch-cluster-master-2}{HwXoAsOuQBescAuj1Iglbw}{IYGWumyjQquN8GOOA8tZZw}{172.16.149.176}{172.16.149.176:9300}{dimr}, {opensearch-cluster-master-1}{I6xUeOjVScuzp4lhdMMgdg}{qIJxNrfXTdG0oCqTYKI2pg}{172.16.100.221}{172.16.100.221:9300}{dimr}]; discovery will continue using [172.16.100.221:9300, 172.16.149.176:9300] from hosts providers and [{opensearch-cluster-master-0}{ghk7yaqUTmu3zU4qS3HEEQ}{SGHYqXbrRMS4EsLfPdNDkQ}{172.16.73.110}{172.16.73.110:9300}{dimr}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[2021-09-27T12:11:26,841][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-cluster-master-0] Exception while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
org.opensearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.opensearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:203) ~[opensearch-1.0.0.jar:1.0.0]
at org.opensearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:189) ~[opensearch-1.0.0.jar:1.0.0]
at org.opensearch.action.get.TransportMultiGetAction.doExecute(TransportMultiGetAction.java:72) ~[opensearch-1.0.0.jar:1.0.0]
at org.opensearch.action.get.TransportMultiGetAction.doExecute(TransportMultiGetAction.java:53) ~[opensearch-1.0.0.jar:1.0.0]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:192) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:141) [opensearch-index-management-1.0.0.0.jar:1.0.0.0]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:190) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:99) [opensearch-performance-analyzer-1.0.0.0.jar:1.0.0.0]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:190) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:234) [opensearch-security-1.0.0.0.jar:1.0.0.0]
at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:154) [opensearch-security-1.0.0.0.jar:1.0.0.0]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:190) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.action.support.TransportAction.execute(TransportAction.java:168) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.action.support.TransportAction.execute(TransportAction.java:96) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:99) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:88) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:428) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.client.support.AbstractClient.multiGet(AbstractClient.java:546) [opensearch-1.0.0.jar:1.0.0]
at org.opensearch.security.configuration.ConfigurationLoaderSecurity7.loadAsync(ConfigurationLoaderSecurity7.java:211) [opensearch-security-1.0.0.0.jar:1.0.0.0]
at org.opensearch.security.configuration.ConfigurationLoaderSecurity7.load(ConfigurationLoaderSecurity7.java:102) [opensearch-security-1.0.0.0.jar:1.0.0.0]
at org.opensearch.security.configuration.ConfigurationRepository.getConfigurationsFromIndex(ConfigurationRepository.java:375) [opensearch-security-1.0.0.0.jar:1.0.0.0]
at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration0(ConfigurationRepository.java:321) [opensearch-security-1.0.0.0.jar:1.0.0.0]
at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration(ConfigurationRepository.java:306) [opensearch-security-1.0.0.0.jar:1.0.0.0]
at org.opensearch.security.configuration.ConfigurationRepository$1.run(ConfigurationRepository.java:166) [opensearch-security-1.0.0.0.jar:1.0.0.0]
at java.lang.Thread.run(Thread.java:832) [?:?]
The text was updated successfully, but these errors were encountered: