Unable to view pipelines UI

[Hmr-ramzi]

Hello,

I am running into this issue while trying to get kubeflow 1.3 running on my kubernetes cluster. All components are deployed successfully (no errors shown in any pod), however, once i pipelines from the centraldashboard i get this error
upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER

I think this must be an envoy/istio error any idea what is going wrong? I am using istio version 1.9.3

[rc-coderepo]

check here

kubectl edit destinationrule -n kubeflow ml-pipeline
kubectl edit destinationrule -n kubeflow ml-pipeline-ui
And edited the spec.trafficPolicy.tls.mode section, changing its value from ISTIO_MUTUAL to DISABLE. The I could visit the pipelines section.

[Hmr-ramzi]

@rc-coderepo This will make the encryption disabled. We would like to keep using mode ISTIO_MUTUAL. Anyidea how to keep using it ?

[thunderboltsid]

Any updates on this? FYI @johnugeorge @deepak-muley

[rc-coderepo]

@thunderboltsid I have reinstalled kubeflow using v1.3.0 using kustiomize and I did not get this issue. I have installed it on a on-prem kubernetes

[GerardoGR]

I'm running version: 3dad839. and the suggestion provided by kubeflow/kubeflow#5271 (comment) worked for me but I needed to make sure to restart ALL the pods in the namespace: there was TLS error from the ml-pipeline component to mysql. Maybe that can also work for you @Hmr-ramzi ?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in one week if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been closed due to inactivity.

[andre-lx]

for anyone facing this issue, check this comment: kubeflow/pipelines#5244 (comment)

with this you don't need to change the istio destination rules from ISTIO_MUTUAL to DISABLE

[saumilsdk]

Here is how I fixed it for KF 1.4 and K8 1.21

[cateto]

ISTIO_MUTUAL -> DISABLE 

@saumilsdk
It works for me.
(manifests-1.5.0)
but why does it work?
I'm curious

[snps-sagarrao]

check here

kubectl edit destinationrule -n kubeflow ml-pipeline kubectl edit destinationrule -n kubeflow ml-pipeline-ui And edited the spec.trafficPolicy.tls.mode section, changing its value from ISTIO_MUTUAL to DISABLE. The I could visit the pipelines section.

This resolution fixed the same error message I was receiving in my Azure AKS's Kubeflow section Experiment (KFP) .