Scope stuck "connecting to scope"

[tomwilkie]

With this in the console:

VM245 vendors-b764d66….js?6ded3cd…:12 Uncaught TypeError: t.filter is not a function
    at f (VM245 vendors-b764d66….js?6ded3cd…:12)
    at i (VM245 vendors-b764d66….js?6ded3cd…:33)
    at f (VM245 vendors-b764d66….js?6ded3cd…:33)
    at l (VM245 vendors-b764d66….js?6ded3cd…:33)
    at VM245 vendors-b764d66….js?6ded3cd…:33
    at dispatch (VM245 vendors-b764d66….js?6ded3cd…:45)
    at VM245 vendors-b764d66….js?6ded3cd…:1
    at VM245 vendors-b764d66….js?6ded3cd…:33
    at Object.success (VM245 vendors-b764d66….js?6ded3cd…:13)
    at Reqwest.o.success._successHandler (VM245 vendors-b764d66….js?6ded3cd…:50)

[davkal]

Looks like requests to the backend are malformed when used in the service UI: http://hostname/api/api/app/{org}/api/topology

Notice the double api in the beginning.

Possible introduced by #2144

[2opremio]

This happened somewhere between 244bba2 and 29edaeb . I doubt this was the backend. My main suspect is #2144

@foot @davkal @jpellizzari @fbarl This is urgent, it broke dev (and then prod)

[2opremio]

Interestingly, I did test prod right after the upgrade and Scope was working. Could it be due to caching of the assets?

[davkal]

Interestingly, I did test prod right after the upgrade and Scope was working. Could it be due to caching of the assets?

That could have been masking the issue.

[2opremio]

So, IMO, we need to do three things to prevent this from happening in the future:

1. Fix the assets caching problem (if there´s any, I believe there is)
2. Add an integration test minimally testing the UI
3. Be more thorough when testing changes

[2opremio]

But let´s of course first figure out what the problem is.

[rade]

I thought we had some metrics/alerts for set up to catch this sort of breakage.

[2opremio]

I thought we had some metrics/alerts for set up to catch this sort of breakage.

Assuming the problem comes from the UI, I don´t think so.

[davkal]

We have monitoring in place, see https://github.com/weaveworks/monitoring/issues/101
but no alerting yet.

[rade]

weaveworks/monitoring/issues/101

exactly.

but no alerting yet.

Understood, but did this problem at least show up in the data?

[jpellizzari]

@rade This was neither a page-load or page-render error. The app was served successfully and the requested page was rendered. This is the result of a bad URL path resulting in index.html being served to a call that expects JSON. Our HTTP library decided it was a string and passed it to a .filter() call.

[2opremio]

This is the result of a bad URL path

Did you track down how the bad URL was generated?

[jpellizzari]

@2opremio Yes, it is a result of #2144. I added some extra logic so that scope will get the right URL path while running as a component, as well as an iFrame. There is one commit I was expecting to see on master that isn't there, so it's likely I forgot to push it before merging my PR.

[jpellizzari]

This was the commit I was referring to that never got pushed: 8ea7de9

[jpellizzari]

Fixed in #2159

[2opremio]

https://cloud.weave.works/demo/ is still broken after the fix. See:

Note how the url is wrong: https://cloud.weave.works/api/demo/api

[2opremio]

I reverted prod to 244bba2 but it's still reproducible at http://frontend.dev.weave.works/demo/

[davkal]

@jpellizzari to debug this, you can simply run npm start and then access http://localhost:4043/scoped/
This was specifically included in the express server to test path prefixes. (But maybe that should have been automated.)

At first glance it looks like this heuristic caused the problem:

const isStandalone = window.location.pathname === '/' || /\/(.+).html/.test(window.location.pathname);

Maybe better to detect the opposite (scope-as-component)?

[2opremio]

@awh discovered we are having the same (or similar problem) at /admin/scope in Weave Cloud. This is not covered by @davkal's hotfix.

For instance, at http://frontend.dev.weave.works/admin/scope/ I get:

Uncaught TypeError: t.filter is not a function
    at f (vendors-58f07ad….js?547653e…:12)
    at i (vendors-58f07ad….js?547653e…:33)
    at f (vendors-58f07ad….js?547653e…:33)
    at l (vendors-58f07ad….js?547653e…:33)
    at vendors-58f07ad….js?547653e…:33
    at dispatch (vendors-58f07ad….js?547653e…:45)
    at vendors-58f07ad….js?547653e…:1
    at vendors-58f07ad….js?547653e…:33
    at dispatch (vendors-58f07ad….js?547653e…:45)
    at Object.success (vendors-58f07ad….js?547653e…:13)

[2opremio]

It's the same problem (the UI is requesting http://frontend.dev.weave.works/api/admin/scope/api)

[jpellizzari]

#2202 Has been merged and pushed to dev and is working correctly.

[ngehani]

Cluster Health