Getting a platform error when accessing to a location

[dpalomino]

I've only been able to reproduce this issue in this project in platform.

Steps to reproduce the error

1. Go to this project and try to open any of the locations
2. You get a 500 error.

Error log here.

I've tried with a some projects both in staging and production and I've been not able to reproduce it.

[dpalomino]

I think I've seen the cause, there is no questionnaire.

Don't know why because it was there when we created the project. Something related with a migration maybe?

[dpalomino]

Taking another look, the link to the questionnaire appears in the project details but it's been renamed to "None". I've been comparing with the original one (this) and it seems correct.

I'm guessing link with the attributes for each location has been lost (don't know the reason). Same error appears when downloading project data (either in xlsx or shp formats).

[oliverroick]

Here's the corresponding Opbeat log.

It looks like there is at least oneSpatialUnit instance where the type field is empty. type is a NOT NULL field, so I'm not sure why this field can be in the first place. We need to look into the database to see if there are more cases of this.

[amplifi]

@oliverroick That Opbeat log corresponds to the errors from another project (Odisha). That one is caused by a single location that has both a blank geometry and a blank type; not sure how that was possible to create. We should update the platform to disallow blank geometries/types, and also to handle it gracefully when there is an invalid location by still loading the others (an error message to the user would be nice, with a link to the location's details page to fix).

@dpalomino The original problem you linked above for Telangana is a known issue. Discussion here: https://cadasta.slack.com/archives/C0CLK7X2M/p1494950213988502 There was no decision made as to how to resolve; still needs discussion.

[dpalomino]

Thanks @amplifi! I didn't notice that.

But then all the geometries in that project are larger than 1MB? Because the problem appears when accessing to any of the locations in that project.

See video:
https://screencast.com/t/OFqY5cKwXuJG

Maybe we can catch up re this tomorrow. Thanks!

[amplifi]

@dpalomino Yes, any time you receive that error, the cause is the same.

[dpalomino]

Thanks @amplifi. But then it is really weird, because those locations were created long long time ago (September 2016) and were working fine. The last update in the project is 23 Sept 2016. Looking at the geometries:

They seem pretty simple. Clicking on any of the geometries showed in the screenshot returns a 500.

Any idea why this is happening? I cannot export the locations either (shp r xlsx), I get the same 500.

[amplifi]

@dpalomino Please read the Slack thread I linked in my first comment above -- it explains this issue in detail along with the fix. The stack traces in the error log you linked in your original post show that the platform caches JSON attributes when accessing a location. When that result is greater than 1MB, it fails to cache. The platform should handle this error gracefully and proceed to load the response without caching it first; that fix has yet to be implemented. Caching for JSON attributes wasn't rolled out until end of October, so this bug could not have occurred before then.

[dpalomino]

Ah thanks @amplifi, now I understand why it was working before and not now (with the deployment of caching for JSON in October).

Regarding the solution you propose:

if the geometry+attributes is > 1MB, this will trigger. but this is really more a lack of error handling/robustness in the code. we can modify the platform code so that if an object can’t be cached, it logs the error and proceeds to load the view anyway. the error just means it can’t store the object in the cache for next time, not that there’s any error in retrieving it from the database or serving it.

Do we have a rough estimation re the effort for implementing this? If it's not much I think we should try to include it in the next sprint.