Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixed week/ endpoint that is not used on staging #10927

Conversation

ihorsokhanexoft
Copy link

@ihorsokhanexoft ihorsokhanexoft commented Jan 20, 2025

Purpose

Throughout the onboarding I got 500 error on _/metrics/query/node_analytics/m8dwk/week/ endpoint but not on staging.
Error I got before fix:

Fielddata is disabled on text fields by default. Set fielddata=true on [pageview_info.referer_domain] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.

Appropriate screenshot
Screenshot from 2025-01-21 11-41-03

After the fix at the bottom of this PR I was able to fix it
Locally (fixed):
Screenshot from 2025-01-21 11-40-05

Staging (https://staging.osf.io/g78bt/analytics):
Screenshot from 2025-01-21 11-37-43

It appears that on staging we don't make a request to that endpoint at all, thus analytics is never loaded, so potentially we may have the same issue on the staging

Changes

We had to add ".keyword" to each text field

Copy link
Contributor

@sh-andriy sh-andriy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good

@brianjgeiger brianjgeiger requested a review from aaxelb January 23, 2025 13:59
Comment on lines -504 to +522
'field': 'pageview_info.referer_domain',
'field': 'pageview_info.referer_domain.keyword',
'size': 10,
},
},
'popular-pages': {
'terms': {
'field': 'pageview_info.page_path',
'field': 'pageview_info.page_path.keyword',
'size': 10,
},
'aggs': {
'route-for-path': {
'terms': {
'field': 'pageview_info.route_name',
'field': 'pageview_info.route_name.keyword',
'size': 1,
},
},
'title-for-path': {
'terms': {
'field': 'pageview_info.page_title',
'field': 'pageview_info.page_title.keyword',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm... i worry this will break when deployed (and it isn't currently broken on prod or on staging)

referer_domain, page_path, route_name, and page_title should already be mapped as keyword fields -- this fix seems to indicate your local indexes were created using dynamic mapping (where text values automatically get a text field with .keyword subfield) instead of using the template that should have been created by manage.py sync_metrics

quick fix (if you're ok with losing data) might be to delete your osf_countedauthusage* indexes and run manage.py sync_metrics before adding more data

(ideally you'd get a warning or error trying to save metrics without up-to-date index templates, but osf-metrics doesn't have those safety rails...)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, it means we just don't do a request to this endpoint on the frontend side (because analytics never loads)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we do -- your screenshot on staging is for a private project ("Analytics are not available for private projects.") but on this public project analytics page i'm seeing a GET /_/metrics/query/node_analytics/a3rz4/week/ and charts loading

@ihorsokhanexoft ihorsokhanexoft deleted the fix/analytics branch January 29, 2025 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants