Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update argument scores #8

Merged
merged 5 commits into from
Jan 7, 2025
Merged

Update argument scores #8

merged 5 commits into from
Jan 7, 2025

Conversation

nickkeesG
Copy link
Collaborator

Split getArgumentScores into one for each score. Each score calculation ONLY considers users who voted AND have a relevant ingroup/outgroup. This should fix the issue we had with unusually low scores.

@sofvanh

Copy link

netlify bot commented Dec 10, 2024

Deploy Preview for mindmeldtool ready!

Name Link
🔨 Latest commit 84f1b55
🔍 Latest deploy log https://app.netlify.com/sites/mindmeldtool/deploys/677cb3deca45dd000824fa24
😎 Deploy Preview https://deploy-preview-8--mindmeldtool.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 48 (🟢 up 5 from production)
Accessibility: 89 (no change from production)
Best Practices: 92 (no change from production)
SEO: 100 (no change from production)
PWA: 60 (no change from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

@sofvanh
Copy link
Owner

sofvanh commented Dec 18, 2024

@nickkeesG Something seems wrong with the calculation of Fragmentation. A lot of the nodes that should score high in fragmentation are showing "Not enough data to calculate scores".

CEALLAR Dec 4 on this branch:
Screenshot 2024-12-18 at 15 31 58

CEALLAR Dec 4 on main:
Screenshot 2024-12-18 at 15 31 53

@nickkeesG
Copy link
Collaborator Author

There are two changes which need to happen in the ArgumentScoreHandler:

  1. I made a mistake in getFragmentationScore. The final return statement should be multiplied by two
    return 2 * fragmentationSum / uniquenessSum // normalize score to range 0 - 1

  2. Currently, a node can only get a fragmentation score if it ALSO gets a consensus score. I recommend we allow for nodes to receive scores, even if the other one isn't computable yet. There's an if statement at the end of argumentScoreHandler which currently requires neither score be null.

This is what is currently causing the graph to look way less orange than it should.

@sofvanh
Copy link
Owner

sofvanh commented Jan 6, 2025

I'll think about how to deal with this - probably we'll show frag/cons even if the other one isn't being calculated.

What happened here is that the nodes that don't show a score on the branch were mostly voted on by people who are friends with one another, so the nodes have a fragmentation score but not a consensus score.

@sofvanh
Copy link
Owner

sofvanh commented Jan 7, 2025

Solution: Allow consensus and fragmentation to be undefined, in which case show "More data needed" in score info.

Screenshot 2025-01-07 at 17 52 51

@sofvanh
Copy link
Owner

sofvanh commented Jan 7, 2025

Notice how with the new scoring algorithm we get more bright yellows!

Add "2 *" to getFragmentationScore
Change `Score` type to allow undefined Consensus and Fragmentation
@sofvanh sofvanh merged commit 5599fdd into main Jan 7, 2025
4 checks passed
@sofvanh sofvanh deleted the updateArgumentScores branch January 7, 2025 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants