Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SCT-3097 - fix clustering viewer pop ups, timeouts, tab crash with large clusters #2385

Open
wants to merge 73 commits into
base: develop
Choose a base branch
from

Conversation

eapearson
Copy link
Contributor

@eapearson eapearson commented Jul 25, 2021

Fix clustering App causing persistent pop-ups in Narratives, viewing output stalls Narrative

This fixes behavior in kbaseExpressionFeatureClusters, the top level viewer, and kbaseExpressionPairwiseCorrelation, a viewer for a tab view, in which large clusters can stall and even crash the browser with long data fetches and processing and error pop-ups, as reported by a user and confirmed.

These are two distinct issues.

The first, the stalling and crashing, was due to attempting to create heat maps for large clusters, greater than about 200 genes.

The second, browser pop-ups, was due to changes in the KBaseGenomes.Feature workspace type between version 2.X and 3.0, in

The changes consisted of those distinct changes, and other changes in those and related files to enable unit testing.

First, in kbaseExpressionFeatureClusters, the kbaseExpressionPairwiseCorrelation will not be accessible with > 200 genes. This was accomplished by disabling the button which displayed the pairwise correlation heatmap tab. Not much point allowing a user to add a new tab which just displays a warning

Second, kbaseExpressionPairwiseCorrelation will just show a warning with > 200 genes (even though the button to access it is disabled). With > 50, <= 200 genes the heatmap is generated but not placed in the browser (this is a modification of existing behavior, which did not have the <= 200 cutoff), but is available for download. Below 50 genes the heatmap is generated and displayed.

Jira Ticket / Issue

Testing Instructions

  • Details for how to test the PR:

There are new tests for kbaseExpressionPairwiseCorrelation and related sub-widgets (error, loading). These test include test data, derived from CI, which invoke the general conditions. Test coverage is pretty high, 100% for the subwidgets, and about 85% for the viewer widget. The untested code consists primarily of that which enables downloading of a heatmap when it cannot be rendered in the browser.

  • Tests pass locally and in GitHub Actions
  • Changes available by spinning up a local narrative and navigating to X to see Y

Dev Checklist:

  • My code follows the guidelines at https://sites.google.com/lbl.gov/trussresources/home?authuser=0
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • [-] I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • [-] Any dependent changes have been merged and published in downstream modules
  • (JavaScript) I have run Prettier and ESLint on changed code manually or with a git precommit hook
  • [-] (Python) I have run Black and Flake8 on changed Python code manually or with a git precommit hook

Updating Version and Release Notes (if applicable)

briehl and others added 22 commits April 21, 2021 12:24
… size and the butchering of bootstrap styling.
…le missing feature type; minor code improvements
… by the table; provide explanation for heatmap download when genes > 50; disable heatmap for pairwise when > 200 genes (not invoked now since the button won't be displayed in this case.)
…etID to get precise object ref - previously version was not included.
…separate out helper widgets, each with tests.
@eapearson
Copy link
Contributor Author

I'll be iterating next on the code quality reporting, will ping when ready for review.

@codecov
Copy link

codecov bot commented Jul 25, 2021

Codecov Report

Merging #2385 (a0a0166) into develop (86c5af6) will increase coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2385      +/-   ##
===========================================
+ Coverage    73.25%   73.26%   +0.01%     
===========================================
  Files           36       36              
  Lines         3903     3901       -2     
===========================================
- Hits          2859     2858       -1     
+ Misses        1044     1043       -1     
Impacted Files Coverage Δ
src/biokbase/narrative/jobs/job.py 90.86% <ø> (+0.35%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 014f72a...a0a0166. Read the comment docs.

@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug C 1 Bug
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug C 1 Bug
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 9, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug C 1 Bug
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@eapearson
Copy link
Contributor Author

@ialarmedalien @briehl I'm going to be iterating on this, want to put it to bed.

The main issues above I think are style -> stylesheet + classes, and css -> scss.

I have small set of additions I could add, but haven't yet. In order to evaluate changing the inline styles to a stylesheet, I created a demo capability (5 files in static/kbase/demo and one change to narrative_paths) in order to evaluate the JSONVIew component. It is problematic to evaluate components which are only shown in certain situations - errors, fleeting moments like loading indicators, etc.

@eapearson
Copy link
Contributor Author

@ialarmedalien I've moved the stylesheets and direct styles into scss, added the demo stuff.
DeepScan is being weird.

@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug C 5 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@@ -0,0 +1,3 @@
{
"extends": "stylelint-config-standard"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we've already got .stylelintrc.yaml so this shouldn't be needed

@@ -0,0 +1,54 @@
define([], () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you integrate these functions into test/unit/testUtil.js?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like they were already copied into mswUtils at some point, so I'll extract from here and mswUtils and place into testUtil.js.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to think that testUtil is a bit too big and sprawling, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, waitFor in testUtil.js is not the same thing at all, so best not to incorporate there, or rename it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a "wait" in testUtil, but it is based on Bluebird, "waitFor" is plain JS so preferable.
I think it is best to keep these async utils separate, and to remove the duplicates from mswUtils, as they are unrelated to msw.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants