feat: Deep zoom #929

tihuan · 2024-05-10T20:03:29Z

This is the feature branch for the Deep Zoom feature. Deep Zoom related PRs will be merged against this branch

5/10:

Screen.Recording.2024-05-10.at.2.42.26.PM-2.mov

tihuan · 2024-05-10T20:05:16Z

client/package.json

@@ -145,6 +146,7 @@
    "@types/lodash.uniq": "^4.5.6",
    "@types/lodash.zip": "^4.2.6",
    "@types/node": "^20.10.6",
+    "@types/openseadragon": "^3.0.10",


WARNING: There's only v3 types, but we're using v4 library 😭

There's a ticket for this: https://app.zenhub.com/workspaces/single-cell-5e2a191dad828d52cc78b028/issues/gh/chanzuckerberg/single-cell-explorer/899

tihuan · 2024-05-10T20:08:42Z

client/src/actions/index.ts

@@ -34,7 +34,7 @@ import { packDiffExPdu, DiffExMode, DiffExArguments } from "../util/diffexpdu";
 import { track } from "../analytics";
 import { EVENTS } from "../analytics/events";
 import AnnoMatrix from "../annoMatrix/annoMatrix";
-import { checkFeatureFlags } from "../util/featureFlags/featureFlags";


Move checkFeatureFlags to /src/index.tsx, so we do the check ASAP

tihuan · 2024-05-10T20:09:20Z

client/src/actions/index.ts

-      // save isCellGuideCxg to the reducer store
-      dispatch({ type: "initial data load complete", isCellGuideCxg });
+      // save `isCellGuideCxg` and `s3URI` to the reducer store
+      dispatch({ type: "initial data load complete", isCellGuideCxg, s3URI });


Adding s3URI to the store, since now we need to parse s3URI to find the right deep zoom assets in S3

tihuan · 2024-05-10T20:09:35Z

client/src/common/selectors.ts

+import { spatialEmbeddingKeyword } from "../globals";
+import { RootState } from "../reducers";
+
+export function isSpatialMode(props: Partial<RootState>) {


create a new selectors file to add selectors

we really should type our state at some point. RootState is just a masked any

tihuan · 2024-05-10T20:11:36Z

client/src/components/graph/drawSpatialImageRegl.ts

I believe this is safe to remove now. But there's probably more MVP related code that needs to be removed in a subsequent PR

Ticket here: https://app.zenhub.com/workspaces/single-cell-5e2a191dad828d52cc78b028/issues/gh/chanzuckerberg/single-cell-explorer/894

tihuan · 2024-05-10T21:36:17Z

client/src/util/camera.ts

      this.canvas,
      clientX,
      clientY,
      projectionInvTF
    );
-    this.zoomAt(1 / Math.exp(scale / height), pos[0], pos[1]);
+
+    this.zoomAt({


passing more args to zoom, since OSD needs the info

tihuan · 2024-05-10T21:36:30Z

client/src/util/camera.ts

+        viewChanged = this.wheelZoom(
+          e as unknown as WheelEvent,
+          projectionTF,
+          openseadragon


passing openseadragon instance here

tihuan · 2024-05-10T21:39:00Z

client/src/util/featureFlags/featureFlags.ts

+  const params = getParams();
+
+  const paramValue = params?.get(key);
+
+  if (paramValue !== undefined) {
+    setFeatureFlag(key, paramValue || "");
+    return paramValue === ParamBoolean.TRUE;
+  }
+


This resolves the following bug:

Clear local storage

Load http://localhost:3000/d/super-cool-spatial.cxg?spatial=true&dl=true

getFeatureFlag("spatial") actually returns false instead of true, because the getter is called before the setter

The solution is to prioritize param as the source of truth for feature flag state, and also set the local storage proactively

tihuan · 2024-05-10T21:40:24Z

server/common/utils/uns.py

+    # Preparing data for pyvips from the cropped numpy array
+
+    # NOTE: Flip the image vertically, since somehow the client flips the image
+    image_pil = image_pil.transpose(Image.FLIP_TOP_BOTTOM)
+
+    # Convert the flipped PIL Image back to a numpy array
+    flipped_array_uint8 = np.array(image_pil)
+
+    h, w, bands = flipped_array_uint8.shape
+    linear = flipped_array_uint8.reshape(w * h * bands)
+    vipsImage = pyvips.Image.new_from_memory(linear.data, w, h, bands, "uchar")
+
+    # Save as Deep Zoom Image (this will create a directory with tiles and a .dzi file)
+    vipsImage.dzsave("./server/common/web/static/deep_zoom/spatial", suffix=".jpeg")
+
+    h = w = min(h, w)  # Adjust for 1:1 aspect ratio


code to generate deep zoom assets locally. We have another ticket to clean this up here:

https://app.zenhub.com/workspaces/single-cell-5e2a191dad828d52cc78b028/issues/gh/chanzuckerberg/single-cell-explorer/920

server/common/utils/uns.py

seve · 2024-05-20T22:42:59Z

client/src/common/selectors.ts

+import { spatialEmbeddingKeyword } from "../globals";
+import { RootState } from "../reducers";
+
+export function isSpatialMode(props: Partial<RootState>) {


we really should type our state at some point. RootState is just a masked any

seve · 2024-05-20T22:44:32Z

client/src/components/graph/util.ts

+  const fractionToUse = isSpatialMode ? 1 : 0.95; // fraction of min dimension to use
+  const topGutterSizePx = isSpatialMode ? 0 : 32; // top gutter for tools
+  const bottomGutterSizePx = isSpatialMode ? 0 : 32; // bottom gutter for tools


Note to conduct extra QA of centroid labels

seve · 2024-05-20T22:46:59Z

client/src/components/graph/util.ts

+  return `${url}spatial.dzi`;
+}
+
+function getSpatialUrl(s3URI: string) {


I wonder if we should just provide this URL on /config like we do with similar pieces of information

tihuan · 2024-05-29T17:21:08Z

discussed with the team that we have enough e2e test coverage to merge this feature branch into main now to get more days for stakeholders to test the feature. And I will write more e2e tests this week to cover new features added! #yolo

tihuan force-pushed the feat-deep-zoom branch from 1f02379 to a14ad13 Compare May 10, 2024 20:04

seve mentioned this pull request May 10, 2024

feat(side by side embedding): deep zoom in panel #927

Closed

tihuan commented May 10, 2024

View reviewed changes

tihuan requested review from codemonkey800, seve and kaloster May 13, 2024 14:21

tihuan changed the title ~~feat: Deep zoom~~ [DO NOT MERGE] feat: Deep zoom May 14, 2024

tihuan mentioned this pull request May 14, 2024

[DO NOT MERGE] spike: deep zoom #879

Closed

codemonkey800 approved these changes May 14, 2024

View reviewed changes

This was referenced May 28, 2024

Remove MVP related code #894

Closed

Only generate deep zoom files in BE when running locally #920

Closed

feat: Deep Zoom

bcaa455

tihuan force-pushed the feat-deep-zoom branch from 4bebcbd to bcaa455 Compare May 28, 2024 17:36

fix: updated super-cool-spatial (#954)

426c678

seve reviewed May 28, 2024

View reviewed changes

Merge branch 'main' into feat-deep-zoom

ace88c0

tihuan changed the title ~~[DO NOT MERGE] feat: Deep zoom~~ feat: Deep zoom May 29, 2024

tihuan merged commit 164aa76 into main May 29, 2024
4 of 5 checks passed

tihuan deleted the feat-deep-zoom branch May 29, 2024 17:21

czi-github-helper bot mentioned this pull request Jun 20, 2024

chore(main): release 1.0.0 #995

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Deep zoom #929

feat: Deep zoom #929

tihuan commented May 10, 2024 •

edited

Loading

tihuan May 10, 2024

tihuan May 10, 2024

tihuan May 10, 2024

tihuan May 10, 2024

seve May 20, 2024

tihuan May 10, 2024

tihuan May 10, 2024

tihuan May 10, 2024

tihuan May 10, 2024

tihuan May 10, 2024

seve May 20, 2024

seve May 20, 2024

seve May 20, 2024

tihuan commented May 29, 2024

feat: Deep zoom #929

feat: Deep zoom #929

Conversation

tihuan commented May 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tihuan commented May 29, 2024

tihuan commented May 10, 2024 •

edited

Loading