space between tiles when using skip-sizes

[danieltbrennan]

When the skip-sizes prop is set true we've been able to observe a small amount of empty space between image tiles. The severity of the issue seems to vary based on screen size with the issue being more prevalent on larger displays of higher resolution and less severe apparent on smaller windows. Noting also that as the zoom level increases the issue seems to be lessen to a point of being almost invisible to the eye, but is most apparent on initial load.

In this example below to highlight the issue I've applied style={{ '--atlas-background': 'red'}} to the "Canvas with Skip Sizes" story:

tagging @itspamlam as she was first to observe this in and may have other useful observations.

[stephenwf]

This is a browser quirk with images and the translate() CSS property when using values less than 1.

I've created a Code Sandbox minimal example.

There are 3 options:

• Force the renderer to only translate in whole pixels (less smooth)
• Increase the width of each image by 1px (less accurate)
• Render a low quality version under

Atlas currently relies on this third option to smooth out these rounding issues. However, this will not work when applying an opacity. We can only render ONE image and then set its opacity. The browser doesn't currently have a way to group together layers of things and then set a opacity on the group as a whole.

https://github.com/digirati-co-uk/iiif-canvas-panel/blob/main/packages/canvas-panel/src/atlas-components/TileSet/TileSet.tsx#L49-L64

This is where we can set different values. The currently supported values:

• renderSmallestFallback - this is where Atlas will always render the very smallest available image to cover up gaps. If you zoom too fast in you will notice atlas shows blurry version of the image where OSD would show black tiles.
• renderLayers - this is the number of tile layers Atlas should try to paint. Its value is usually set to 2. For reference, OSD will render all previous layers. If the smallest fallback is included in the list then it will be 2, and if not then it will be 3.
• minSize - the smallest acceptable size to use. Filters out scaleFactors that are just useless showing very tiny images, cutting down on HTTP requests. Default: 255
• maxImageSize - Can be useful to avoid making very large requests (e.g. a max/full) and the default for this is 1024 in either dimension.
• quality - high number, the more likely a higher quality image will be used on a layer. (needs work, default value: 1.2).

What we could do is add an extra config value to increase the size of images by 1px if there is only one layer being drawn.

In this file, anywhere where there is a Number.MIN_VALUE is where the adjustment needs to be. Changing this to + Number.MIN_VALUE + 1 was enough to fix the rendering. The MIN_VALUE is all we need when rendering on an HTML Canvas to account for the rounding errors. However, because some browsers don't accept long floating point translations() in CSS, we have to clamp the value to 2 fixed places, resulting in a larger adjustment needed (between 0-1px, the exact value would be difficult to calculate).
https://github.com/atlas-viewer/atlas/blob/master/src/modules/static-renderer/static-renderer.ts#L205-L206

[abrin]

@stephenwf I just took a deeper look at this, and it looks like it's happening on the canvas-renderer not the static renderer. Any ideas where to look for a good adjustment there?

I added a little debug to the paint method and it broke the image into 4 tiles. you can see the subtle black line between the two left and right sets of tiles:

Img	Height	Width	x	y
1	322.0442199707031	322.04425048828125	612.0675659179688	0
2	322.0442199707031	30.82061767578125	934.11181640625	0
3	189.95578002929688	322.04425048828125	612.0675659179688	322.0442199707031
4	189.95578002929688	30.82061767578125	934.11181640625	322.0442199707031

The maths look right, but wondering if there's a similar issue with rounding when drawing on the canvas somewhere else?

[abrin]

@stephenwf I did a bit more testing and put up a draft PR....

1. I agree with Dan that this feels like a math issue somewhere (he suggested IIIF - possible wrong calculation of tile width openseadragon/openseadragon#2198).
2. We are able to see the lines when the user has zoomed in as well (not just the initial zoom)
3. adjusting the size of the box the canvas is written to by a small amount (.5) seems to fix the problem specifically (https://github.com/atlas-viewer/atlas/pull/43/files#diff-b29909e8cebfd9dafa4d7d8c6b948266501d53ca56f7dd0153f5a4abf71ce00bL492)
4. But... this feels like something is getting dropped further up the line, but when I look at the tiles requested from the image service, the math looks right. Wondering if you have any ideas?
5. is there an easy way you'd recommend to loop this into a conditional flag that we could set in CanvasPanel?

Links:

• Change in Atlas to the CanvasRenderer
• Simple PR that exposes the changes to CanvasPanel

[abrin]

Throwing this up as an idea for how we might turn this on more selectively -- atlas-viewer/atlas@eeff34b

[stephenwf]

Thanks @abrin, looks like a deep dive.

1. I agree with Dan that this feels like a math issue somewhere (he suggested openseadragon/openseadragon#2198)

It's possible that the math could be wrong - the main goal of Atlas is compatibility with 2.1 and 3.0 images with level0, since there is no room for error there. The rounding and calculations tries to align with level0, with the hope that any caching for level1 and level2 use similar calculations.
https://github.com/atlas-viewer/atlas/blob/master/src/spacial-content/tiled-image.ts#L135-L155
Atlas should, in theory, be easy to test for issues with the Math. Each tiled image layer (quality level) is distinct, and every tile is calculated up front. There are some tests for the current version: https://github.com/atlas-viewer/atlas/blob/master/src/__tests__/spacial-content/tiled-image-test.ts but they only cover scaling up and down.

(The V2 WIP code also has a tile grid implementation, suitable for maps and infinite spaces, with some tests. If the current implementation requires too much work, it might be better using this as a starting point. I hope thats not the case though).

4. But... this feels like something is getting dropped further up the line, but when I look at the tiles requested from the image service, the math looks right. Wondering if you have any ideas?
Honestly, I think adding the extra 0.5 is a good option. OpenSeadragon will allow small gaps when the image is moving, preserving the smoothness, but will round to the nearest pixel when the image isn't moving. This seems to ensure there are no gaps, but the result is that overlays (like HTML Annotations) can jump around a bit and not follow the image perfectly.

5. is there an easy way you'd recommend to loop this into a conditional flag that we could set in CanvasPanel?

I think we could try just changing the default to 0.5 for the canvas renderer (HTML translate requires something closer to full pixel) and look at making that configurable later. OpenSeadragon has another trick we could look into for rendering items with opacity values, first clearing the pixels (rounded). That would allow us to continue to render the lowest quality version behind and avoid the sub-pixel issues similar to OSD. However, that would require more work.

[abrin]

@stephenwf I will take a closer look later today. I think from our end, we're feeling a bit of a time-crunch, go live is Monday for everything. Would it be possible to merge a version with the temporary fix of .5 (or for something that's a bit more configurable) and then to come back to this. I'm happy to dedicate a little time looking at the map option and tests in option #1 if you like, but, it would be nice to know we're clear to go with this issue resolved.

thanks

[stephenwf]

Yes no problem, I'll make/merge the 0.5 fix and test to make sure its working later today (~3 hours). I'll mention you in a PR to this repository with the fix.

[stephenwf]

@abrin I've pushed a branch here: atlas-viewer/atlas#44

It's from Atlas v2.0.8 (so we can make a 2.0.9 as 2.1.0 is a more involved change in #237)

I did some testing, and I found that FireFox produced gaps. However, the performance dropped a bit when adding the +1. On Chrome-based browsers I didn't notice the gaps. I tired Math.round() which did work for firefox, but made it jump around. I settled for a quick UA check for FireFox and only applied the +1 there.

There is a hosted storybook example here:
https://deploy-preview-44--atlas-viewer-storybook.netlify.app/?path=/story/tests--test-static-render

This is configured to only render one layer. Can you confirm if this fixes the issue on your end.

[abrin]

@stephenwf I'm seeing lines in the example story -- adding some screenshots. also wondering if the addition was transposed?

[stephenwf]

Just so I can try to replicate, what is the OS/Browser/Resolution?

I think you're right, the missing .5. I've pushed that, the storybook should be available shortly.

[abrin]

I'm on MacOS 14.2 / Safari testing mainly on the LG, but it's actually worse on the native MacBook Pro Retina display (screenshot). tested in Firefox and Chrome, and I think they're ok

LG ULTRAWIDE:
Resolution: 3440 x 1440 (UWQHD - Ultra-Wide Quad HD)
UI Looks like: 3440 x 1440 @ 60.00Hz
Display Serial Number:
Main Display: Yes
Mirror: Off
Online: Yes
Rotation: Supported
Color LCD:
Display Type: Built-in Liquid Retina XDR Display
Resolution: 3456 x 2234 Retina
Mirror: Off
Online: Yes
Automatically Adjust Brightness: Yes
Connection Type: Internal

[stephenwf]

Awesome. I think the 0.5 fixed it. Was able to replicate on Safari with a HiDPI display.

Before

After

[danieltbrennan]

I just tested the example story on Firefox/Chrome/Safari on both the retina screen and an LG set up similar to Adam's and I'm not seeing the issue in any instance.

[stephenwf]

Excellent! Then the PR is ready, I can merge and make a new v62 of canvas panel web components 👍

[stephenwf]

Published in @digirati/[email protected]