Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG Alpha blending on hard edge textures #40

Open
ryotoko opened this issue Sep 25, 2020 · 2 comments
Open

BUG Alpha blending on hard edge textures #40

ryotoko opened this issue Sep 25, 2020 · 2 comments

Comments

@ryotoko
Copy link

ryotoko commented Sep 25, 2020

Tracking this separately, as it's different than the fog problem.

93632751-b7afee80-f9a2-11ea-93a1-15639486d05f
As you can see in this image, the outline of the tree is clipping tris with other alpha functions: the tree's shadow, and other trees. Strangely, the tree on the right is not being clipped, but the tree on the left is.

Edit:
The problem is that the 3DS does not have a fragment shader or a programmable pixel shader, which is where true depth and alpha tests usually operate. The 3DS implements these at the last stage of the vertex shading pipeline and simulates them, and a depth and alpha test can only be applied once per frame. (This is one way the code might be cleaned up slightly, since Gericom added a lot of depth and alpha test calls apparently assuming the 3DS had fragment shading capabilities when it doesn't.)

As it stands, the texture filter is filtering the textures, which interpolates not only the color values but alpha values as well, and the result is a semi-transparency on the edge. You can cull all alpha values below 255 but you will lose the smoothed edge effect (and part of the texture). Any alpha values not culled will be the testing point of the depth test, which is next in the pipe, and considers the remaining alpha values as solid and culls what's behind them.

The reason that it only happens sometimes has to do with the draw order. The depth test only culls textures that fail the depth test. If the tree was properly drawn in front of a tree that it should be in front of, it is actually not culled and then proper blending occurs, which can be seen by disabling the depth mask.

As changing the depth test and alpha test values will adversely affect something else, the solution here needs to be fairly creative. I've come up with the following:

  • Disable the depth mask and create a render queue that first sorts the Z value of the closest vertex, for alpha textures, before sending them to the pipe, drawing them in their proper order and eliminating the need for the depth mask
  • Create a custom linear filter function for alpha textures that sets a hard gradient for alpha levels, setting all above a certain line to 255 and all below to 0, eliminating the semi-transparency. Depending on where filtering happens, this could be difficult or computationally expensive
  • Create "HD" textures for the problem textures (pretty much just trees) that are pre-filtered and have no semi-transparency, and flag them so the renderer knows not to filter them again
@mkst
Copy link
Owner

mkst commented Sep 25, 2020

Does this happen on Gericom's version or is this a regression?

@ryotoko
Copy link
Author

ryotoko commented Sep 25, 2020

Gericom's is also affected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants