Add Custom Projection to Camera3D #84454

DxUr · 2023-11-04T17:36:58Z

Add custom_projection property to Camera3D

Bugsquad edit, closes: Add custom projection mode to Camera3D, make Camera3D store its CameraMatrix and expose CameraMatrix to user code godot-proposals#2713*

scene/3d/camera_3d.cpp

editor/plugins/gizmos/camera_3d_gizmo_plugin.cpp

AThousandShips · 2023-11-05T09:44:40Z

Can you test what happens if you create a projection that produces invalid output values, like NaN and what happens if they are injected into the renderer, it could cause serious issues (like what happens when you do the same for audio, where it cuts all audio)

DxUr · 2023-11-05T11:36:52Z

Can you test what happens if you create a projection that produces invalid output values, like NaN and what happens if they are injected into the renderer, it could cause serious issues (like what happens when you do the same for audio, where it cuts all audio)

Actually this implementation affect just the projection matrix it works very well with oblique projection... , then it's for people that want to do some tweaks like 3d looks parallel in one dimensions, then it doesn't cause issues with viewports ..., I think it's so useful.
I started working on global shader overriding which will allow doing stuffs like curved world..., my problem now with the camera frustum.
Edit: you can't past Null value to set_custom_projection;

AThousandShips · 2023-11-05T11:38:01Z

What do you mean? I'm not asking for that, I'm asking what happens if you provide a broken Projection, that transforms values to NaN, the problem isn't "what limitations does it have?", it is "will adding an invalid projection mess up the rendering or crash?"

DxUr · 2023-11-05T11:41:34Z

What do you mean? I'm not asking for that, I'm asking what happens if you provide a broken Projection, that transforms values to NaN, the problem isn't "what limitations does it have?", it is "will adding an invalid projection mess up the rendering or crash?"

I tested playing with the projection as i can it never crash the rendering.

AThousandShips · 2023-11-05T11:42:46Z

Try with something like: Projection.create_orthogonal(-1, 1, -1, 1, 0, INF)

If it causes problems then that would be helped by having the methods suggested in:

Add helper methods to projection #77481

DxUr · 2023-11-05T22:00:52Z

Projection.create_orthogonal(-1, 1, -1, 1, 0, INF)

I tested it, then it does not cause any problems, even with the frustum.

PitouGames · 2023-11-10T00:09:40Z

I also need this for my project, and I also have my own implementation (roughly the same): https://github.com/PitouGames/godot/tree/custom_projection

I don't understand why the projection matrix is accessible in the editor GUI in this PR after your comment: godotengine/godot-proposals#2713 (comment). User will still be able to show it with scripts like that:

extends Camera3D


@export var projection_matrix: Projection

func _process(delta):
	custom_projection = projection_matrix

There are the camera's functions set_perspective, set_orthogonal, set_frustum but the set_custom to switch the camera to the custom mode from code is missing (or I didn't found it).

Can we also take advantage of this PR to move the field "Current" up? It's in the middle of projection settings.

Calinou · 2023-11-10T10:48:27Z

Can we also take advantage of this PR to move the field "Current" up? It's in the middle of projection settings.

While we're at it, I'd move Current at the very top and move Keep Aspect just above FOV. The way FOV is interpreted (vertical or horizontal) depends on the value of Keep Aspect.

DxUr · 2023-11-10T17:28:54Z

I don't understand why the projection matrix is accessible in the editor GUI in this PR after your comment

Actually I just make it accessible because it makes it a little bit harder to test some values theoretically then making it accessible from the editor is pointless as the projection must be recalculated if the viewport had resized but still annoying to export another projection as a script variable then make a setter..., I think it's OK now.

AThousandShips · 2023-11-30T09:36:58Z

See also:

Permit overriding the projection matrix passed to shaders without affecting culling logic. #85529

BrunkMortel · 2024-01-03T09:32:04Z

Hello,
I stumbled upon a problem playing with this PR. It appear that shadows doesn't render correctly when the camera projection is set to custom and mimic orthogonal view (even if we just copy the projection matrix from another camera) :

It seems to work just fine when mimic a perspective view :

I'm sure that the two camera have the same projection matrix since I basically did :
$SubViewport2/Camera3D.custom_projection = $SubViewport/Camera3D.get_camera_projection()

I know that the problem as something to do with the shadow rendering but can't figure out what exactly ...

DxUr · 2024-01-06T09:07:42Z

@BrunkMortel I'll check it because I don't have this issue in my projects, can provide which renderer are you using?

BrunkMortel · 2024-01-06T11:11:26Z

@DxUr I'm using the Forward+ renderer and everything else is by default.
Here is a simple scene setup that show the probleme : simple_scene.zip

Calinou · 2024-01-29T17:00:49Z

Tested locally (rebased against master fa48a51) on NVIDIA. I get a Vulkan device lost error if I adjust the Camera3D's projection matrix in the editor while the project is running:

ERROR: Vulkan: Did not create swapchain successfully. Error code: VK_NOT_READY
   at: prepare_buffers (drivers/vulkan/vulkan_context.cpp:2491)
ERROR: Vulkan: Cannot submit graphics queue. Error code: VK_ERROR_DEVICE_LOST
   at: swap_buffers (drivers/vulkan/vulkan_context.cpp:2571)

This only occurs if SDFGI is enabled. VoxelGI doesn't have these errors but renders incorrectly if any values in the custom projection are modified.

Also, running a project with a Camera3D that has its projection mode to Custom prints the following error (even if run outside the editor):

ERROR: Camera is not inside the scene tree.
   at: get_camera_projection (./scene/3d/camera_3d.cpp:212)

Directional shadow rendering also breaks if you change any values in the custom camera projection, with the following errors printed:

ERROR: Condition "!res" is true. Returning: false
   at: get_endpoints (./core/math/projection.cpp:502)
ERROR: Condition "!res" is true. Continuing.
   at: _light_instance_setup_directional_shadow (./servers/rendering/renderer_scene_cull.cpp:2153)

Point light shadow rendering breaks with certain values too (even though the rest of the scene still renders), but not as often as directional shadows. There are also no errors printed in this case.

Testing project: test_pr_84454.zip

BryceBarbara · 2024-02-23T05:04:27Z

@DxUr what's the status on this? Is there work to be done or are you waiting on more feedback?

otonashixav · 2024-03-08T13:26:07Z

@DxUr I would like to continue working on this PR, since it's nearly complete and I would benefit from it.

DxUr · 2024-03-08T14:59:48Z

I will fix all the issues that people face with this PR as soon as possible so I would like to be merged before 4.3

Co-authored-by: A Thousand Ships <[email protected]>

… own function for standardization

…thogonal

…a_3d with custom projection

…ints.

DxUr · 2024-03-23T07:18:38Z

Hope this helps then: https://github.com/otonashixav/godot/tree/pr/DxUr/84454; I've fixed the issues reported so far, though I don't know if my solutions are satisfactory since I'm not really well versed in 3D.

Thank you for fixing all the issues, I checked your solution and I think it's enough.

otonashixav

I'm not sure if any of these are blockers, as the PR works as-is.

otonashixav · 2024-03-23T10:40:03Z

scene/3d/camera_3d.cpp

+	Vector<Plane> planes = c_proj.get_projection_planes(Transform3D());
+	const Projection::Planes intersections[8][3] = {
+		{ Projection::PLANE_FAR, Projection::PLANE_LEFT, Projection::PLANE_TOP },
+		{ Projection::PLANE_FAR, Projection::PLANE_LEFT, Projection::PLANE_BOTTOM },
+		{ Projection::PLANE_FAR, Projection::PLANE_RIGHT, Projection::PLANE_TOP },
+		{ Projection::PLANE_FAR, Projection::PLANE_RIGHT, Projection::PLANE_BOTTOM },
+		{ Projection::PLANE_NEAR, Projection::PLANE_LEFT, Projection::PLANE_TOP },
+		{ Projection::PLANE_NEAR, Projection::PLANE_LEFT, Projection::PLANE_BOTTOM },
+		{ Projection::PLANE_NEAR, Projection::PLANE_RIGHT, Projection::PLANE_TOP },
+		{ Projection::PLANE_NEAR, Projection::PLANE_RIGHT, Projection::PLANE_BOTTOM },
+	};
+
+	for (int i = 0; i < 8; i++) {
+		Plane a = planes[intersections[i][0]];
+		Plane b = planes[intersections[i][1]];
+		Plane c = planes[intersections[i][2]];
+		ERR_FAIL_COND_MSG(!a.intersect_3(b, c), "Camera3D custom projection has non-intersecting planes; skipping update.");
+	}


My bad! I think this check should be in set_custom_projection instead, before c_proj is ever set. I'm also not sure if it makes sense to disallow these projections, since it seems like only lighting and the editor gizmo are dependent on having endpoints, if someone familiar could weigh in.

Edit: Okay actually I'm not too sure if changing this makes sense, since moving it before c_proj is set means you can't change the value through the invalid values in the editor.

Suggested change

Vector<Plane> planes = c_proj.get_projection_planes(Transform3D());

const Projection::Planes intersections[8][3] = {

{ Projection::PLANE_FAR, Projection::PLANE_LEFT, Projection::PLANE_TOP },

{ Projection::PLANE_FAR, Projection::PLANE_LEFT, Projection::PLANE_BOTTOM },

{ Projection::PLANE_FAR, Projection::PLANE_RIGHT, Projection::PLANE_TOP },

{ Projection::PLANE_FAR, Projection::PLANE_RIGHT, Projection::PLANE_BOTTOM },

{ Projection::PLANE_NEAR, Projection::PLANE_LEFT, Projection::PLANE_TOP },

{ Projection::PLANE_NEAR, Projection::PLANE_LEFT, Projection::PLANE_BOTTOM },

{ Projection::PLANE_NEAR, Projection::PLANE_RIGHT, Projection::PLANE_TOP },

{ Projection::PLANE_NEAR, Projection::PLANE_RIGHT, Projection::PLANE_BOTTOM },

};

for (int i = 0; i < 8; i++) {

Plane a = planes[intersections[i][0]];

Plane b = planes[intersections[i][1]];

Plane c = planes[intersections[i][2]];

ERR_FAIL_COND_MSG(!a.intersect_3(b, c), "Camera3D custom projection has non-intersecting planes; skipping update.");

}

Separately, it seems that for testing only the planes for Transform3D() is not sufficient, since it seems that it is still possible to have no endpoints if the camera position/rotation is different, which means the reported vulkan crash can still happen, albeit rarely (I spent about 30 seconds dragging every projection value back and forth before getting a crash). I don't think testing every time the camera transform changes makes sense, so there should be a more robust way of checking if a projection has endpoints regardless of transform?

FWIW you can replicate the error without a custom projection by setting a funky transform (e.g. scale = 0), which isn't guarded against, so I also have to wonder if it makes sense to guard against weird projections in the first place.

otonashixav · 2024-03-23T10:41:03Z

scene/3d/camera_3d.cpp

 void Camera3D::set_projection(ProjectionType p_mode) {
-	if (p_mode == PROJECTION_PERSPECTIVE || p_mode == PROJECTION_ORTHOGONAL || p_mode == PROJECTION_FRUSTUM) {
+	if (p_mode == PROJECTION_CUSTOM && mode != PROJECTION_CUSTOM && is_inside_tree()) {
+		c_proj = get_camera_projection();


This doesn't always work as expected in the editor¹. However, it works fine at runtime. I do wonder if it is better not to set the custom projection this way though, and instead let it keep its existing value, since it is trivial to retain the existing projection anyway, and most of the time when setting a custom projection it's something not possible with the other projection modes.

Footnotes

get_viewport seems to give the 2D viewport, which has its size out of sync with the 3D viewport sometimes (e.g. when the editor is first opened, or when the viewport area is resized, the viewport which isn't being rendered doesn't have its size updated) ↩

otonashixav · 2024-03-23T10:44:50Z

editor/plugins/gizmos/camera_3d_gizmo_plugin.cpp

(Referring to lines 79-153)

I think these need to be updated somehow, but am not really sure how.

otonashixav · 2024-03-23T10:48:29Z

scene/3d/camera_3d.cpp

@@ -641,6 +702,11 @@ void Camera3D::set_far(real_t p_far) {
 	_update_camera_mode();
 }

+void Camera3D::set_custom_projection(Projection p_proj) {


Refer to the other comment.

Suggested change

void Camera3D::set_custom_projection(Projection p_proj) {

void Camera3D::set_custom_projection(Projection p_proj) {

Vector<Plane> planes = p_proj.get_projection_planes(Transform3D());

const Projection::Planes intersections[8][3] = {

{ Projection::PLANE_FAR, Projection::PLANE_LEFT, Projection::PLANE_TOP },

{ Projection::PLANE_FAR, Projection::PLANE_LEFT, Projection::PLANE_BOTTOM },

{ Projection::PLANE_FAR, Projection::PLANE_RIGHT, Projection::PLANE_TOP },

{ Projection::PLANE_FAR, Projection::PLANE_RIGHT, Projection::PLANE_BOTTOM },

{ Projection::PLANE_NEAR, Projection::PLANE_LEFT, Projection::PLANE_TOP },

{ Projection::PLANE_NEAR, Projection::PLANE_LEFT, Projection::PLANE_BOTTOM },

{ Projection::PLANE_NEAR, Projection::PLANE_RIGHT, Projection::PLANE_TOP },

{ Projection::PLANE_NEAR, Projection::PLANE_RIGHT, Projection::PLANE_BOTTOM },

};

for (int i = 0; i < 8; i++) {

Plane a = planes[intersections[i][0]];

Plane b = planes[intersections[i][1]];

Plane c = planes[intersections[i][2]];

ERR_FAIL_COND_MSG(!a.intersect_3(b, c), "Camera3D custom projection has non-intersecting planes; skipping update.");

}

DxUr · 2024-03-23T11:19:12Z

@otonashixav But it works fine for me, I had tried every possible case!

otonashixav · 2024-03-23T11:40:41Z

I mean it does work fine. I'm just concerned about some points that the reviewers might be able to clear up.

DxUr · 2024-03-23T22:41:11Z

If I tilt the camera perspective upwards, the object "moves" downwards, but the background sky also moves downwards (into the brown area) instead of showing the sky above the object (see video). The same thing happens when you tilt the camera perspective down, only in the opposite direction. Is this intentional or a bug?

This is not a bug, this is expected due to what a projection matrix is. The Projection matrix is 4x4 Matrix, you should not change it manually you must use a script to do that, Than you can fix the tilting by playing with all components.
Note that using a proper calculations is required.

Calinou · 2024-04-04T00:42:39Z

The Projection matrix is 4x4 Matrix, you should not change it manually you must use a script to do that, Than you can fix the tilting by playing with all components.
Note that using a proper calculations is required.

In this case, it should be marked read-only in the inspector to prevent accidental changes. This can be done using PROPERTY_USAGE_READ_ONLY in _bind_methods().

BryceBarbara · 2024-04-12T19:31:26Z

Is the PROPERTY_USAGE_READ_ONLY flag all that is needed for this PR to be merged or are we waiting on responses to the comments from @otonashixav?

mattdiener · 2024-04-23T18:31:54Z

Thanks for writing this PR. I don't mind building from source but would love to see this merged so I can be confident relying on it in my project. Are there any remaining blockers? If more needs to be done, I'd be happy to help out.

AThousandShips · 2024-04-23T18:33:35Z

This hasn't been reviewed entirely yet, so it needs to be reviewed properly first, so nothing you can help with necessarily, we're getting close to feature freeze as well meaning this is likely not going to be in 4.3

spoxii · 2024-10-10T22:06:36Z

I have been using this feature for a few weeks now and would appreciate a merge for the 4.4 release.
From a user-perspective this feature seems to be stable.
My use case is a top-down projection with some additional care to make it pixel-perfect.

EnlightenedOne · 2024-10-17T21:55:41Z

FYI I attended the Rendering Meeting, the main takeaway was that without screen space effects and all forward lighting pathways working with custom projection this would not get merged:
"Not clear how to maintain support for depth-related effects (SSAO, SSIL, SSR, SSS, lighting, SDFGI, VoxelGI, etc.)"

If you are using a custom projection with capacity to fully change the matrix I am not sure if you can deliver all those features reliably or not. Verifying what you can support lighting and effects wise and under what conditions seems to be the key to advancing this PR.

DxUr requested review from a team as code owners November 4, 2023 17:36

AThousandShips added enhancement topic:rendering topic:3d labels Nov 4, 2023

AThousandShips added this to the 4.x milestone Nov 4, 2023

AThousandShips reviewed Nov 4, 2023

View reviewed changes

scene/3d/camera_3d.cpp Outdated Show resolved Hide resolved

scene/3d/camera_3d.cpp Show resolved Hide resolved

scene/3d/camera_3d.cpp Outdated Show resolved Hide resolved

AThousandShips reviewed Nov 4, 2023

View reviewed changes

editor/plugins/gizmos/camera_3d_gizmo_plugin.cpp Outdated Show resolved Hide resolved

DxUr mentioned this pull request Nov 4, 2023

Add custom projection mode to Camera3D, make Camera3D store its CameraMatrix and expose CameraMatrix to user code godotengine/godot-proposals#2713

Open

DxUr mentioned this pull request Nov 5, 2023

Add a way to override global shaders godotengine/godot-proposals#8355

Open

AThousandShips mentioned this pull request Nov 30, 2023

Permit overriding the projection matrix passed to shaders without affecting culling logic. #85529

Open

qaptoR mentioned this pull request Mar 4, 2024

Add new Camera3D type: Oblique #89140

Open

DxUr and others added 13 commits March 23, 2024 08:09

Update editor/plugins/gizmos/camera_3d_gizmo_plugin.cpp

f4b43bb

Co-authored-by: A Thousand Ships <[email protected]>

Update camera_3d.cpp

75febc0

Update camera_3d.cpp

6e37790

Update camera_3d.cpp

449314f

Update camera_3d.cpp

87a20e3

Update Camera3D.xml

d3904d3

Update RenderingServer.xml

07108a9

Extract switch-case case PROJECTION_CUSTOM _update_camera_mode to its…

3cb5d72

… own function for standardization

Fix Camera3D custom projection which are orthogonal not setting is_or…

ee7fafe

…thogonal

Add missing function declaration in camera_3d.h

89e53b1

Fix camera is not inside the scene tree error when initializing camer…

6e06fd0

…a_3d with custom projection

Fix SDFGI/Directional Lights spamming errors when unable to get endpo…

18bb7b1

…ints.

Add Camera3d custom projection gizmo.

e360ab6

DxUr force-pushed the custom_camera3d_projection branch from a69bea8 to c637a49 Compare March 23, 2024 07:06

DxUr force-pushed the custom_camera3d_projection branch from c637a49 to e360ab6 Compare March 23, 2024 07:37

otonashixav reviewed Mar 23, 2024

View reviewed changes

A better gizmo

eb4bacf

DxUr force-pushed the custom_camera3d_projection branch from 9ec1408 to eb4bacf Compare March 23, 2024 12:08

This comment was marked as resolved.

Sign in to view

Norrox mentioned this pull request Oct 3, 2024

Add custom projection to camera3d Redot-Engine/redot-engine#134

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Custom Projection to Camera3D #84454

Add Custom Projection to Camera3D #84454

DxUr commented Nov 4, 2023 •

edited by AThousandShips

Loading

AThousandShips commented Nov 5, 2023

DxUr commented Nov 5, 2023 •

edited

Loading

AThousandShips commented Nov 5, 2023

DxUr commented Nov 5, 2023

AThousandShips commented Nov 5, 2023 •

edited

Loading

DxUr commented Nov 5, 2023

PitouGames commented Nov 10, 2023

Calinou commented Nov 10, 2023 •

edited

Loading

DxUr commented Nov 10, 2023

AThousandShips commented Nov 30, 2023

BrunkMortel commented Jan 3, 2024

DxUr commented Jan 6, 2024

BrunkMortel commented Jan 6, 2024

Calinou commented Jan 29, 2024 •

edited

Loading

BryceBarbara commented Feb 23, 2024

otonashixav commented Mar 8, 2024

DxUr commented Mar 8, 2024 •

edited

Loading

DxUr commented Mar 23, 2024

otonashixav left a comment

otonashixav Mar 23, 2024 •

edited

Loading

otonashixav Mar 23, 2024 •

edited

Loading

otonashixav Mar 23, 2024

otonashixav Mar 23, 2024 •

edited

Loading

otonashixav Mar 23, 2024 •

edited

Loading

DxUr commented Mar 23, 2024

otonashixav commented Mar 23, 2024

This comment was marked as resolved.

DxUr commented Mar 23, 2024

Calinou commented Apr 4, 2024

BryceBarbara commented Apr 12, 2024

mattdiener commented Apr 23, 2024

AThousandShips commented Apr 23, 2024

spoxii commented Oct 10, 2024

EnlightenedOne commented Oct 17, 2024

Add Custom Projection to Camera3D #84454

Are you sure you want to change the base?

Add Custom Projection to Camera3D #84454

Conversation

DxUr commented Nov 4, 2023 • edited by AThousandShips Loading

AThousandShips commented Nov 5, 2023

DxUr commented Nov 5, 2023 • edited Loading

AThousandShips commented Nov 5, 2023

DxUr commented Nov 5, 2023

AThousandShips commented Nov 5, 2023 • edited Loading

DxUr commented Nov 5, 2023

PitouGames commented Nov 10, 2023

Calinou commented Nov 10, 2023 • edited Loading

DxUr commented Nov 10, 2023

AThousandShips commented Nov 30, 2023

BrunkMortel commented Jan 3, 2024

DxUr commented Jan 6, 2024

BrunkMortel commented Jan 6, 2024

Calinou commented Jan 29, 2024 • edited Loading

BryceBarbara commented Feb 23, 2024

otonashixav commented Mar 8, 2024

DxUr commented Mar 8, 2024 • edited Loading

DxUr commented Mar 23, 2024

otonashixav left a comment

Choose a reason for hiding this comment

otonashixav Mar 23, 2024 • edited Loading

Choose a reason for hiding this comment

otonashixav Mar 23, 2024 • edited Loading

Choose a reason for hiding this comment

otonashixav Mar 23, 2024

Choose a reason for hiding this comment

Footnotes

otonashixav Mar 23, 2024 • edited Loading

Choose a reason for hiding this comment

otonashixav Mar 23, 2024 • edited Loading

Choose a reason for hiding this comment

DxUr commented Mar 23, 2024

otonashixav commented Mar 23, 2024

This comment was marked as resolved.

DxUr commented Mar 23, 2024

Calinou commented Apr 4, 2024

BryceBarbara commented Apr 12, 2024

mattdiener commented Apr 23, 2024

AThousandShips commented Apr 23, 2024

spoxii commented Oct 10, 2024

EnlightenedOne commented Oct 17, 2024

DxUr commented Nov 4, 2023 •

edited by AThousandShips

Loading

DxUr commented Nov 5, 2023 •

edited

Loading

AThousandShips commented Nov 5, 2023 •

edited

Loading

Calinou commented Nov 10, 2023 •

edited

Loading

Calinou commented Jan 29, 2024 •

edited

Loading

DxUr commented Mar 8, 2024 •

edited

Loading

otonashixav Mar 23, 2024 •

edited

Loading

otonashixav Mar 23, 2024 •

edited

Loading

otonashixav Mar 23, 2024 •

edited

Loading

otonashixav Mar 23, 2024 •

edited

Loading