Double Precision builds import with worst LODs

[lyuma]

Godot version

4.0.beta 0b627c4 + doubles

System information

Windows 10, NVidia RTX 2080

Issue description

On double precision builds, importing particular skinned models will cause LODs to look broken. cc @Zylann @fire

Example of the issue with and without Disable LODs selected in the advanced menu:

Here is some data from the generated tscn that could help debugging. Some of these were done with the V-Sekai/groups-4.x branch, but I did a test to ensure there were no differences. The model has three submeshes.

beta3_lods.tscn: [0.112869, 0.229829], [0.141503, 0.214244], [0.182762] (beta3 official without doubles)
float_lods.tscn: [0.112869, 0.229829], [0.141503, 0.214244], [0.182762] (control-- V-Sekai fork without doubles)
revert_65493_goodish.tscn: [3.45235e+19, 3.68926e+19], [1.76996e+19], [1e-10] (new build of V-Sekai fork 65493 reverted)
really_bad_lods.tscn: [6.01057e+19], [1e-10], [1e-10] (new build of godot master, revision 0b627c4)
mostly_bad_lods.tscn: [6.01057e+19, [1e-10], [1e-10] (build of V-Sekai fork from Oct. 12)

Reverting #65493 appears to make the issue go away, but I believe this to be a red herring. If you read the generated .tscn, it is clear that the underlying bug still exists even if 65493 is reverted. Somehow reverting it makes it more likely for LODs to be large positive numbers and never hit. Maybe it's all just random.

Final note: The test model is Godette by SirRichard94, converted to .vrm and renamed to .glb (Valid since VRM is a glTF extension). For some reason, this process seems to reliably reproduce the issue, while some ordinary glTF documents (such as those in godot-tests) did not appear to repro the issue.

Steps to reproduce

1. Delete the .godot folder if one is present.
2. Open the attached project with a float=64 editor build.
3. When main.tscn opens, zoom into the model and look for artifacts. The left arm usually gets broken.
4. Close the project and clear .godot.
5. Try again from a stock build or one without float=64
6. The arms should look like cylinders without artifacts.

Minimal reproduction project

Simple project with just .glb: BadLODTest.zip

Since it might help debugging, here is a project with Make Local .tscn of that .glb from each engine version.
BadLODTest_tscn.zip

[fire]

Notes:

The c cast to float* to the mesh optimizer simplify function is wrong.

[Zylann]

I reproduced the issue with a 64-bit float build:

I fixed the float* casts in simplification usages in #67853:

Open the attached project with a float=64 editor build.
When main.tscn opens, zoom into the model and look for artifacts. The left arm usually gets broken.

Left arm looks fine:

I tried playing with lod bias but couldn't get it to smoothly go over various degrees of decimation, it's all or nothing. But that's probably a separate issue.

Also due to how the menus work, it is impossible to disable LODs AND see the wireframe. So while I can't see bad polygons, I can't tell if LOD is doing anything either. With tiny number it does stuff though, just not easy to be gradual.
Tried the same tests without my changes in a 32-bit float build, same results.

You can check with the PR below