Shaders not WebGL compatible

[robibkuiper]

Hello,

I just came across this project today and i'm amazed by its performance so far. It blows away everything i've done so far trying to load a point cloud in unity.

Right now i'm working on a case that requires point clouds to be loaded in browser so i changed this package to load the files from StreamingAssets and tried building it for WebGL. Sadly it doesn't want to load the shaders. These are the errors i get in the console:

WARNING: Shader
Unsupported: 'Custom/QuadGeoScreenSizeShader' - Pass '' has no vertex shader
ERROR: Shader
Shader is not supported on this GPU (none of subshaders/fallbacks are suitable)
WARNING: Shader
Unsupported: 'Custom/QuadGeoScreenSizeShader' - Setting to default shader.

I'm not really a shader magician so i'm struggling to find a fix for this.

I build it in Unity 2019.1.8f1 with both webgl 2.0 and 1.0 as graphics api without any difference in result. i've also tried to use some of the other shaders in the project with the same result. Do you have any suggestions what could cause the shaders not to load?

Thanks in advance for your answer!

[akharroubi]

Hi !

I do not know exactly the source of error, but you can use Potree(WebGL point cloud viewer for large datasets, http://potree.org), it allows a good visualization in browser.
https://github.com/potree/potree

[robibkuiper]

Thanks for the quick response!

I see. I would like to build in some custom tools to be able to load custom data in the pointcloud, like bullet points etc. not to mention i want to be able to develop a VR and WebGL application in one go. Maybe i'm just asking to much of Unity :)

Since i prefer to work in unity i'm going to try to fix this issue first before checking alternatives, also because i'm quite amazed by the performance of this package (also potree).

I'm still open for suggestions ofcourse!

Into the shader magic I go! I'll post potential fixes in this thread.

[robibkuiper]

So i've finally managed to find some time to look into this further... So a quick update. I've been working on removing all multithreaded and System.File code and replacing it with coroutines and webrequest. This gives good results so far.

Things that work so far in webgl/browser:
-loading a pointcloud (1.5 billion pointcloud is loading, didn't test performance yet)
-visualizing a pointcloud with a line mesh renderer

Things i'm currently trying to fix:
-Unity apparently has trouble rendering point meshes in webgl(i believe this is a Unity bug)
-It seems to load point according to the point budget, but it doesn't render all the loaded points. i think this happens because with multithreading its waiting to do this loop(correct me if i'm wrong) LoadThread->TraverselThread->RenderThread. With current coroutines these 3 are all running seperate without correlation.
-The ParaboloidFragScreenSizeShader and QuadGeoScreenSizeShader are not going to work in webgl. but this is not the biggest problem for now.

I made a package of the changes I made if someone wants to check it out. Its quite a mess since i basicly copied the necessary scripts and butchered them to work with coroutines.

https://drive.google.com/open?id=1r6VT_SngwcQFA7VjVMPAlF6sfEwOd2ab

Cheers!

[robibkuiper]

Another update:
The problem with not rendering all the points is fixed now. The LoadHierarchy in CloudLoader is recursive that wasn't transalated to a coroutine yet. The package is updated with the fix.

[noghabaei]

There are two problems for using this project in unity WebGL:

1. unity does not support multi-threading in WebGL.
2. unity cannot render points in WebGL.

If anyone is interested in collaboration to fix this problem, please send me an email.

[robibkuiper]

Yes you are correct about both issues.

In this package
https://drive.google.com/open?id=1r6VT_SngwcQFA7VjVMPAlF6sfEwOd2ab
there is a version of the renderer that is rewritten to work with coroutines it is tested and works in browser . Only a bit slower than when its done multi threaded ofcourse. Its also a bit messy but i did not found the time yet to continue working on this. The package does work in WebGL on mac, just not on windows because of unity's point render problem. If you want to test it put your pointcloud in the StreamingAssets folder and change the path on the CRLoader object in de main_webgl scene. Than you can simply build for webgl and test it in browser, like said before it works on mac, but for windows it loads all the necessary files but just doesn't render the points.

About point 2 I made a thread on the Unity WebGL forum. One of the Unity employees recommended trying particle systems for rendering the points in WebGL. I've also not found time yet to look into this yet. the thread is found here https://forum.unity.com/threads/meshtopology-point-not-rendering-workaround.758369/#post-5051375

I would like to work on this together but i'm not always available.

[SFraissTU]

Thank you robibkuiper and mojtaba1995 for your interest. Maybe you can figure out together how to solve these problems. I only have limited availability, but if I can help you in any way, let me know. I haven't checked out your package, might do so when I have some time. My knowledge about WebGL and the threading issues there is also a bit rusty.
Regarding point rendering being impossible in WebGL: You can try the approach that's followed in Quad4PointMeshConfiguration. This was an experimental feature I created when I wrote my bachelor thesis, which is not really useful anymore, as it is slower than DefaultMeshConfiguration. Basically, instead of rendering points and using a geometry shader to create screen faced splats, it already quadruples each point on the CPU and renders quads. Maybe this would work in WebGL as well? Feel free to try, if you haven't done so already.

[noghabaei]

Yes, the quads will work, but that's not efficient as you said. robibkuiper you can send me an email and we can work on it together to fix this issue ([email protected]).

[TuomasK1989]

Hi,I have been trying this robibkuiper's modified version with Quad4PointMeshConfiguration to load point cloud with Unity webgl and it is performing quite good when the build and data are on my local machine. When I transfer this to my server it seems that either the data streaming is very slow or there is still some problem with loading-travelsel-rendering coroutines. It seems that loading or rendering comes behind tens of seconds so that if I move camera from room1 to room2 it still loads/renders room1 data for a while before starting to load/renser room2 data. My application is running steadily on 60fps and server cpu/memory does not seem to have affect to this at all. Did robibkuiper or mojtaba1995 get any improvements on this or does anyone has any hints how to continue with this? Networking should

[robibkuiper]

@TuomasK1989
I haven't been working on this for quite a while because of other projects i'm working on.

I'm not sure whats going on but it might be that the program keeps pushing new nodes on the loading coroutine list, but because the previous ones are not downloaded yet (probably your internet connection can't keep up) it just keeps adding the newest ones to the back of the list. What you might try to do is stopping all coroutines and clean the loader list before entering the new room. its been a while so im not exactly sure anymore in which script you can find theses lists.

[TuomasK1989]

I have tried this modified webgl code with somewhat larger datasets and it seems that when there is more points the initial point cloud loading takes a while. I am not sure what there happens when loading starts but for 500M point it takes about 90 seconds until Unity starts rendering point cloud and creating gameobjects. Is this some thing that is normal and how could this be avoided so that loading starts immediately after startup. In the main branch where multiple threads is used this startup is instant.

I am also interested if this solution could be adjusted to work with potree 2.0 converted datasets. What do You think about this?

[MerryLovasz]

Heya! I managed to get @robibkuiper version of the plugin working in webGL using Unity's Particle Systems. I've tested it with a pointcloud that's 340 million points. It takes about 4-5 seconds to load and the performance is 150-200+ fps (so it runs quite smoothly).

if you wanna recreate what I did, edit PointMeshConfiguration.cs to include a particle system prefab:

[SerializeField]
private GameObject particles;

And in the CreateGameObject function, add the particlesystem as a child and set the mesh renderer of the particle system to the renderer that Potree created:

var particleSystem = Instantiate(particles, gameObject.transform);
var shape = particleSystem.GetComponent<ParticleSystem>().shape;
shape.meshRenderer = renderer;

renderer.enabled = false;

Also don't forget to set
mesh.SetIndices(indecies, MeshTopology.Points, 0);
to
mesh.SetIndices(indecies, MeshTopology.Triangles, 0);
because otherwise the particle system won't set the particles on the correct position.

I've included the particle system prefab I used as a Unity package:
PointCloudParticleSystem.zip

You can tweak the StartSize and Max particles/Rate over Time settings to control how many points and detail you want to render.

Hope this can help someone!

[McSwan]

Using a particle system works on the web

Here's a raw example of creating a star field. There can be some issues using the particle system though. ie If we upload the points to gpu, then we can't delete them from local memory - saving space on webgl builds. Can't do that with particlesystem by the looks because of the setparticle update.

public class ParticlePoints : MonoBehaviour
{

    public int maxStars = 1000;
    public int universeSize = 10;
    public ParticleSystem theParticleSystem;
    private ParticleSystem.Particle[] points;

    private void Create()
    {
        points = new ParticleSystem.Particle[maxStars];

        for (int i = 0; i < maxStars; i++)
        {
            points[i].position = Random.insideUnitSphere * universeSize;
            points[i].startSize = Random.Range(0.05f, 0.05f);
            points[i].startColor = new Color(1, 1, 1, 1);
        }

    }

    void Start()
    {
        Create();
    }

    // Update is called once per frame
    void Update()
    {
        if (points != null)
        {
            theParticleSystem.SetParticles(points, points.Length);
        }
    }
}