Enhancement: shader code in js as verbatim function instead of array of strings

[estani]

I see text as follow:

vertexShader: [
    "varying vec2 vUv;",
    "void main() {",
    "vUv = uv;",
    "gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );",
    "}"`
].join("\n"),

It's not easy to read or modify. I propose using verbatim functions for that. This means:

Define helper function once (might be in-place if only used once):

function verbatim(fn){return fn.toString().match(/[^]*\/\*\s*([^]*)\s*\*\/\}$/)[1]}

then use as follows:

vertexShader: verbatim(function(){/*
    varying vec2 vUv;
    void main() {
        vUv = uv;
        gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
    }
*/})

IMHO this helps a lot to maintain and read the shader code more easily as it may be indented and there's no need to escape anything. The more complex the shader is, the greater the benefit.

I'm using this for a while now, but sadly I cannot assure it will always work, since the specification does not define how the Function.prototype.toString() should work. That being said, I haven't found out a browser that does something different.

[Mugen87]

I would prefer to use ES6 Template Literals some day. As you can see here, they are already supported in many browsers and even in node.js environments.

[estani]

It's similar, but not the same. Template strings (or literals) are meant for templating, which means they are not vebatim. I think it only bothers if you use ${...} formed strings, which I don't think you would anywhere in a shader segment... but still, varying vec2 u; //${b} won't work as intended.
In any case support is indeed pretty decent this days... besides the mentioned drawback I have nothing against it. As long as the text remains formatted and in one piece, I think is a cool option.

[makc]

you can always do

'\
    varying vec2 vUv;\
    void main() {\
        vUv = uv;\
        gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );\
    }';

or use \n\ if you want shaders to be readable in e g ff shader editor

[Mugen87]

From MDN docu:

Template literals are string literals allowing embedded expressions. You can use multi-line strings and string interpolation features with them.

I just wanted to draw attention to this great feature, because it finally avoids former inconvenient approaches (e.g. string concatenation or escaping newlines).

[Mugen87]

@makc It seems that escaping newlines is not uncritical. See here

[makc]

use string concatenation instead of escaping newlines because the latter isn't a part of ECMAScript

@Mugen87 oh yeah, what's this then?

[Mugen87]

@makc Um, probably the comment at stackoverflow is not up to date any more. The referred answer was from 2011.

If the LineTerminatorSequence is part of the standard now, it's of course a valid solution 👍

[bhouston]

Comments are removed in production code by many systems. Thus relying on them to exist seems dangerous.

[Mugen87]

Good point.

[bhouston]

My suggestion is to use a system like ThreeJS uses. Basically have raw *.glsl files and then have a compile time preprocessor that puts them into strings in a JS class. I recently moved the ShaderLib glsl code into their own files as well here in this PR:

#8223

The reason why this is preferred is because then you can use syntax highlighting and auto-completion and all those other nice tools on *.glsl files.

Maybe there is a way to make ThreeJS's build system more re-usable if we were to convert more of it into JavaScript.

[Mugen87]

@bhouston Can you recommend an editor or IDE for editing *.glsl files? Currently i'm using Atom editor with this glsl language package. It's okay, but unfortunately there is no real auto-completion...

[bhouston]

I am using Atom with glsl language pack as well now. I have auto-completion, it seems to just be the standard Atom auto-completion, thus it isn't that smart. Someone should integrate a glsl syntax checker into the glsl language package on Atom -- that would be pretty awesome.

[Mugen87]

Someone should integrate a glsl syntax checker into the glsl language package on Atom -- that would be pretty awesome.

I've checked out the package linter-glsl today. It uses the Khronos GLSL Validator to verify the correctness of the shader code. As you can see in the screenshot, errors are shown in the respective lines as soon as they occur.

IMO this Atom package is great! Unfortunately, it does not recognize our shader files in three.js because the GLSL reference compiler apparently expects *.vert and *.frag file extensions. With *.glsl you get the error message "Unknown shader type" in Atom when you open the file. Besides, the linter expects complete vertex and fragment shaders. If you open just a single chunk, it reports inappropriate errors like "float" type requires declaration of default precision qualifier.

[backspaces]

Just to add to the template string conversation & es6 in general:

I took Ed Angel's recent Mooc and used all es6. It was great! One of several features I liked was the template strings for shaders. Reduced confusing bunches of files, just included the template strings for shaders, below.

I used jspn to manage the modules and bundling, although today Rollup is a reasonable alternative.

One thing I didn't get around to was building shader modules. The idea would be to have shader fragments that the template strings could "import" as variables.

Es6 really does change everything. Modules, and generators for building trivial async functions before they arrive in es8. And modern workflow was very useful. And eslint for JS (great in Atom editor) .. finding bugs before running the code. Is there a glsl plugin for eslint yet? That'd be sweet!

I'm investigating three.js for the conversion of http://agentscript.org to es6 from coffeescript. Looks like fun! Any pointers to using three.js appreciated (docs look great, tutorials? books?) and tips on how to integrate into es6.

-- Owen

var fragmentShader = `
  precision highp float;
  varying vec4 vColor;
  void main() {
    gl_FragColor = vColor;
  }
`

var vertexShader = `
  attribute vec3 aPosition;
  attribute vec3 aNormal;

  uniform mat4 uIMatrix;
  uniform mat4 uMVMatrix;
  uniform mat4 uPMatrix;

  uniform vec4 uMaterialDiffuse;

  uniform vec3 uLight1Direction;
  uniform vec4 uLight1Diffuse;

  uniform vec4 uMaterialSpecular;
  uniform float uShininess;
  uniform vec4 uLight1Specular;

  uniform vec3 uEyePosition;

  varying vec4 vColor;
  void main() {
    vec3 E = normalize(uEyePosition);
    vec3 N = aNormal;
    vec3 L1 = normalize(uLight1Direction - aPosition);
    vec3 H1 = normalize( L1 + E );

    float Kd1 = max( dot(-L1, N), 0.0 ); // lambertTerm
    vec4 Id1 = uMaterialDiffuse * uLight1Diffuse * Kd1;

    float Ks1 = pow( max(dot(N, H1), 0.0), uShininess );
    vec4 Is1 = uMaterialSpecular * uLight1Specular * Kd1;
    if( dot(-L1, N) < 0.0 ) {
        Is1 = vec4(1,1,0,1);
    }

    vColor = Id1; // + Is1;
    vColor.a = 1.0;

    gl_Position = uPMatrix * uMVMatrix * uIMatrix * vec4(aPosition, 1);
  }
`

[dmnsgn]

@bhouston This PR might help recovering syntax highlighting for glsl.js files: #15473