Documenting available attributes and uniforms on the Reference page

[JetStarBlues]

Actual Behaviour

There is currently no documentation on the attributes and uniforms that are available when writing shaders in p5. For example, the aPosition attribute is used in all vertex shader examples, but there is no associated documentation on the Reference page. Similarly, there are a lot of useful attributes and uniforms (such as uProjectionMatrix), which do not appear in any of the available examples. The only way to learn about them is to dig into the p5 source code.

Expected Behaviour

Ideally, a list of the available uniforms would be listed in the setUniform() documentation page. Each listing would have a brief description of what the uniform does. (This would have the added bonus of giving the user a hint as to which uniform names are unavailable for use).

Similarly, a list of available attributes would be listed. If a setAttribute() method existed in p5, its documentation page would be the ideal place to list them. However, they can be (together with the default uniforms for consistency) be listed in the p5.Shader documentation page.

Since a lot of these default attributes and uniforms are used for the default shaders, not all are immediately relevant to outside use. Here is a list of the most useful I have been able to identify so far (source1, source2):

• Attributes
  • aPosition
  • aTexCoord
  • aNormal
  • aMaterialColor (aVertexColor might be a better name)
• Uniforms
  • uProjectionMatrix
  • uModelViewMatrix
  • uViewMatrix
  • uNormalMatrix

Ideally, additional examples (such as this one (live, src by aferriss ) would be added to demonstrate the usage of some of these attributes and uniforms.

Would you like to work on the issue?

Yes

[aferriss]

I think this would be a great addition to the docs. I've often found myself struggling to find this information; it would be great if we could more easily surface it for more people :)

It warrants a separate issue, but looking at this I also think that it would be worth also making an additional uModelViewProjectionMatrix uniform that has already done the matrix multiplication on the cpu side. I think this is a pretty common pattern and should be an easy performance win.

[JetStarBlues]

Ah, sounds good! I'll make generous use of the great descriptions you have written in the comments in the examples =)

[JetStarBlues]

Hi, just wanted to check in and mention that I do plan on creating a pull request for this issue. I haven't been able to find as much time as I would like to tackle this.

[aferriss]

Thanks @JetStarBlues !

I think to make it easier (both on yourself and to get the PR approved) I suggest starting with smaller PR's than trying to do the whole thing.

Would it make sense to add a separate documentation page for "Built in Uniforms and Attributes", and that can be linked to from the setUniform page?

I think that expanding the breadth and completeness of our documentation fits into the accessibility statement. We can make these features easier to find, read, use, and understand for people who are new to shaders / graphics / p5. Providing small examples of usage will also help people understand how each variable might be used.

@stalgiag Any notes? Does this proposal seem sound to you?

[JetStarBlues]

Thank you for the feedback 🙏
I love the idea of starting with a separate page and linking to it!

[stalgiag]

Hi all! I love the idea and I agree with @aferriss' suggestion of separating it. This will free us up to have an extensive entry on attributes and uniforms without adding information overload to the reference.

I am unsure of what the best approach would be for making this separate page. Would you want to have embedded examples or would it also work as a wiki?

[aferriss]

Placing it on the wiki is probably the lowest friction option, though I do think it would be a good addition to the official reference as well.

I'm actually not sure how to spin off a separate page for something that doesn't already have a dedicated method, since all the docs are auto-generated. I'm sure it's possible but might take some poking around.

[JetStarBlues]

Another place it can potentially go is examples. Where it can be considered "optional" reading for the curious (linked to by the setUniform reference page), and can support embedded examples.

[stalgiag]

Hm it depends on how you want to design it. This is a bit outside of the norm for the examples page but I think there is some flexibility if you were to think through it with a specific example in mind. I don't know if it would be possible to make a multi-sketch 'Example'.

[JetStarBlues]

I've had a chance to start working on this.

Here's some screen captures of what I have so far. I can create a pull request if the direction is agreeable.

Screen capture of rendered page

Screen capture of example in index page

---

Placing the descriptions as an example page has the upside of supporting a live demo.

The downside, is that the vertex and fragment shader source code (demonstrating the use of these values) is not immediately visible. Like other shader examples, it is up to the user to dig into the p5.js-website repo to find them.

To make the page slightly more useful/accessible, it would be ideal to have a link to it in the p5.Shader and setUniform reference pages. However, it seems that the content in the reference pages is generated automatically from the comments in the main p5 repo (and would require a separate PR).

---

Here is the full (modified) source code of the example provided.

---

Here are the descriptions in text form.

p5.js provides some default attributes and uniforms that can be used in your shader programs. Attributes provide per vertex information that can be used in a vertex shader. Uniforms provide information that can be used in both vertex and fragment shaders. (All vertices in a vertex shader, and all pixels in a fragment shader receive the same uniform value as input.)

Attributes

• aPosition
  • vec3 (x, y, z)
  • position associated with a vertex
• aTexCoord
  • vec2 (x, y)
  • texture coordinates associated with a vertex
• aNormal
  • vec3 (x, y, z)
  • normal associated with vertex
• aMaterialColor (aVertexColor might be a better name)
  • vec4 (r, g, b, a)
  • color associated with a vertex
• The following attribute names are reserved:
  • aAmbientColor

Uniforms

• uViewMatrix
  • mat4 (4x4 matrix)
  • defines attributes about the camera?? such as position and focal length
• uProjectionMatrix
  • mat4 (4x4 matrix)
  • used to convert 3D world coordinates into screen coordinates
• uModelViewMatrix
  • mat4 (4x4 matrix)
  • combination of the model and view matrices
• uNormalMatrix
  • mat3 (3x3 matrix)
  • ?
• The following uniform names are reserved:
  • uViewport
  • uPerspective

🙏
I would love any suggestions on any improvements that can be made.

For the matrices, I am not familiar enough with them to write proper descriptions. Suggestions are very much welcome.

For the view matrix, which camera is used? Does each WEBGL sketch start with a default camera initialized?

---

I have excluded the following reserved uniform keywords from the documentation (see list below).

These uniforms are set by a call to _setFillUniforms. They are used by the default fill shaders, but are not needed if we are using a user defined shader.
A better solution (than telling users to not use these uniform names) is to modify the _setFillUniforms function, to first check if we are using a default shader, before proceeding to set these uniforms.

p5.RendererGL.prototype._setFillUniforms = function(fillShader) {
	fillShader.bindShader();

	if ( fillShader !== this.userFillShader )  // NEW
	{
		fillShader.setUniform('uMaterialColor', this.curFillColor);
		...
	}

	fillShader.bindTextures();
}

• uMaterialColor
• isTexture
• uSampler
• uTint
• uSpecular
• uEmissive
• uShininess
• uUseLighting
• uPointLightCount
• uPointLightLocation
• uPointLightDiffuseColors
• uPointLightSpecularColors
• uDirectionalLightCount
• uLightingDirection
• uDirectionalDiffuseColors
• uDirectionalSpecularColors
• uAmbientLightCount
• uAmbientColor
• uSpotLightCount
• uSpotLightAngle
• uSpotLightConc
• uSpotLightDiffuseColors
• uSpotLightSpecularColors
• uSpotLightLocation
• uSpotLightDirection
• uConstantAttenuation
• uLinearAttenuation
• uQuadraticAttenuation

[aferriss]

This is a great start! I think you could create a draft PR, it may be easier to keep track of changes that way.

A couple suggestions...
For the attributes and uniforms, I think it would be helpful to give context about what each line represents. So instead of

• aPosition
  • vec3(x, y, z)
  • position associated with a vertex

we do this instead:

aPosition
Type: vec3(x, y, z);
Description: The position associated with each vertex.

Alternatively, I think a table could work as well:

Name	Type	Description
aPosition	vec3(x, y, z)	The xyz position value associated with each vertex
aTexCoord	vec2(u, v)	The uv texture coordinate values associated with each vertex

Here's a sample description for normal matrix: The transpose inverse of the model matrix. Typically used in lighting calculations.

I believe we're also missing uModelMatrix

[JetStarBlues]

create a draft PR

Sounds good.

aPosition
Type: vec3(x, y, z);
Description: The position associated with each vertex.

I see, will do!

Alternatively, I think a table could work as well:

I agree, it would definitely be a cleaner way to present the material. However, the example descriptions are constrained to using only elements that can appear within a <p> tag. (When the example pages are generated, everything following @description gets wrapped in a <p> tag. The list above is achieved by a bit of a hack using unicode dots and  .)

Here's a sample description for normal matrix: The transpose inverse of the model matrix. Typically used in lighting calculations.

Noted. Thank you!

I believe we're also missing uModelMatrix

Unfortunately no such uniform exists. The RendererGL code uses only the following matrices (which are passed to uniforms here). I like the idea of having a uModelMatrix that users can access. A separate PR can be made to add it? Maybe the same PR that adds the uModelViewProjectionMatrix you mentioned above.

/**
 * model view, projection, & normal
 * matrices
 */
this.uMVMatrix = new p5.Matrix();
this.uPMatrix = new p5.Matrix();
this.uNMatrix = new p5.Matrix('mat3');

[JetStarBlues]

Updated style

[JetStarBlues]

For the view matrix, which camera is used? Does each WEBGL sketch start with a default camera initialized?

It seems the answer is yes:

// Camera
this._curCamera = new p5.Camera(this);
this._curCamera._computeCameraDefaultSettings();
this._curCamera._setDefaultCamera();

The default camera has position and orientation:

• position (eye) : (0, 0, (height / 2) / tan(PI / 6))
• where camera is pointing (center of scene): (0, 0, 0)
• direction of 'up' from camera: (0, 1, 0)

It is also defaults to perspective projection:

• vertical field of view: PI / 3
• aspect ratio: width / height
• near clipping plane: eye.z / 10
• far clipping plane: eye.z * 10

---

• uViewMatrix
  • Type: mat4 (4x4 matrix)
  • Description: defines attributes about the camera such as position and focal length

In the p5.Camera context, would it be better to replace focal length with projection in the description? (Disclaimer, I don't have a strong background on this topic.)

And have the description updated as,

contains attributes about the current camera such as position and projection

[JetStarBlues]

I was thinking about how to add uModelMatrix.
Since we have a uViewMatrix and a uModelViewMatrix:

modelMatrix * viewMatrix = modelViewMatrix
modelMatrix = modelViewMatrix / viewMatrix 

It seems that matrix division is not a thing. Luckily the p5.Matrix class already has the inversion function that is used instead.

So _setMatrixUniforms() can be changed to:

p5.Shader.prototype._setMatrixUniforms = function() {
  ...

  const projectionMatrix = this._renderer.uPMatrix;
  const viewMatrix = this._renderer._curCamera.cameraMatrix;
  const modelViewMatrix = this._renderer.uMVMatrix;

  /* Extract modelMatrix from modelViewMatrix

                   view * model = modelview
     viewInverse * view * model = viewInverse * model
                          model = viewInverse * model
  */
  let inverseViewMatrix = new p5.Matrix();
  inverseViewMatrix.invert(viewMatrix);  // set as inverse of viewMatrix

  inverseViewMatrix.mult(modelViewProjectionMatrix);
  const modelMatrix = inverseViewMatrix;  // alias

  // Order of multiplication is: projection * view * model
  const modelViewProjectionMatrix = projectionMatrix.copy();
  modelViewProjectionMatrix.mult(modelViewMatrix);

  //
  this.setUniform('uModelMatrix', modelMatrix.mat4);
  this.setUniform('uViewMatrix', viewMatrix.mat4);
  this.setUniform('uModelViewMatrix', modelViewMatrix.mat4);
  this.setUniform('uProjectionMatrix', projectionMatrix.mat4);
  this.setUniform('uModelViewProjectionMatrix', modelViewProjectionMatrix.mat4);

  ...
};

Not sure what this extra calculation (per draw() frame) means for performance.

What do you think @aferriss?

[aferriss]

This feels a little backwards to me. Possibly this is just a byproduct of the way that the p5 renderer is set up, but for whatever reason the model matrix never exists on its own and is always applied to the view matrix. I think conceptually it would be good to separate the construction of the two, which would eliminate the need for the inverse calculation you've come up with.

You can see where uMVMatrix gets created here and then later is translated / rotated / scaled. I think what should happen instead is that we have a separate model matrix object that has the translation / rotation / scale applied, and then is later multiplied with the view matrix.

In general I think exposing and separating those matrices like you did is a good thing and feels much closer to how matrices are set up in other environments. Your solution is possibly the simplest to implement, but I feel it's more of a hack than a proper fix. Unfortunately, I think the uMVMatrix get's used in many places around the code base so it could be a tricky thing to fix and debug. I doubt it would have much impact on performance. Doing that calculation on the cpu side is vastly more efficient than if you had to do it on the gpu.

I think we're veering off topic a bit here as well, and since we're talking about adding some additional functionality to the library, would you mind making a new topic to outline these ideas? It may be a tough sell in terms of accessibility. Maybe there is a case to be made that this makes p5 more aligned with the wider ecosystem of tutorials, books, and other educational materials, and thus makes p5 an easier stepping stone to learn graphics / shader programming with.

Thanks again for looking into this, it's all super interesting!

[JetStarBlues]

Thank you for the feedback!

I agree that drawing a distinction between the two within the source code would be the ideal solution.
I don't have a strong understanding of the model and view matrices to be able to separate them without guidance.

Would you mind making a new topic to outline these ideas?

Will do

[JetStarBlues]

Here's a screenshot of the current documentation state:

---

And here is the content as text:

p5.js provides some default attributes and uniforms that can be used in your shader programs. Attributes provide per vertex information that can be used in a vertex shader. Uniforms provide information that can be used in both vertex and fragment shaders. (All vertices in a vertex shader, and all pixels in a fragment shader receive the same uniform value as input.)

Attributes

• aPosition

  • type: vec3 (x, y, z)
  • description:
    • position associated with a vertex

• aTexCoord

  • type: vec2 (x, y)
  • description:
    • texture coordinates associated with a vertex

• aNormal

  • type: vec3 (x, y, z)
  • description:
    • normal associated with vertex

• aMaterialColor (aVertexColor might be a better name)

  • type: vec4 (r, g, b, a)
  • description:
    • color associated with a vertex

• The following attribute names are reserved:

  • aAmbientColor

Uniforms

• uViewMatrix

  • type: mat4 (4x4 matrix)
  • description:
    • contains information about the current camera such as position and rotation
    • used to ?

• uProjectionMatrix

  • type: mat4 (4x4 matrix)
  • description:
    • contains information about the current projection such as near and far clipping planes
    • used to convert 3D world coordinates into 2D screen coordinates

• uModelViewMatrix

  • type: mat4 (4x4 matrix)
  • description:
    • combination of the model and view matrices
    • used to ?

• uModelViewProjectionMatrix

  • type: mat4 (4x4 matrix)
  • description:
    • combination of the model, view, and projection matrices
    • used to ?

• uNormalMatrix

  • type: mat3 (3x3 matrix)
  • description:
    • transpose inverse of the model matrix
    • typically used in lighting calculations

• The following attribute names are reserved:

  • uStrokeWeight

---

I've added uModelViewProjectionMatrix to the uniform list as I'm currently working on a PR to add (just) it to the source code.

I've removed uViewport and uPerspective from the list of reserved names, as I'm currently working on a PR to limit their scope. Like the default fill shader uniforms mentioned earlier above, they will only be set when not using a user-defined shader.

[JetStarBlues]

I've added uStrokeWeight to the reserved list because its presence in a shader file is what is used to determine whether a given shader is a stroke- or fill-shader (src).

p5.Shader.prototype.isStrokeShader = function() {
  return this.uniforms.uStrokeWeight !== undefined;
};

If a shader is marked as a stroke-shader, then the list of attributes available to it changes. This documentation assumes that a user shader is always created as a fill-shader.

We could potentially encourage the use of user-defined stroke-shaders. This would require documenting:

• the use of uStrokeWeight as an indicator
• the separate attributes available to a stroke shader:
  • vec3 aPosition which now corresponds to lineVertices
  • vec4 aDirection which corresponds to lineNormals

If we go down this path, the code that marks a user-defined shader as a stroke-shader is currently faulty and would need to be fixed. (Maybe this should be fixed regardless...)

• it currently checks (via isStrokeShader()) whether the uStrokeWeight uniform is set to determine whether the shader is or isn't a stroke-shader
• however, this uniform does not exist until the shader is initialized via the subsequent call to s.init()
• this means that the first time shader(s) is called, the user shader is marked as a fill-shader (this._renderer.userFillShader = s)
• it is only on the second call of shader(s) that the uStrokeWeight uniform is available, and the shader is correctly recognized as a stroke-shader (this._renderer.userFillShader = s).
• however at this point, the shader has now been marked as both a fill-shader and a stroke-shader, and is used twice accordingly.

p5.prototype.shader = function(s) {
  ...

  if (s.isStrokeShader()) {
    this._renderer.userStrokeShader = s;
  } else {
    this._renderer.userFillShader = s;
    this._renderer._useNormalMaterial = false;
  }

  s.init();

  return this;
};

p5.RendererGL.prototype.drawBuffers = function(gId) {
  ...

  if (this._doStroke && geometry.lineVertexCount > 0) {
    // user shader is used here
  }

  if (this._doFill) {
    // and also here!
  }
  ...
};

• here's a live demo showing this bug
  • I think the solution would be to call s.init() first so that the uniforms are available. Then do the isStrokeShader() check to determine type.

[aferriss]

@JetStarBlues I wanted to see if this was still in progress and where you ended up with documenting the uniforms. I've seen some other issues lately for people asking for documentation. Since this work was already done it might be worth putting up as a PR to review.

Thanks!

[JetStarBlues]

Hi @aferriss

I will clean this PR up. There are a few outstanding questions, but I'll compile a list at the end.

One thing I would love your opinion on is this PR (and related issue) that limits the scope of the default uniforms... this would greatly reduce the number of things we have to tell people they can't use.

---

I will also create a single PR that updates all the shader examples. It will be a combination of the discussions (and draft PRs) here:

• https://github.com/processing/p5.js-website/issues/1021
• https://github.com/processing/p5.js-website/issues/1016
• https://github.com/processing/p5.js-website/issues/1020

---

Two PRs in total.
Or potentially one, since the order of the examples seems to be determined by the number prefix on the filename, and it would be ideal to place this documentation as the first entry (which would require a rename of all examples to place this as 00_)

00_geometries.js
01_sine_cosine_in_3D.js
02_multiple_lights.js
03_materials.js
04_textures.js
...

[aferriss]

Thanks @JetStarBlues! I left some comments on your PR

[matdombrock]

I found this issue trying to get to the bottom of the aPosition attribute in the shader examples. It would be great if this were officially added to the docs!

[Kusou1]

It would be greatly appreciated if the p5.js team could prioritize updating the documentation to include a comprehensive explanation of the Default Shader Input. This would greatly benefit the community and ensure a better learning experience for all users.