UV offset / repeat should be part of materials rather than textures

[QuaziKb]

UV offset and repeat (and probably some of the other texture properties) are uniforms which are strangely passed from the texture into the material, which leads to issues where you have to clone textures per material (wasting a ton of memory) if you want per-material offsets/repeats, or roll your own custom shader. It's very inefficient to force people to do this, the best case being if someone wants to tile a texture across a surface based on the size of the object it's applied to, or use a texture as a spritesheet for animations. I can only see the current system as a hindrance with no real advantages, since the likelihood of needing shared UV offsets/Repeats on a "shared" texture is low, and would normally be better served by sharing a material. In anycase updating the uniforms off the texture is weird, since it really has no business being part of the texture, and ends up being confusing to the end user. For example, changing the offset/repeat when more than one map is applied to the material, example diffuse+normalmap, only uses the offset/repeat of the diffuse map, so the value of the normalmaps offset/repeat is completely useless in that context, when it really suggests it should affect the normal map offset/repeat. It's all around confusing.

I'm pretty sure the change is required in the "refreshUniformsCommon" function, but there's probably more to it. There's some changes required in the sprite plugin aswell. This would probably break a lot of peoples projects but it's a pretty big inconsistency in the texture/material API. It might be a better idea to make it an override that's normally null for materials, and when set ignores the values in the textures, just so it doesn't break everyones stuff.

[WestLangley]

Related post: #3549

you have to clone textures per material (wasting a ton of memory)

In what sense are tons of memory wasted?

the likelihood of needing shared UV offsets/Repeats on a "shared" texture is low

What do you mean by that?

If we keep the current approach, one thing that needs to be changed, is when a texture is cloned, the shared imaged should only be passed to the GPU once.

[QuaziKb]

if you have an object with a large number of frames/sheets, there's many useless objects that are created, and as you stated, the Image is copied multiple times onto the GPU which is extremely wasteful. Even more wasteful, if you have multiple objects which need to be at different points in different animations, you'll have to create a unique set of textures per unique material/object, so it quickly becomes annoying to handle, where a material uniform could much more easily be manipulated without having these extra texture offset/repeat data-structures built around a badly thought out API just to get it working. I have major slowdown in a certain use case of my application where i'm having to create these unique texture groups everywhere, and feel like the only solution would be to replace all the stock materials i use with shadermaterials to work around this single issue, or else fork my THREE.js version and make the necessary modifications.

In regards to the second statement:

I meant that the current paradigm only facilitates work in cases where you want to apply the same texture with the same UV offset/repeat, but thats generally a rare case, since it seems much more likely that you'd want to define this like other uniforms, per material, and share a material rather than just the texture.

The main caveat with the current sys. is that the offset/repeat only really exists as a single uniform which affects all textures in the shader, yet the fact it's attached to THREE.Texture means it can't be used properly as a uniform and fools people into thinking that offsets/repeats can be chosen differently for the various textures which can be applied on stock materials (i.e. you can define a different diffuse offset and normal map offset, but this isn't actually possible even though they can be set uniquely on the different textures). I can see it might have been remnant from the canvas renderer but it just makes no sense with the webGLrender / GLSL material system, which has largely usurped it.

mrdoob stated that it would need to be of the following form as a reason against having it per material,

material.map
material.mapOffset
material.mapRepeat
material.env
material.envOffset
material.envRepeat
... etc.

but again, the offset / repeat don't work per-map-type even now, so this isn't really a valid argument. It would waste too many uniforms anyway to have it per-map-type, (not to mention you usually only use one set of UV's so you wouldn't really have multiple offsets for a normalmap / bumpmap / diffusemap / specularmap). I think if you weigh all the options reasonable, it really should be a material property rather than texture property. I really don't feel there are any advantages to the current system. I feel animating materials through the UV offset is a major reason to even have the property at all, and can't even properly used for that. Not to mention if you didn't need those uniforms they could be omitted from the shader compilation, whereas now they're there as long as a map is.

[WestLangley]

@QuaziKb With all due respect, if you are going to propose a fundamental change to the library, such as the one you have proposed here, you have to provide a clear and compelling argument for that change. Arguing from the frame of reference of your particular application is not going to cut it.

That being said, I too, would like to see offset/repeat moved from Texture to the materials.

I believe it is reasonable to assume the following maps have the same offset/repeat values:

map
specularMap
normalMap
bumpMap
alphaMap
aoMap (future)
glossMap (future)
displacementMap (future)

The single exception is

lightMap

This is the way it is implemented now; even though each map has its own offset/repeat values, they are not honored.

So, we could add to MeshPhongMaterial, MeshLambertMaterial, MeshBasicMaterial, and SpriteMaterial, we would add

mapOffset // or mapTranslate
mapRepeat // or mapScale

We would remove offset and repeat from Texture.

In this way, implementing a sprite sheet is straight-forward. Each sprite has a material, and those materials share a single texture. The texture has an image. The texture no longer has to be cloned. On the GPU side, the materials would share a single shader program.

IMHO, this is a more-natural API.

I am trying to recall why the API was implemented the way it was. I expect there was a good reason.

[QuaziKb]

Sorry didn't mean to suggest it was particular to my example, simply that the current api leads to significant bloat when used for animating sprite sheets / offsets per unique object/material, with no real solution. Naturally the main purpose of having this as a uniform rather than simply baking the offset/repeat into the vertex UVs of the model is to animate the offset, and i was suggesting this can't be done without going around the API, making the uniform much less useful attached to textures than if it were to the material.

[jcarpenter]

I believe it is reasonable to assume the following maps have the same offset/repeat values:
map
...
alphaMap

FWIW, this behaviour (the current implementation) is causing significant frustration for me at this very moment. I am attempting to animate the reveal of a mesh by tweening the mesh material's alphaMap offset independently from it's diffuse map (which stays fixed). After some frustration I discovered via #4987 and this bug that this is not possible.

[WestLangley]

@jcarpenter What is the "bug" you are referring to? What is your suggestion for improvement?

[jcarpenter]

Correction: by "bug" I meant this issue. A mixup due to excessive time in a Bugzilla culture. :p I do understand that this is not a bug, but rather the intended behavior.

WRT to an improvement, based on my experience with traditional 3D content creation apps like Cinema 4D, I imagine the user being able to both:

• define offset/scale/repeat for each texture within a material, independent of the other textures, and
• define the parent material's offset/scale/repeat

Alternatively for the use case I'm working on ("attempting to animate the reveal of a mesh"), it would be fantastic to have an option for wrapS and wrapT that does not wrap at all. Per the attached GIF from Cinema4D, which has an option to disable tiling entirely for UVW mapping. Based on fairly extensive testing with the existing wrapS and wrapT methods, nothing like this is possible. Meaning my options for animate the reveal of items in three.js seems limited to tweening position and opacity of the entire mesh... Unless I'm missing something.

[mrdoob]

Agreed. The plan is to simplificate all this (using a single matrix3 in the shader) and having a offset/repeat (or translate/scale) per texture.

Anyone that wants to help with this would be much appreciated!

[QuaziKb]

@jcarpenter

sadly the only way to do this right now is with multiple meshes w/ different materials, or a custom shadermaterial. Both of which are kind of involved for a user who's just jumping into three.js workflow.

There's a constant tradeoff between usability, performance and extensibility. My suggestion would be to rewrite the way textures and materials work at present in the api, so that textures are strictly parameters you plug in, and the values that are actually uniforms in the shader like offset/repeat/wrap mode are linked specifically to the material. In some cases you don't want a uniform wasted on controlling UVs for textures that never need them to change (it'd be a huge waste to have 4*5 extra uniforms in a phong material if you only need it for diffuse), so it'd be cool if there was some magic behind the scenes that detected if specific UV's are needed and the texture is rebuilt to meet those demands, or if some parameter could optionally be passed in to specify the number of required adjustable UV's & what maps they offset & how, but it's a difficult issue to resolve.

[WestLangley]

The plan is to simplify all this (using a single matrix3 in the shader) and having a offset/repeat (or translate/scale) per texture.

@mrdoob

1. Do you mean per material.map so the texture transform is in the material? There are a lot of material maps, unfortunately. We could continue to assume all texture transforms are the same, except for lightmap.
2. Do you want to support rotation, too? If so, you have to also add the rotation center -- unless you want to hardwire the rotation center to the center of the texture.

[titansoftime]

This would be a much appreciated change. Having to clone textures just to put them at unique repeat values feels awfully cumbersome. Is this how other libraries do it?

[mrdoob]

1. Do you mean per material.map so the texture transform is in the material? There are a lot of material maps, unfortunately. We could continue to assume all texture transforms are the same, except for lightmap.

I think we should have a mat3 per map and should be composed from the Texture properties.

1. Do you want to support rotation, too? If so, you have to also add the rotation center -- unless you want to hardwire the rotation center to the center of the texture.

Rotation yes. Center or not... I'm not sure, but as far as I understand, all this can be encoded in a single mat3 for the shader (per map).

[WestLangley]

Here is a prototype showing how a Matrix3 can be passed to a shader and represent a transform defined by offsetX, offsetY, repeatX, repeatY, rotation, rotationCenterX, and rotationCenterY.

If center is not allowed as an option, then it should be hardwired to ( 0.5, 0.5 ).

Comments welcome.

EDIT: demo updated

[mrdoob]

THIS IS GREAT! 👍 👍

I think I would go with translation and scale (instead of offset and repeat) though.

[WestLangley]

What, exactly, should the new material properties be? There are a lot of material maps.

What should be removed from the texture properties?

I am assuming that wrapS/T should remain on the texture.

[mrdoob]

I think texture.offset and texture.repeat should be removed.

I think the new properties should be... texture.translation, texture.rotation, texture.scale, texture.center and texture.matrix. We will probably also need a texture.updateMatrix() method which would be called at render time. This, of course, has the challenge of making sure we only do it once even if the texture is reused.

[WestLangley]

Referring to the title of this post, and to the arguments in #5876 (comment), I thought the point was to move these properties to the material.

[WestLangley]

/ping @bhouston for comments.

[bhouston]

My thoughts are that the texture offset/repeat should be baked into the UVs as much as possible. It is easier. This is how UE4/Unity 5 do it for the most part.

The exception is that Unity 5 does have the ability to specify one global offset/repeat per material that is shared across all textures though -- but it doesn't affect what are considered to be baked maps such as the lightMap or ambientOcclusion maps (those do not make sense to be adjusted.)

The reason why I do not advocate for a lot of flexibility here is that professionally created models have proper UVs that make this generally not necessary -- content creation tools have ways of baking this. The other issue is that WebGL has a ridiculously low lower limit for Fragment Uniforms -- 16 Vec4. If you have to have a repeat/offset per map, and we will get a lot of maps soon, we are wasting Fragment Uniforms for very little value.

I actually added a repeat/offset and then brightness/gain controls per texture in Clara.io recently and I will be undoing these changes because it is leading to overflowing the Fragment Uniforms on low end devices -- such as every Apple iOS device. Although having repeat/offset and brightness/gain works great on desktops with NVIDIA 980s, but we have to design for everyone, not the high end machines. ;)

We have to treat Fragment Uniforms with respect and only use them when necessary. This I believe isn't one of those cases.

[WestLangley]

The exception is that Unity 5 does have the ability to specify one global offset/repeat per material

This is basically what three.js is doing now -- although it gets the offset/repeat from the diffuse map, and all other material textures use the setting from that map.

The proposal is to remove the offset/repeat from the texture, and add it to the material instead -- with perhaps a name change. Again, all the material textures would share the same settings (even though some users would not be happy about it). This would keep uniform usage low.

Unfortunately, we are not getting a lot of consensus on this.

[bhouston]

Doing what Unity 5 is doing is a good idea. I would put it on the material instead of the texture as well. You have my support.

[QuaziKb]

Per map isn't really ideal, but it could be solved by having it specified with flags/keys somehow when the materials built, in case it is needed.

[bhouston]

Generally the only reason to modify UVs in the shader is because you want them animated. If you just want to transform the UVs statically, we shouldn't be modifying the shader to add this functionality.

[WestLangley]

The only proposed change to the shader is to replace

vUv = uv * offsetRepeat.zw + offsetRepeat.xy;

with

vUv = ( uvTransform * vec3( uv, 1 ) ).xy;

[mrdoob]

Ok. How about this... We add texture.dynamic (false by default) which produces:

vUV = uv;

If the user sets texture.dynamic to true then we compute texture.matrix out of texture.translation, texture.rotation, texture.scale and texture.center, we pass that to the programa and we produce:

vUv = ( uvTransform * vec3( uv, 1 ) ).xy;

Of course, if texture.dynamic changes we need to recompile the program.

That way we get the best of both worlds?

[WestLangley]

@mrdoob

1. In your view, is scale a property of the texture or is texture.scale a property of the material? I hope it is the latter, because that is what this thread is all about.
2. We do not need a .matrix property. The new Matrix3 is a uniform that replaces the material uniform offsetRepeat. It is computed from the other parameters in the renderer and passed to the GPU just like any other uniform.

[mrdoob]

1. In your view, is scale a property of the texture or is texture.scale a property of the material? I hope it is the latter, because that is what this thread is all about.

I disagree with this thread. I don't think we should pollute the materials with mapMatrix, envMapMatrix, ... Nor mapTranslation, mapRotation, mapScale. I think it's cleaner if THREE.Texture has translation, rotation, scale and, maybe center.

1. We do not need a .matrix property. The new Matrix3 is a uniform that replaces the material uniform offsetRepeat. It is computed from the other parameters in the renderer and passed to the GPU just like any other uniform.

We kind of need something like that. Say that one reuses the same texture in different maps. We don't want to be computing the Matrix3 for every instance.

[titansoftime]

I disagree with this thread. I don't think we should pollute the materials with mapMatrix, envMapMatrix, ... Nor mapTranslation, mapRotation, mapScale. I think it's cleaner if THREE.Texture has translation, rotation, scale and, maybe center.

Would a texture still need to be cloned to have different repeat properties? In small scenes this is probably no big deal, but in large ones where there already are 40+ textures this a memory nightmare.

[bhouston]

UE4 has a sprite texture that allows for repeat/offset within the material:

https://docs.unrealengine.com/latest/INT/Engine/Paper2D/Sprites/index.html

[bhouston]

Maya, Softimage, 3DS Max, and UE4 have the separation of Bitmap/Image from Texture (and from the UV Generation) as @mrdoob is suggesting, but they also all use shader graphs to achieve this. Unity3D, which does not have shader graphs is the tool that incorporates the offset/repeat into the material itself, probably because it can not properly separate it into a shader graph node.

[mrdoob]

Maybe it is possible to have an Image created automatically per Texture if one didn't specify it separately, then that would achieve backwards compatibility? And you would only need to manipulate the Image directly if you wanted to do something tricky.

Exactly 😊

[MasterJames]

Probably a bit out of place to mention but I would like to recommend PTEX be incorporated as soon as possible.
http://ptex.us/PtexFile.html
If there's a way to make typical projections etc cast/convert to a NodeTexture (?) Or option that is a new more powerful and comprehensive base level texture mapping system? ...well then maybe that's something to consider ahead of time.
[Further to the same point:]
The concept with ptex is its not really a 2d image but a uv relationship so you can paint/stamp/project around a complex surface without the concept/challange of how to translate 2d to 3d, which is mathimaticaly a hack at best in comparison (always technically battling with destortion).
I just point out that ptex should have made more sense and been a priority 20+ years ago and should not to be considered an extension or second class performer, but really for me it's the other way around. It should be the old original way of declaritively saying how a 2d image is projected/stamped into the one true always functioning base level system of ptex. Anyway just enforcing the idea it should be integrated if not best to take a more central roll.
Thanks for the opportunity to make lofty suggestions. I was so happy that the ptex was made to a spec. I thought of it myself 10+ year prior but as a child had no power to define new specs etc. In hindsight I should have seen that maybe I could have tried to make a difference instead of the roll of observer I still maintain. So here is an attempt to undo a long standing wrong.

Maybe someone can start a new thread if anyone with a deeper understanding of the current methods potentially in Flux can make a more applicable proposal of how that would work in THREEjs.
Again thanks for the opportunity to be heard.

[mrdoob]

@MasterJames kind of off-topic... create a new thread please?

[QuaziKb]

Even if we had image so that "texture" could still be used, all the materials would need to be rewritten, since they dont actually support more than one uv offset/repeat. Obviously this could change, but would probably end up adding complexity since then redundant uniforms would be needed (how often do you want more than one set of offsets/repeats so that e.g. A normal map is offset from a diffuse) i think in the end for web where performance is premium the most common use case is one where there is one global offset/repeat which affects every map, and it just make sense for this to be on the material since it ends up being part of the materials architecture. Custom shader materials can handle edge cases just fine.

[mrdoob]

@QuaziKb Yep. That's what NodeMaterial tackles.

[evrimoztamur]

Isn't it the case that Texture uses the same OpenGL texture instance for each .clone(), or am I missing something. Is it actually reuploading it for each clone? If that's the case, then this is a very serious issue.

[rhys-vdw]

@evrimoztamur, I believe it copies the texture each time. You can check out what's going on using WebGL Inspector.

I tried tried every approach I could think of, including @sunag's workaround, but nothing worked. Based on my experimentation not yielding results, and the discussion above, I had a look around for how other other libraries handle sprite animation. I found Babylon.js's Sprite and SpriteManager API to be a solution that caters to my particular needs. It handles sprite sheets, offset and repeat, and animations. Maybe this is a higher level of abstraction than THREE.js aims to provide, but might be worth a look as a reference.

[karimbeyrouti]

@rhys-vdw: For a current project I ended up with a bastardized version of MeshBasicMaterial:
https://gist.github.com/karimbeyrouti/790d2e1a8c0137b16bae

When you set the Map, this automatically assigns the offset / repeat uniforms (which are in tucked away material ). You can easily set them separately - that will stop you needing to clone textures for now. ( working with r73)

[bhouston]

I've just submitted a PR that should address this issue, PR #8278

[EliasHasle]

@WestLangley wrote:

I believe it is reasonable to assume the following maps have the same offset/repeat values:

map
specularMap
normalMap
bumpMap
alphaMap
aoMap (future)
glossMap (future)
displacementMap (future)

Not always. I am currently using different repeat values for a map and a bumpmap on the same material (asphalt) to conceal the fact that both are tiled with rather small tiles. That way, I don't need to generate/have a big texture. It is very convenient. :-)

EDIT: Well, that's at least what I thought I had done. The trick was probably ignored. And I can achieve similar results by adding noise in the shader or something. The WestLangley's Matrix3 demo is cool.

[sunag]

I think this solves the problem of instances with different UV in Sprite. It is possible modify the vertex and pixel shader preserving the same texture.

https://threejs.org/examples/#webgl_sprites_nodes

It use SpriteNodeMaterial and Mesh with a shared PlaneBufferGeometry. The interface is not appropriate to Sprite but work. Maybe it can evolve to SpriteMesh to make an interface more friendly

[makc]

this thread is TL;DR for me. but I have just now discovered this code in r68 (old project, yep):

		// uv repeat and offset setting priorities
		//  1. color map
		//  2. specular map
		//  3. normal map
		//  4. bump map
		//  5. alpha map

		var uvScaleMap;

		if ( material.map ) {

			uvScaleMap = material.map;

		} else if ( material.specularMap ) {

			uvScaleMap = material.specularMap;

		} else if ( material.normalMap ) {

			uvScaleMap = material.normalMap;

		} else if ( material.bumpMap ) {

			uvScaleMap = material.bumpMap;

		} else if ( material.alphaMap ) {

			uvScaleMap = material.alphaMap;

		}

		if ( uvScaleMap !== undefined ) {

			var offset = uvScaleMap.offset;
			var repeat = uvScaleMap.repeat;

			uniforms.offsetRepeat.value.set( offset.x, offset.y, repeat.x, repeat.y );

		}

so, yeah...

if you are going to propose a fundamental change to the library, such as the one you have proposed here, you have to provide a clear and compelling argument for that change

I was trying to set different repeat on diffuse and normal maps and pulling my hair out because it did not work. since API places this setting in texture, I thought I could do that. so yes, how about saving my hair for compelling argument? this ticket is still open, 3js still has this setting on textures, and it is only respected for one of the textures = this is, in effect, already IS per-mateterial setting.

If you are not going to move this where it belongs, at least say it has no effect in the docs?

[gigablox]

@sunag For PR:
#11531

One issue I ran into:
https://jsfiddle.net/f0j2v3s8/

It seems like SpriteNodeMaterial transparency is lost, there is no combination of blending I can use on to get this example to work.

Any ideas?

[jimtang]

@karimbeyrouti wrote:

When you set the Map, this automatically assigns the offset / repeat uniforms (which are in tucked away material ). You can easily set them separately - that will stop you needing to clone textures for now. ( working with r73)

I believe this is obsolete since uniforms.offsetRepeat is changed to uniforms.uvTransform (r88).

Regarding the 'reuse a texture atlas with multiple Object3D instances' use case, I suggest a simple walk around:

1. store the UVs data (offset, repeat) in an atlas json object;
2. hook the onBeforeRender\onAfterRender function pair of Object3D;
3. in the before render callback, load the UVs data from atlas json object and set to material.map;
4. in the after render callback, reset it back;

It will result in:

1. only one Texture & one Material shared by multiple objects, no Clone is needed and the info.memory.textures counter will not increase;
2. but still all the other maps(normal, ao, displacement...) must comply with the same UV translation;

[Mugen87]

The topics in this issue are a bit mixed and already separately discussed in other issues. Merging this issue into two others.

The leading issue for uv channels/transform per texture #9457.

The leading issue for THREE.Image is #17766.

[QuickMick]

I was trying to set different repeat on diffuse and normal maps and pulling my hair out because it did not work. since API places this setting in texture, I thought I could do that. so yes, how about saving my hair for compelling argument? this ticket is still open, 3js still has this setting on textures, and it is only respected for one of the textures = this is, in effect, already IS per-mateterial setting.

If you are not going to move this where it belongs, at least say it has no effect in the docs?

exactly the same happened to me today :D

am i blind, or is this still missing?

it would just require e.g. following lines in front of the if/else chain, isn't it?

if(material.customUVTransform){
  uvScaleMap = material.customUVTransform;
}else ...

this is my current work around using the onBeforeCompile

onBeforeCompile(shader){
  shader.uniforms.uvScale = { type: "v2", value: new THREE.Vector2(x,y) };
  
    shader.vertexShader = shader.vertexShader.replace(
    "#include <clipping_planes_pars_vertex>",
    `#include <clipping_planes_pars_vertex>
     uniform vec2 uvScale;
    `
    );

    shader.vertexShader = shader.vertexShader.replace(
    "#include <uv_vertex>",
    `#include <uv_vertex>
      #ifdef USE_UV
        vUv = ( uvTransform * vec3( uv, 1 ) ).xy * uvScale;
      #endif`
    );
}