Feature Request: OnBeforeRender on Material

[DavidPeicho]

Is your feature request related to a problem? Please describe.

Let's say I have a custom material that is shared by two meshes. This material uses the inverse transform of the mesh. In this case, I have no way to send the inverse transform because it's dependent from the actual bound mesh.

When creating an app with Three.js, it's not really an issue because we control the flow of data. However, when wrapping Three.js in another library, it makes it somewhat difficult to share a ShaderMaterial whose parameters aren't independent from the camera / mesh drawn.

Describe the solution you'd like

On the top of my mind:

• Create a onBeforeRender callback on the material, that would be in charge of updating per-object uniforms. This is similar
  to the onBeforeCompile callback, but called each time something is rendered with this material.

Describe alternatives you've considered

• Use onBeforeRender on each meshes. Annoying because all meshes using this material must be updated and a user would need to attach the callback himself
• Clone the material for each mesh

Additional context

This is highly related to the issue #16922. I think the problem is similar.

[mrdoob]

• Use onBeforeRender on each meshes. Annoying because all meshes using this material must be updated and a user would need to attach the callback himself

I was going to suggest that...

This whole onBefore*, onAfter* part of the API is ugly and fragile... Have you tested NodeMaterial?

[DavidPeicho]

I haven't because I will most likely just add my entire GLSL in one node 😄 Also Right now I like how my shaders are clearly separated and I just use includes. Maybe I should take a closer look at NodeMaterial, any feature there that could help me achieve that on a material level?

I wish I could just do it on the meshes but that's not that easy, because I just don't know in advanced where my material will be attached.

It's fragile I agree, but right now what's happening is that there is a custom path basically for the default materials. Ideally I think it would be nice to be able to do exactly the same thing with any custom material, including having a way to modify transforms / camera.

[mrdoob]

@sunag how do you think this could be done with NodeMaterial?

[sunag]

@sunag how do you think this could be done with NodeMaterial?

Yes. This already is possible in the new NodeMaterial system.

three.js/examples/jsm/renderers/nodes/accessors/ModelNode.js

Line 17 in ea11dda

this.updateType = NodeUpdateType.Object;

You can update a shared uniform for each render object.

three.js/examples/jsm/renderers/nodes/accessors/ModelNode.js

Lines 39 to 55 in ea11dda

	update( frame ) {
	const object = frame.object;
	const inputNode = this._inputNode;
	const scope = this.scope;
	if ( scope === ModelNode.VIEW ) {
	inputNode.value = object.modelViewMatrix;
	} else if ( scope === ModelNode.NORMAL ) {
	inputNode.value = object.normalMatrix;
	}
	}

Is possible update per frame too with objects like camera

three.js/examples/jsm/renderers/nodes/accessors/CameraNode.js

Line 16 in ea11dda

this.updateType = NodeUpdateType.Frame;

[sunag]

I can implement a solution for WebGL too after this PR ( #21117 )... I am currently implementing a light system for selective lights or global using NodeMaterial.

[DavidPeicho]

I had a look a bit at the Node system today. I still need to connect myself the mesh to the update though no? I don't see how that would work automatically.

Where is the frame.object attribute field?

EDIT: Ok my bad, now I understand the code to hook that up is only for WebGPU right now

[sunag]

https://raw.githack.com/sunag/three.js/nodematerial-shared-uniforms/examples/webgpu_instance_uniform.html

Source: https://github.com/sunag/three.js/blob/8f9cbfa9c15d45a5ce8d3edba8cebc246c039592/examples/webgpu_instance_uniform.html

I made a example, the color is updated per mesh.color in render object moment.
But it seems there is a problem... WebGPURenderer still does not support material instance? @Mugen87

[Mugen87]

WebGPURenderer still does not support material instance?

What do you mean with "material instance"?

[sunag]

What do you mean with "material instance"?

A single material for multiples meshes.

[sunag]

If replace materialTemp to material in addMesh() only a mesh is rendered on screen at a time.
https://github.com/sunag/three.js/blob/8f9cbfa9c15d45a5ce8d3edba8cebc246c039592/examples/webgpu_instance_uniform.html#L104-L109

[Mugen87]

Okay, after investigating this issue a refactoring of how bindings are managed is required. Right now, the binding and the actual data are handled in a single object. In the below example, bufferGPU represents the UBO and array the actual data.

three.js/examples/jsm/renderers/webgpu/WebGPUUniformsGroup.js

Lines 22 to 23 in cb99e7e

	this.array = null; // set by the renderer
	this.bufferGPU = null; // set by the renderer

If a material is shared among multiple 3D objects, there is only one instance of bindings objects and thus only one data storage. This is not a problem for textures (because textures are defined on material level) however model related data like the MVP matrix can't be processed this way. They have to be managed per 3D object and the UBO needs to be updated per object with the correct data.

I need some time to think about a possible solution for this...

[DavidPeicho]

@sunag let me know if you need any help for the WebGL version (or even WebGPU).

[sunag]

@Mugen87 I found the problem, is related with the implementation of NodeMaterial... I will create um PR to fix this.