WebGLRenderer: Proof of concept for copying transmission pass, improving rendering performance #28423
Conversation
gkjohnson
changed the title
gkjohnson
changed the title
📦 Bundle sizeFull ESM build, minified and gzipped.
🌳 Bundle size after tree-shakingMinimal build including a renderer, camera, empty scene, and dependencies.
|
# Conflicts: # src/renderers/WebGLRenderer.js
I'm not yet sure what all these limitations mean. Any chances to show with live examples what use cases would break? E.g. there are users who use stencil with transmission so would the new code path break their apps? Besides, you did not update the array camera/XR code path which also calls |
Stencil Buffers Right now we render all the geometry twice which means we're writing stencil state to both the canvas buffer and the transmission render target buffer. That means that after rendering with transmission the canvas stencil buffer has stencil state remaining that can be used again in a subsequent "render" call as long as the buffer is not cleared. However generally the stencil buffer cannot be copied between buffers. That means if we transition to copying contents from the rendered transmission buffer then we cannot copy the stencil state, and stencil buffer contents will not be present after calling "render". Internally it won't even be available after the opaque rendering as it was previously so transparent objects won't be able to use the stencil state, either. Here's a quick example that wouldn't work afterward. It's a scene that renders a sphere to the stencil buffer & renders a transmissive box. Then afterward renders a blue quad which colors in the pixel with stencil values: https://jsfiddle.net/n15tdLv2/1/ Blending Again if we're rendering a scene without autoclear and the scene includes objects in the opaque queue that use custom blending (additive, multiplicative, etc) then they will not be blended properly with existing canvas contents. This is because we're rendering all contents to the transmission buffer (which will likely not have the same contents as the canvas) so the blending results will not be the same. And when copying data from the transmission buffer to the canvas we cannot perform the appropriate blending operations per pixel. Unique Backgrounds It's not always the case that the background we're rendering in the transmission buffer is the same as the one we want when rendering to the canvas. As commented in #28434 & #25819 - we want to render the background as transparent white when clear alpha is not set to 1. But we'll still want to respect the use set clear color and alpha when rendering the canvas background.
Once these other questions are figured out I'll update the PR. |
|
Hmm, these limitations seem like something developers will have to battle with and we'll have to explain every time... |
I agree to an extent - on the other hand it's a pretty huge performance gain. And people already complain about transmission performance, no? (1, 2, 3, #28073, #27108, ...more) Also I think it's the role of documentation to explain in what conditions the "slow" path kicks in so users can decide what to do. Right now everything is slow even though it doesn't need to be. Transmissive materials seem almost unusable in seemingly simple use cases since it nearly doubles the render time for even a small transmissive surface. We had to remove it from our application due to this which is why I took a look at what could be done. Something like Batching geometry may help with draw calls but it still won't help performance degradation due to rendering expensive shaders twice. It would just be nice to be able to use this in some way without such a large performance hit because it's otherwise a nice effect. Even if the solution is less "smart" - like adding a flag to the renderer. |
|
It occurs to me that we could render everything to the target buffer first (canvas, render target) and then copy that into the transmission buffer using a
The only thing that isn't immediately clear is how to address is the forced white background clear from #25819 in order to ensure the transmission tint can be rendered. Some possible solutions:
Both solutions would mean we would retain the existing background tint in addition to the white color which I think is okay (and possibly an improvement). I'm not sure if there's a way to copy directly from the canvas frame buffer and immediately blend on a white background otherwise this would be the easiest approach. Do you know of one @Mugen87? If we don't foresee any issues for merging the above approach and we can agree on a solution to the background issue then I can adjust this PR to try it out and see how it works. edit: it seems tonemapping + color spaces, etc would be an issue, though 😮💨 |
From my experience transmission and good performance in real production is already an hard subject. Also I like the performance Warning they add in drei about the transmission property, explaining to people its not a trivial task and might have a strong impact on performance. |
@Makio64 So you suggest to use the material from drei-vanilla? Is it better for performance you think? |
|
@about-daniel Yes, when i wrote that it was more performant. The main reason I think its because you have more control on the render target (render of whole scene) use as the base of the effect ( especially to select the appropriate size, when to render it, etc.. ) |
@Makio64 Thanks! Because I'm using the MeshTransmissionMaterial from react drei in a project for a client with nextjs, fiber and drei. The problem is that with that material the GPU goes up to 100, my Mac Air M2 heats up and on other computers it lags and performances are poor. If I disable the material everything goes back to normal and working. I don't know how to optimize it while maintaining the transmission effect. Any suggestions?? 
|
5 participants
Related issue: --
Description
Currently WebGLRenderer renders all opaque objects twice when rendering a transmission material - once for the transmissive pass and once for the main render pass. This PR demonstrates the performance improvements gained if we render the opaque materials to a render target with a depth attachment and then copy both the depth and color values to the final target instead of rerendering the objects.
To exaggerate the performance gains I added a ball of 10,000 cubes to the transmission example:
Before this change calling "render" takes ~27-28ms and after it takes ~15-16ms on my machine. An improvement of ~42%.
The limitations of this approach are that if the background is not cleared then unique styles of blending will not work correctly with existing render target / canvas contents. And likewise if stencil rendering is used then it will not be able to work with existing target values and they cannot be copied across buffers.
We could handle the cases specially, though, and skip copying the transmission buffer contents if stencil rendering is used on opaque rendered materials or color buffer is not cleared.
Edit: I guess we'll need to account for the case background colors are handled differently, as well?