reusing a past video texture

[mattetti]

I've been trying to write a shader that would use 2 samplers, one coming from the current video frame I'm using as a base, and one from a previous frame.

I hacked my way into adding }).onVideoFrameCallback((metadata, texture) => to the loader so I can get the last modified frame, store the frame used 30frames ago and I'd like to reuse that texture in my shader via an uniform sampler2D.

However, I'm getting stuck making the texture available. I tried to clone the texture instance, rename it to match the new sampler uniform (altTexture) and tried to force the uniform binding but without any luck:

            const altTexture = new Texture(texture.renderer);
            Object.assign(altTexture, texture);
            altTexture._sampler.name = "altTexture";
            altTexture._textureMatrix.name = "altTextureMatrix";
            videoPlane.textures[1] = altTexture;
            videoPlane.textures[1]._setTextureUniforms();

I realize I could use another video element as a source, but I'm exploring interesting glitching effects and would prefer to have fun with the one texture.

Is it something that is currently doable and I just missed a step? If it's not doable, coud you point me towards a potential hack/solution? I'd be more than happy to contribute back if that's something that would be interesting to the project.

[martinlaxenaire]

Hey @mattetti,

I remember having experimented with that kind of glitches effects a while ago. I was just using the same video twice with 2 separate textures and started one of the video playback after a small delay. It may not be the most optimized way to do this tho.

Looking at your code block I wonder why you'd not written something like that instead:

const altTexture = new Texture(texture.renderer, {
    sampler: "altTexture"
});
altTexture.copy(texture);

But maybe I've misunderstood what you were actually doing?

Cheers,

[mattetti]

@martinlaxenaire setting up the texture as shown makes a lot of sense. Thank you. But I'm still not sure how to get the texture passed via uniforms.
Is calling _setTextureUniforms() on the new texture, the right way?

[martinlaxenaire]

@mattetti I'm still not sure to understand what you are trying to do?

Anyway you shouldn't try to access those private properties and methods. The copy() method should do the trick.

[mattetti]

Unfortunately, that doesn't work and even break the main texture. (everything runs but I don't have any textures).

What i'm trying to do is what you achieved by using 2 instances of the same video but delayed. But I'm trying to achieve the same thing with only 1 video and sample the video at different times. Does that make sense?

[martinlaxenaire]

Hey @mattetti,

Yeah I mean I understood what you were trying to do, I just didn't understand what you were doing with your custom onVideoFrameCallback function.

Anyway I think the simplest way to achieve that is to just clone/duplicate the video element, create a second texture with it and start its playback after a small delay.

Cheers,