gltfpack: Support KHR_implicit_shapes and KHR_physics_rigid_bodies

[Shinmera]

Hi,

I'd really love to use gltfpack to optimise the files I have for use in release scenarios. However, I rely heavily on the KHR_implicit_shapes and KHR_physics_rigid_bodies extensions' metadata for definition of physics properties and colliders and so on (along with another extension of my own to add extra level and animation data).

As I understand, gltfpack strips unknown extensions out entirely, making the files unusable in my context. I suppose for a first step it would be great if those two extensions could be supported, but in general I would be interested in a way to preserve third-party extension metadata.

I understand that due to the transformations gltfpack makes some extensions that rely on the original data indexing would invariably be thrashed. However, a lot of metadata could be transparently preserved by simply keeping the relevant extension blocks attached and avoiding reduction of the nodes the blocks are attached to.

Whatever the case, I thank you for all the work on this so far, and would be very interested to hear about possible workarounds or ways forward.

[zeux]

I understand that due to the transformations gltfpack makes some extensions that rely on the original data indexing would invariably be thrashed. However, a lot of metadata could be transparently preserved by simply keeping the relevant extension blocks attached and avoiding reduction of the nodes the blocks are attached to.

The problem is that this is usually insufficient, and glTF files don't by themselves provide enough information to identify specific data that becomes invalid if a JSON subtree is kept as is. In some tools like glTF-Transform you can implement extension support via pluggable modules, but I have no plans to provide support like this for gltfpack.

This can perhaps be best seen in the two extensions you noted. KHR_implicit_shapes actually will work as is from what I can tell by skimming through the extension text - this one could be preserved by simply keeping the JSON. However, without KHR_physics_rigid_bodies it is not very useful, and KHR_physics_rigid_bodies allows to reference nodes via geometry.node to be able to specify collision shapes as meshes. (there might be other references to the rest of the tree, unsure)

Because gltfpack will both remove existing and insert new nodes, the existing node indices can become invalid; so any extension that refers to existing nodes (as well as existing meshes, materials, images) can not be preserved exactly and requires reprocessing. Moreover, if gltfpack were to preserve unknown extensions (in hopes that it will work in some cases), it would sometimes generate invalid files from valid files because it can't correctly fix the data up.

Regarding the extensions themselves: they could be supported in the future. I generally prefer to only support extensions once they go outside of a draft status (which both physics related extensions are in); also, for read/write support the extensions need to be first implemented in https://github.com/jkuhlmann/cgltf which gltfpack uses for parsing.

[Shinmera]

I understand that, though at least I also see a bunch of extensions that just add metadata without referring to other nodes or indexing that would thrash them. As such, it would be great to be able to specify a list of extension names that will be preserved, because you know them to be good under that constraint.

Gotcha on the cgltf dependency and the ratification process, I figured that that would be required. I'll ping this again when the ratification has completed.

[zeux]

Previously I found very few extensions that can be preserved as is without any custom logic. I'm sure they exist; some KHR material extensions that just add a float or two work like this, but once textures are added custom processing is required. Similarly, any extensions that add extra binary data require using accessors which has the same index distortion problem. There's other cases where gltfpack needs to understand the data, when the data is sensitive to scale (e.g. KHR_material_volume needs special treatment for thickness) or has dependencies with appearance (e.g. any extension that affects material transparency may require handling to keep BLEND mode).

gltfpack expects application specific data that is safe to just copy around to be stored using extras (that are preserved on the main scene elements like nodes/materials/animations if requested), but otherwise the majority of extensions require first class support.