Morph Target colors aren't additive but replacing (?)

Feature Request
1

This is a mix of question/bug/feature request in reference to

and
Validator fails for COLOR_0 in MorphTargets · Issue #178 · KhronosGroup/glTF-Validator · GitHub.

I went ahead and updated a glTF exported from the current version with COLOR_0 instead of _COLOR_0, fixed a couple attribute issues in three.js, and can now load color blendshapes in three:
ezgif.com-gif-maker (6)

However, it seems that (in line with what you explained before regarding the alpha channel / absolute vs. relative colors) colors exported from Nomad are supposed to be used with absolute values, that is, color blend shapes replace the original color, by a factor of COLOR_0.a.

I think for better compatibility to the spec (relative application of morph targets), it would be interesting to have a way to store colors more spec-conform on export, so that additively blending color values ends up being the correct layer result (e.g. a layer might have to store negative values to get the right result). I understand that is not always desired (especially not when someone has control over the shader being used and wants to mix these differently).

My test case here was that the color layers can be additively used, that is, the colors of the chamaeleon would nicely blend into each other. Happy to provide a video of how it’s supposed to look, and open for guidance :slight_smile: I fully agree that this doesn’t seem to be thought through in the spec though.

Would love to learn more about how this is supposed to work - I think the main reason vertex color morph targets aren’t used is that nobody knows this is possible :slight_smile: it’s a pretty cool workflow with Nomad.

1 Like
2 
_COLOR_0 VEC3 (RGB=Color)
_COLOR_1 VEC3 (R=Roughness, G=Metalness, B=SharedAlpha)

In the current version, the layer alpha is in the _COLOR_1 channel.
You need to take it and then using the standard blend operator.
In pseudo code:

vec3 blend(vec3 dst, vec3 src, float srcA) {
    return src * srcA + dst * (1 - srcA);
}

vec3 color = baseColor;
color = blend(color, layer0.color, layer0.alpha * layer0.weight);
color = blend(color, layer1.color, layer1.alpha * layer1.weight);
...

Nomad uses gltf for the internal project, so I need the glb export to fully support Nomad’s features.

I thought about using relative values but not for compatibility but rather size optimisation.
With relatives values, I can use a sparse accessor if the layer is partially painted (and since you can use ton of layers in Nomad, it’s a very good candidate).

However with signed difference I’d need to use 16 bit instead of 8 bit, so if the sparse version is bigger than my absolute, then the file will be much bigger if I want to use 16 bit in my absolute accessor.

I was thinking on using COLOR_0 16 bit for signed difference (if sparse smaller) and using _COLOR_0 8 bit for absolute.

I could add a checkbox force relative layer color but I’m not sure yet if it’s worth it.

To be honest I’m constantly struggling with glTF because it’s a Transmission format instead of being an exchange format (no quad, very annoying padding alignement).

1 Like
3

Thanks for the details!

For my understanding, currently “Export glTF” then just dumps the existing internal glTF representation into a new file?

At least from a user perspective the expectation would kind of be that while “Save” and “Save As” use the internal representation, on “Export glTF” the goal is to have wider compatiblity to external tools and solutions, so that wouldn’t need to be 1:1 the same as the internal file (e.g. the export could always use sparse accessors and relative mode).

4

TLDR; the only issue is memory, I need to ensure the export are using low amount of memory (glb, obj, stl).
I don’t know yet what I’ll do but there’ll be a checkbox in case I don’t export relative morph by default.

Internal save vs External save

Nomad internal representation has nothing to do with glTF. But I need glTF export to fully supports Nomad’s stuffs (at least default settings).
People use glb export as a way to store their file somewhere, and then reimport it in Nomad.
Save/SaveAs only saves internally, you cannot choose where to export.

I can have differences between internal and external glb, but if I do so I use the extras glTF key to make sure things can be read back in Nomad.
Color Morph is different in that I decided to use the glTF accessors, but I could have dumped them in extras as well.

In case it’s not clear, I already use sparse accessor in glTF and I couldn’t care less about how things are represented in glTF (I already adjust my base position and most paint channel for example,).

Memory

You could argue I need an export internal button, but right now the default glb checkboxes plays that role.

Note that I don’t loose data when using relative Color Morph in glTF, my only issue is the potential higher memory usage.

I can guarantee 100% that I’ll receive “bug report” if either the internal or external save use more memory, because some users use tons of layers (+40), I’ll get stuff like:

Hey, I cannot save/export my project anymore :cry:, please help

That’s why export Normal is off by default because it requires tons of memory (the fact that glTF doesn’t support 16bit normal is a bummer as well).

5

I understand the part about memory, for sure. Not quite sure I get that quote right though - isn’t part of the idea that the files created in Nomad can be used elsewhere?

You might still like this, got most things working in three.js with relatively minor changes:

Still have to think about how to integrate it properly (so that others can use that as well), but it’s a start.

6

A file format shouldn’t impact how an application represents its data internally at runtime, the conversion is done during the export or import.

There’ll be a checkbox next release.
I don’t know yet what is going to be the default value.

1 Like
7

This I fully agree with, sorry if my comments came over otherwise :slight_smile:
Looking forward to the new release!