Pitting/roughness after voxel merging closely matched surfaces

I’ve noticed that if I try to voxel merge objects whose surfaces blend closely with each other, I get bad surface pitting/roughness. Is there a fix for this, can I change settings to improve results. Here is an example:
Here are two separate objects which intersect (scale has been baked)

I now carefully flatten part of one of the objects so that it’s surface blends almost perfectly with the other

To make it clearer to understand, in the next image I have painted green the area where the surfaces of the two objects are almost perfectly matched

But if I now voxel merge the objects (at 350 resolution in this case), the area where the surfaces are closely matched becomes pitted/rough:

I’ve noticed this always seems to happen when 2 surfaces are very close together and I voxel merge.
Is this a bug, or is it unavoidable? Any suggestions appreciated.

Does it happen if you merge the layers and bake the matrix for both object?
(Gizmo menu → bake)

If so, it’s probably unavoidable, you try « keep sharp edges », it usually gets better result for simple shapes.

Yes it does still seem to happen. In this particular example there were no layers. All scalings and transformations were baked. It always seems to happen when the surfaces of different objects are very close together; it’s almost as if the algorithm can’t quite decide which surface to use to create the new remeshed version (although I obviously don’t really understand any of the maths that’s being used!)
This screenshot shows how you can see both wireframes at once in the problem area before remeshing

Remeshing at a very high resolution minimises the issue, but it is still present. A reasonable workaround is to remesh at very high resolution, and then remesh again at a lower resolution.