We have made a fix (or workaround) for this issue; it will be released in the coming update. The changes are described bellow in case you need it in an older release. (There are some differences in the plugin code between versions, so it may look different.)
1. Replace the whole imageProjection calculation code in CameraImageRenderer.cs,
it may look like
bool cameraFront = ...
var imageProjection = cameraParameters.imageProjection(...
if (renderImageHFlip)
{
...
imageProjection = ...;
}
material.SetMatrix("_TextureRotation", imageProjection);
replace it with
var imageProjection = ImageProjection(cameraParameters, controller.TargetCamera.aspect, arSession.Assembly.Display.Rotation, renderImageHFlip);
material.SetMatrix("_TextureRotation", imageProjection);
ImageProjection method is from internal CameraParameters.imageProjection implementation, with modifications for UV conversion to replace original vertex conversion.
private static Matrix4x4 ImageProjection(CameraParameters parameters, float viewportAspectRatio, int screenRotation, bool manualHorizontalFlip)
{
var size = parameters.size();
float imageAspectRatio = (float)size.data_0 / size.data_1;
var imageRotationDegree = parameters.imageOrientation(screenRotation);
var imageRotation = imageRotationDegree * Mathf.Deg2Rad;
var requiredImageAspectRatio = (imageRotationDegree % 180) == 0 ? viewportAspectRatio : 1.0f / viewportAspectRatio;
Matrix4x4 scale = Matrix4x4.identity;
if (imageAspectRatio < requiredImageAspectRatio)
{
scale.m11 = imageAspectRatio / requiredImageAspectRatio;
}
else
{
scale.m00 = requiredImageAspectRatio / imageAspectRatio;
}
Matrix4x4 rotation = Matrix4x4.identity;
rotation.m00 = Mathf.Cos(-imageRotation);
rotation.m01 = -Mathf.Sin(-imageRotation);
rotation.m10 = Mathf.Sin(-imageRotation);
rotation.m11 = Mathf.Cos(-imageRotation);
var flip = Matrix4x4.identity;
if (manualHorizontalFlip)
{
flip.m00 = -1;
}
var imageProjection = scale * rotation * flip;
return imageProjection;
}
2. Replace one line in all CameraImage_* shaders
replace
o.vertex = mul(_TextureRotation, o.vertex);
with
o.uv = MultiplyUV(_TextureRotation, o.uv - float2(0.5, 0.5)) + float2(0.5, 0.5);
There are some other partial workarounds in previous released samples, when you use the above solution, those workarounds should be removed. Search for metal in the whole project including shaders and scripts and remove those code paths.