在 Unity 纹理上绘制 ARCore Camera Feed

Question

我正在尝试将 ARCore 相机馈送显示到我在 Unity 中创建的纹理上，但我不知道如何让它正常工作。

我在 Unity 中创建了这样的纹理：

Texture2D unityTexture = new Texture2D(
        480,
        640,
        TextureFormat.RGBA32,
        false);
 unityTexture.filterMode = FilterMode.Point;
 unityTexture.Apply();

然后我使用 Texture2D#GetNativeTexturePtr() 返回的值将我的纹理 ID 传递给 Java。

在我的 Java 中，我尝试按如下方式设置纹理：

  // Called from Unity layer when camera texture is created.
  public void setUnityCameraTexture(long texId, int width, int height) {
    int textureId = (int) textId;
    mSession.setCameraTextureName(textureId);
    mSession.setDisplayGeometry(width, height);
  }

...

  // Called on each Unity MonoBehaviour.Update().
  public void renderFrame() {
    try {
      mSession.update();
    } catch (CameraException e) {
      Log.w(TAG, "Error when updating session", e);
    }
  }

使用这种方法，纹理不会渲染到屏幕上，我收到以下错误日志：

E GLConsumer: [SurfaceTexture-1337-7436-0] bindTextureImage: error binding external image: 0x502
E tango_camera_native: Couldn't get image exposure/skew for timestamp offset!
E ndk_camera: metadata has: 780757.577090
E ndk_camera: metadata has: 780757.890735
E ndk_camera: metadata has: 780757.924221
E ndk_camera: metadata has: 780757.957402
E ndk_camera: metadata has: 780757.990704
E ndk_camera: metadata has: 780758.024007
E ndk_camera: metadata has: 780758.057341
E ndk_camera: metadata has: 780758.090735
E ndk_camera: metadata has: 780758.124007
E ndk_camera: metadata has: 780758.157341
E ndk_camera: metadata has: 780758.190613
E ndk_camera: metadata has: 780758.223977
E ndk_camera: metadata has: 780758.257432
E ndk_camera: metadata has: 780758.290643
E ndk_camera: metadata has: 780758.323946
E ndk_camera: metadata has: 780758.349104
W tango_camera_native_jni: jint Java_com_google_tango_jni_TangoCameraNative_UpdateTextureExternalOes(JNIEnv*, jobject, int, int, jdoubleArray): Got a negative timestamp -0.000000 from camera with id 0.
I Unity   : AndroidJavaException: java.lang.RuntimeException: Error during updateTexImage (see logcat for details)
I Unity   : java.lang.RuntimeException: Error during updateTexImage (see logcat for details)
I Unity   :     at android.graphics.SurfaceTexture.nativeUpdateTexImage(Native Method)
I Unity   :     at android.graphics.SurfaceTexture.updateTexImage(SurfaceTexture.java:244)
I Unity   :     at com.google.tango.jni.TangoCameraNative.UpdateTextureExternalOes(Native Method)
I Unity   :     at com.google.tango.jni.TangoCameraNative.updateTextureExternalOes(TangoCameraNative.java:222)
I Unity   :     at com.google.tango.loader.ITangoCameraNative$Stub.onTransact(ITangoCameraNative.java:162)
I Unity   :     at android.os.Binder.transact(Binder.java:499)
I Unity   :     at com.google.tango.loader.ITangoCameraNative$Stub$Proxy.updateTextureExternalOes(ITangoCameraNative.java:436)
I Unity   :     at com.google.atap.tangoservice.TangoCameraNativeLoader.updateTextureExternalOes(TangoCameraNativeLoader.java:178)
I Unity   :     at com.google.atap.tangoservice.Tango.updateTextureExternalOes(Tango.java:512)
I Unity   :     at com.google.ar.core.TangoWrapper.updateTextureExternalOes(Tan

不确定我的纹理设置是否不正确，但我们将不胜感激！

Answer 1

想通了。我没有正确设置我的纹理。 ARCore documentation 声明纹理必须使用glGenTextures(int, int[], int) 创建并绑定到GL_TEXTURE_EXTERNAL_OES 目标。我现在不是让 Unity 创建纹理，而是在 Java 层创建纹理，将其绑定到 GL_TEXTURE_EXTERNAL_OES，然后将纹理 ID 传递回 Unity。

我的 Java 现在看起来像：

// Called on app resume.
public void resume() {
  mSession.resume(mConfig);
  mSession.setCameraTextureName(mTextureName[0]);
}

// Called on each Unity MonoBehaviour.Update().
public void renderFrame() {
  try {
    mSession.update();
  } catch (CameraException e) {
    Log.w(TAG, "Error when updating session", e);
  }
}

public int getTextureName() {
  mTextureName = new int[1];
  GLES20.glGenTextures(1, mTextureName, 0);
  GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, mTextureName[0]);
  return mTextureName[0];
}

在我的 Unity 脚本中：

// Called on Unity Start()
public Texture2D GetCameraTexture() {
  IntPtr textureId = GetCameraTextureId();
  Texture2D cameraTexture = Texture2D.CreateExternalTexture(
      1080,
      1920,
      TextureFormat.RGBA32,
      false,
      false,
      GetTextureId());
  return cameraTexture;
}

private IntPtr GetCameraTextureId() {
  // Calls into Java getTextureName method.
}

除了这些更改之外，我还必须创建一个自定义 Unity GLSL 着色器，它使用 samplerExternalOES 采样器访问纹理，并使用它在我的一个游戏对象上显示纹理。这也被指定为documentation 中的要求。