如何在 iPad 上的 Xamarin iOS ImageView 中显示相机框架？

Question

我的应用从另一台计算机的套接字接收 USB 相机帧流：从 USB 相机读取帧的 Windows 10 UWP 应用，格式为 ARGB32。

我想在 iPad 上显示每一帧，以获得实时视图。

我认为 ImageView 控件是最好的。

帧的来源是 Windows 10 上的 .NET UWP MediaFrameReader，来自 Logitech HD1080p USB 网络摄像头。

有关在 Windows 计算机上创建框架的更多详细信息，然后将其发送到 iPad 套接字...

WINDOWS 10 上的此代码从相机接收帧...

        private void Reader_FrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
    {

        // Update GUI frame count:
        SDKTemplate.Scenario1_DisplayDepthColorIR.present_frame_count++;

        // TryAcquireLatestFrame will return the latest frame that has not yet been acquired.
        // This can return null if there is no such frame, or if the reader is not in the
        // "Started" state. The latter can occur if a FrameArrived event was in flight
        // when the reader was stopped.
        //using (MediaFrameReference frame = sender.TryAcquireLatestFrame())
        using (MediaFrameReference frame = sender.TryAcquireLatestFrame())
        {
                // Display locally:
                _frameRenderer.ProcessFrame(frame);
        }
    }

此代码准备帧从 WINDOWS 发送到 IPAD 套接字...

        public void ProcessFrame( MediaFrameReference frame)
    {
        var softwareBitmap = FrameRenderer.ConvertToDisplayableImage(frame?.VideoMediaFrame);

        if (softwareBitmap != null)
        {
                ++arrived_frames;
            uint frame_size_bytes = frame.Buffer​Media​Frame.Buffer.Length;
            // Swap the processed frame to _backBuffer and trigger UI thread to render it
            softwareBitmap = Interlocked.Exchange(ref _backBuffer, softwareBitmap);

            // UI thread always reset _backBuffer before using it.  Unused bitmap should be disposed.
            softwareBitmap?.Dispose();

            ////////////////////////   DISPLAY FRAME LOCALLY   /////////////////////////

            // Changes to xaml ImageElement must happen in UI thread through Dispatcher
            var task = _imageElement.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
                async () =>
                {
                    // Don't let two copies of this task run at the same time.
                    if (_taskRunning)
                    {
                        return;
                    }
                    _taskRunning = true;

                    // Keep draining frames from the backbuffer until the backbuffer is empty.
                    SoftwareBitmap latestBitmap;
                    while ((latestBitmap = Interlocked.Exchange(ref _backBuffer, null)) != null)
                    {
                        var imageSource = (SoftwareBitmapSource)_imageElement.Source;
                        await imageSource.SetBitmapAsync(latestBitmap);

                        SDKTemplate.SOURCE.Frames_To_iPad.send_frame_packet( latestBitmap, frame_size_bytes );

                        latestBitmap.Dispose();
                    }

                    _taskRunning = false;
                });

            ////////////////////////////  SEND TO IPAD   //////////////////////////////





        }
    }

Answer 1

您可以尝试在每次收到新帧时设置UIImageView 的Image 属性，但我怀疑这是否会发生得足够快以流畅地显示视频，因为您需要将每个帧转换为UIImage，然后才能将其分配给 Image 属性。

我刚刚使用本地高清大小的图像进行了快速测试，它的跟踪似乎相当流畅。这些图像是存储在应用程序包中的 PNG 图像，我不确定当您从套接字接收它们时您的图像将采用什么格式。