【问题标题】:RealSense OpenCV Depth Image Too DarkRealSense OpenCV 深度图像太暗
【发布时间】:2017-02-21 00:25:25
【问题描述】:

伙计们,

我有一个 realsense SR300,但是当我在 opencv 窗口中显示我的深度图像时,它看起来太暗了。我怎样才能解决这个问题?当我运行 realsense 示例时,图像看起来不错,但示例使用 OpenGL。但我的项目需要 OpenCV。这是我的代码:

int main(int argc, char ** argv)
{
  // realsense camera setup
  rs::log_to_console(rs::log_severity::warn);
  // Create a context object. This object owns the handles to all connected realsense devices
  rs::context ctx;
  if (ctx.get_device_count() == 0)
  {
    throw std::runtime_error("No device detected. Is it plugged in?");
  }
  // Access the first available RealSense device
  rs::device * dev = ctx.get_device(0);
  // Configure depth to run at VGA resolution at 30 frames per second
  dev->enable_stream(rs::stream::depth, 640, 480, rs::format::z16, 30);
  rs::intrinsics depth_intrin;
  rs::format depth_format;
  depth_intrin = dev->get_stream_intrinsics(rs::stream::depth);
  depth_format = dev->get_stream_format(rs::stream::depth);
  cv::namedWindow("Send Display Image", CV_WINDOW_AUTOSIZE);

  /* Set callbacks prior to calling start(). */
  auto depth_callback = [depth_intrin, depth_format](rs::frame f)
  {
    cv::Mat image(cv::Size(640, 480), CV_16UC1,
      (void*)f.get_data(), cv::Mat::AUTO_STEP);
    cv::imshow("Send Display Image", image);
    cv::waitKey(1000/80);
  };
  /* callback to grab depth fream and publish it. */
  dev->set_frame_callback(rs::stream::depth, depth_callback);
  // Start streaming
  dev->start();
  While(1)
  {

  }
  return 0;
}

我不知道为什么我的图像这么暗。当我从 ROS 运行 openni_launch 时,我希望它看起来像 kinect 或 Xtion

【问题讨论】:

    标签: c++ opencv opengl depth realsense


    【解决方案1】:

    编辑:

    下面的归一化函数会产生一些闪烁:

    • 我怀疑这是由于最大深度值闪烁。
    • 最小深度值始终为0,因为当深度无效并因此深度范围变为假时使用此值。

    你应该使用这个:

    void make_depth_histogram(const Mat &depth, Mat &normalized_depth) {
      normalized_depth = Mat(depth.size(), CV_8U);
      int width = depth.cols, height = depth.rows;
    
      static uint32_t histogram[0x10000];
      memset(histogram, 0, sizeof(histogram));
    
      for(int i = 0; i < height; ++i) {
        for (int j = 0; j < width; ++j) {
          ++histogram[depth.at<ushort>(i,j)];
        }
      }
    
      for(int i = 2; i < 0x10000; ++i) histogram[i] += histogram[i-1]; // Build a cumulative histogram for the indices in [1,0xFFFF]
    
      for(int i = 0; i < height; ++i) {
        for (int j = 0; j < width; ++j) {
          if (uint16_t d = depth.at<ushort>(i,j)) {
            int f = histogram[d] * 255 / histogram[0xFFFF]; // 0-255 based on histogram location
            normalized_depth.at<uchar>(i,j) = static_cast<uchar>(f);
          } else {
            normalized_depth.at<uchar>(i,j) = 0;
          }
        }
      }
    }
    

    您观察到的是因为深度流被编码为 16 位 (rs::stream::z16),而显示时仅使用 8 位。

    你可以标准化你的深度图:

    double min, max;
    minMaxLoc(depth, &min, &max);
    Mat depth_normalized;
    double alpha = 255.0/(max-min);
    depth.convertTo(depth_normalized, CV_8U, alpha, -min*alpha);
    

    或者使用一种颜色图来显示深度:make_depth_histogram()。

    完整的演示代码:

    inline void make_depth_histogram(const Mat &depth, Mat &color_depth) {
      color_depth = Mat(depth.size(), CV_8UC3);
      int width = depth.cols, height = depth.rows;
    
      static uint32_t histogram[0x10000];
      memset(histogram, 0, sizeof(histogram));
    
      for(int i = 0; i < height; ++i) {
        for (int j = 0; j < width; ++j) {
          ++histogram[depth.at<ushort>(i,j)];
        }
      }
    
      for(int i = 2; i < 0x10000; ++i) histogram[i] += histogram[i-1]; // Build a cumulative histogram for the indices in [1,0xFFFF]
    
      for(int i = 0; i < height; ++i) {
        for (int j = 0; j < width; ++j) {
          if (uint16_t d = depth.at<ushort>(i,j)) {
            int f = histogram[d] * 255 / histogram[0xFFFF]; // 0-255 based on histogram location
            color_depth.at<Vec3b>(i,j) = Vec3b(f, 0, 255 - f);
          } else {
            color_depth.at<Vec3b>(i,j) = Vec3b(0, 5, 20);
          }
        }
      }
    }
    
    int main(int argc, char *argv[]) {
        // Create a context object. This object owns the handles to all connected realsense devices
        rs::context ctx;
    
        // Access the first available RealSense device
        rs::device * dev = ctx.get_device(0);
    
        // Configure Infrared stream to run at VGA resolution at 30 frames per second
        dev->enable_stream(rs::stream::depth, 640, 480, rs::format::z16, 30);
    
        // Start streaming
        dev->start();
    
        // Camera warmup - Dropped several first frames to let auto-exposure stabilize
        for(int i = 0; i < 30; i++)
           dev->wait_for_frames();
    
        // Creating OpenCV Matrix from a color image
        Mat depth(Size(640, 480), CV_16U, (void*)dev->get_frame_data(rs::stream::depth), Mat::AUTO_STEP);
    
        // Create a color depth
        Mat color_depth;
        make_depth_histogram(depth, color_depth);
    
        // Create a normalized depth
        double min, max;
        minMaxLoc(depth, &min, &max);
        Mat depth_normalized;
        double alpha = 255.0/(max-min);
        depth.convertTo(depth_normalized, CV_8U, alpha, -min*alpha);
    
        // Display in a GUI
        imshow("Display normalized depth", depth_normalized);
        imshow("Display color depth", color_depth);
    
        waitKey(0);
    
        return 0;
      }
    

    【讨论】:

    • 当我执行“标准化深度”步骤时,我的新 8 位图像出现脉动,这意味着图像中的所有像素变得非常亮,然后非常暗(所有像素的数量相同)。这有点奇怪。真正变亮大约需要一秒钟,然后再变暗,然后重复。你知道那可能是什么吗?可能是一些自动曝光问题或相机的问题?
    • 当我将最大值和最小值设置为常数时,闪烁消失了。看静态物体时没有闪烁。当事物移动时,就会发生闪烁。这是有道理的,因为至少 alpha 总是在变化。
    • 您应该使用make_depth_histogram() 来可视化深度图。这有点费时,但可视化应该只用于调试目的。
    【解决方案2】:

    我发现这个问题的唯一解决方案是:

    • 将图像保存为 PNG 文件。 (PNG支持保存16位图片)

    • 使用matplotlib 在彩色地图中查看:

       #!/usr/bin/python3
       import numpy as np
       import cv2
       import sys
       from matplotlib import pyplot as plt
      
       def printCoordinates(event):
           x,y = event.xdata,event.ydata
           if x != None:
               print("X : ",x," Y: ",y," Value = ",img[np.int(y),np.int(x)])
      
       img = cv2.imread(sys.argv[1],cv2.CV_16UC1)
       #img = img/65535
      
       fig = plt.figure()
       plt.imshow(img,cmap='nipy_spectral')
       cid = fig.canvas.mpl_connect('button_press_event',printCoordinates)
       plt.colorbar()
       plt.show()
      

    button_press_event 是在点击的像素上打印准确的像素值。

    RGB 图像:

    对应的深度图像:

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2019-08-12
      • 1970-01-01
      • 1970-01-01
      • 2021-09-02
      • 2011-10-17
      • 2013-07-21
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多