【问题标题】:kinect/ processing / simple openni - point cloud data not being output properlykinect/处理/简单的openni-点云数据没有正确输出
【发布时间】:2012-07-19 04:37:48
【问题描述】:

我创建了一个处理草图,它将点云数据的每一帧从 kinect 保存到一个文本文件中,其中文件的每一行都是 kinect 注册的一个点(或顶点)。我计划将数据提取到 3d 程序中,以在 3d 空间中可视化动画并应用各种效果。问题是,当我这样做时,第一帧似乎是正确的,而其余的帧似乎吐出了看起来像第一张图像的东西,加上一堆随机噪声。这是我的完整代码。它需要简单的openni才能正常工作。你可以看到cmets

import SimpleOpenNI.*;
//import processing.opengl.*;

SimpleOpenNI context;
float        zoomF =0.5f;
float        rotX = radians(180);  // by default rotate the hole scene 180deg around the x-axis, 
float        rotY = radians(0); // the data from openni comes upside down

int maxZ = 2000;
Vector <Object> recording = new Vector<Object>(); 
boolean isRecording = false;
boolean canDraw = true;
boolean mouseMode = false;
int currentFile = 0;
int depthWidth = 640; //MH - assuming this is static?
int depthHeight = 480;
int steps = 5;
int arrayLength = (depthWidth/steps) * (depthHeight/steps); //total lines in each output file


void setup()
{
  size(1024,768,P3D);  // strange, get drawing error in the cameraFrustum if i use P3D, in opengl there is no problem
  //size(1024,768,OPENGL); 

  context = new SimpleOpenNI(this);
  context.setMirror(true);
  depthWidth = context.depthWidth();
  depthHeight = context.depthHeight();

  // enable depthMap generation 
  if(context.enableDepth() == false)
  {
     println("Can't open the depthMap, maybe the camera is not connected!"); 
     exit();
     return;
  }

  stroke(255,255,255);
  smooth();

  perspective(radians(45),
  float(width)/float(height),
  10.0f,150000.0f);
 }

void draw()
{

  //println(isRecording);

  // update the cam
  context.update();

  background(0,0,0);

  // set the scene pos
  translate(width/2, height/2, 0);
  rotateX(rotX);
  rotateY(rotY);
  scale(zoomF);

  // draw the 3d point depth map
  int[]   depthMap = context.depthMap();
  int     index = 0;
  PVector realWorldPoint;
  PVector[] frame = new PVector[arrayLength];

  translate(0,0,-1000);  // set the rotation center of the scene 1000 infront of the camera
  stroke(200); 
  for(int y=0;y < context.depthHeight();y+=steps)
  {
    for(int x=0;x < context.depthWidth();x+=steps)
    {
      int offset = x + y * context.depthWidth();
      realWorldPoint = context.depthMapRealWorld()[offset];
      if (isRecording == true){
        if (realWorldPoint.z < maxZ){
          frame[index] = realWorldPoint;
        } else {
          frame[index] = new PVector(-0.0,-0.0,0.0); 
        }
        index++;
      } else {
        if (realWorldPoint.z < maxZ){
          if (canDraw == true){
            point(realWorldPoint.x,realWorldPoint.y,realWorldPoint.z);
          }
        }
      }
    } 
  }

  if (isRecording == true){
   recording.add(frame); 
  }

 if (mouseMode == true){
   float rotVal = map (mouseX,0,1024,-1,1); //comment these out to disable mouse orientation
   float rotValX = map (mouseY,0,768,2,4);
   rotY = rotVal;
   rotX = rotValX;
 } 

}

// -----------------------------------------------------------------
// Keyboard event
void keyPressed()
{
  switch(key)
  {
    case ' ':
      context.setMirror(!context.mirror());
      break;
    case 'm':
      mouseMode = !mouseMode;
      break;
    case 'r':
      isRecording = !isRecording;
      break;
    case 's':
      if (isRecording == true){
        isRecording = false;
        canDraw = false;
        println("Stopped Recording");
        Enumeration e = recording.elements();
        int i = 0;
        while (e.hasMoreElements()) {

          // Create one directory
          boolean success = (new File("out"+currentFile)).mkdir(); 
          PrintWriter output = createWriter("out"+currentFile+"/frame" + i++ +".txt");
          PVector [] frame = (PVector []) e.nextElement();

          for (int j = 0; j < frame.length; j++) {
           output.println(j + ", " + frame[j].x + ", " + frame[j].y + ", " + frame[j].z );
          }
          output.flush(); // Write the remaining data
          output.close();
          //exit();
        }
        canDraw = true;
        println("done recording");
      }
      currentFile++;
      break;
  }

  switch(keyCode)
  {
    case LEFT:
      if(keyEvent.isShiftDown())
        maxZ -= 100;
      else
        rotY += 0.1f;
      break;
    case RIGHT:
      if(keyEvent.isShiftDown())
        maxZ += 100;
      else
        rotY -= 0.1f;
      break;
    case UP:
      if(keyEvent.isShiftDown())
        zoomF += 0.01f;
      else
        rotX += 0.1f;
      break;
    case DOWN:
      if(keyEvent.isShiftDown())
      {
        zoomF -= 0.01f;
        if(zoomF < 0.01)
          zoomF = 0.01;
      }
      else
        rotX -= 0.1f;
      break;
  }
}

我想循环是问题开始发生的地方:for(int y=0;y

【问题讨论】:

    标签: processing kinect openni point-clouds


    【解决方案1】:

    很遗憾,我现在无法解释很多,但几个月前我已经将类似的东西保存到 PLY 和 CSV:

    import processing.opengl.*;
    import SimpleOpenNI.*;
    
    
    SimpleOpenNI context;
    float        zoomF =0.5f;
    float        rotX = radians(180);  
    float        rotY = radians(0);
    
    boolean recording = false;
    ArrayList<PVector> pts = new ArrayList<PVector>();//points for one frame
    
    float minZ = 100,maxZ = 150;
    
    void setup()
    {
      size(1024,768,OPENGL);  
    
      context = new SimpleOpenNI(this);
      context.setMirror(false);
      context.enableDepth();
      context.enableScene();
    
      stroke(255);
      smooth();  
      perspective(95,float(width)/float(height), 10,150000);
     }
    
    void draw()
    {
      context.update();
      background(0);
    
      translate(width/2, height/2, 0);
      rotateX(rotX);
      rotateY(rotY);
      scale(zoomF);
    
      int[]   depthMap = context.depthMap();
      int[]   sceneMap = context.sceneMap();
      int     steps   = 10;  
      int     index;
      PVector realWorldPoint;
      pts.clear();//reset points
      translate(0,0,-1000);  
      //*
      //stroke(100); 
      for(int y=0;y < context.depthHeight();y+=steps)
      {
        for(int x=0;x < context.depthWidth();x+=steps)
        {
          index = x + y * context.depthWidth();
          if(depthMap[index] > 0)
          { 
            realWorldPoint = context.depthMapRealWorld()[index];
            if(realWorldPoint.z > minZ && realWorldPoint.z < maxZ){//if within range
              stroke(0,255,0);
              point(realWorldPoint.x,realWorldPoint.y,realWorldPoint.z);
              pts.add(realWorldPoint.get());//store each point
            }
          }
        } 
      } 
      if(recording){
          savePLY(pts);//save to disk as PLY
          saveCSV(pts);//save to disk as CSV
      }
      //*/
    }
    
    // -----------------------------------------------------------------
    // Keyboard events
    
    void keyPressed()
    {
      if(key == 'q') minZ += 10;
      if(key == 'w') minZ -= 10;
      if(key == 'a') maxZ += 10;
      if(key == 's') maxZ -= 10;
    
      switch(key)
      {
        case ' ':
          context.setMirror(!context.mirror());
        break;
        case 'r':
          recording = !recording;
        break;
      }
    
      switch(keyCode)
      {
        case LEFT:
          rotY += 0.1f;
          break;
        case RIGHT:
          // zoom out
          rotY -= 0.1f;
          break;
        case UP:
          if(keyEvent.isShiftDown())
            zoomF += 0.01f;
          else
            rotX += 0.1f;
          break;
        case DOWN:
          if(keyEvent.isShiftDown())
          {
            zoomF -= 0.01f;
            if(zoomF < 0.01)
              zoomF = 0.01;
          }
          else
            rotX -= 0.1f;
          break;
      }
    }
    void savePLY(ArrayList<PVector> pts){
      String ply = "ply\n";
      ply += "format ascii 1.0\n";
      ply += "element vertex " + pts.size() + "\n";
      ply += "property float x\n";
      ply += "property float y\n";
      ply += "property float z\n";
      ply += "end_header\n";
      for(PVector p : pts)ply += p.x + " " + p.y + " " + p.z + "\n";
      saveStrings("frame_"+frameCount+".ply",ply.split("\n"));
    }
    void saveCSV(ArrayList<PVector> pts){
      String csv = "x,y,z\n";
      for(PVector p : pts) csv += p.x + "," + p.y + "," + p.z + "\n";
      saveStrings("frame_"+frameCount+".csv",csv.split("\n"));
    }
    

    我使用 if 语句仅保存某个 Z 阈值内的点,但可以随意更改/使用您认为合适的。 后处理的想法让人想起Moullinex video for Catalina。检查一下,它有很好的文档记录,并且还包含源代码。

    更新 发布的代码每帧保存 1 个文件。即使播放速度很低,草图仍应为每一帧保存一个文件。代码简化一点:

    import processing.opengl.*;
    import SimpleOpenNI.*;
    
    
    SimpleOpenNI context;
    float        zoomF =0.5f;
    float        rotX = radians(180);  
    float        rotY = radians(0);
    
    boolean recording = false;
    String csv;
    
    void setup()
    {
      size(1024,768,OPENGL);  
    
      context = new SimpleOpenNI(this);
      context.setMirror(false);
      context.enableDepth();
    
      stroke(255);
      smooth();  
      perspective(95,float(width)/float(height), 10,150000);
     }
    
    void draw()
    {
      csv = "x,y,z\n";//reset csv for this frame
      context.update();
      background(0);
    
      translate(width/2, height/2, 0);
      rotateX(rotX);
      rotateY(rotY);
      scale(zoomF);
    
      int[]   depthMap = context.depthMap();
      int[]   sceneMap = context.sceneMap();
      int     steps   = 10;  
      int     index;
      PVector realWorldPoint;
      translate(0,0,-1000);  
      //*
      beginShape(POINTS);
      for(int y=0;y < context.depthHeight();y+=steps)
      {
        for(int x=0;x < context.depthWidth();x+=steps)
        {
          index = x + y * context.depthWidth();
          if(depthMap[index] > 0)
          { 
            realWorldPoint = context.depthMapRealWorld()[index];
            vertex(realWorldPoint.x,realWorldPoint.y,realWorldPoint.z);
            if(recording) csv += realWorldPoint.x + "," + realWorldPoint.y + "," + realWorldPoint.z + "\n";
          }
        } 
      }
      endShape(); 
      if(recording) saveStrings("frame_"+frameCount+".csv",csv.split("\n"));
      frame.setTitle((int)frameRate + " fps");
      //*/
    }
    
    // -----------------------------------------------------------------
    // Keyboard events
    
    void keyPressed()
    {
    
      switch(key)
      {
        case ' ':
          context.setMirror(!context.mirror());
        break;
        case 'r':
          recording = !recording;
        break;
      }
    
      switch(keyCode)
      {
        case LEFT:
          rotY += 0.1f;
          break;
        case RIGHT:
          // zoom out
          rotY -= 0.1f;
          break;
        case UP:
          if(keyEvent.isShiftDown())
            zoomF += 0.01f;
          else
            rotX += 0.1f;
          break;
        case DOWN:
          if(keyEvent.isShiftDown())
          {
            zoomF -= 0.01f;
            if(zoomF < 0.01)
              zoomF = 0.01;
          }
          else
            rotX -= 0.1f;
          break;
      }
    }
    

    预览可以通过不同的循环与录制分开,您可以进行低分辨率预览,但保存更多数据,仍然会很慢。

    我有另一个建议:改为记录到.oni format。如果您已经安装了 OpenNI,您可以使用几个示例,例如 NiViewerNiBackRecorder。 SimpleOpenNI 也公开了此功能,请查看 RecorderPlay 示例。

    我建议尝试这样的事情:

    1. 将场景录制到 .oni 文件中。它应该快速/响应迅速
    2. 当您对 .oni 录制感到满意时,处理每一帧(将深度转换为 x、y、z 点/根据需要进行过滤/保存为所需的格式/等)

    这里有另一个草图来说明这个想法:

    import SimpleOpenNI.*;
    
    SimpleOpenNI  context;
    boolean       recordFlag = true;
    
    int frames = 0;
    
    void setup(){
      context = new SimpleOpenNI(this);
    
      if(! recordFlag){
        if(! context.openFileRecording("test.oni") ){
          println("can't find recording !!!!");
          exit();
        }
        context.enableDepth();
      }else{  
        // recording
        context.enableDepth();
        // setup the recording 
        context.enableRecorder(SimpleOpenNI.RECORD_MEDIUM_FILE,"test.oni");
        // select the recording channels
        context.addNodeToRecording(SimpleOpenNI.NODE_DEPTH,SimpleOpenNI.CODEC_16Z_EMB_TABLES);
      }
      // set window size 
      if((context.nodes() & SimpleOpenNI.NODE_DEPTH) != 0)
        size(context.depthWidth() , context.depthHeight());
      else 
        exit();
    }
    void draw()
    {
      background(0);
      context.update();
      if((context.nodes() & SimpleOpenNI.NODE_DEPTH) != 0) image(context.depthImage(),0,0);
      if(recordFlag) frames++;
    }
    void keyPressed(){
      if(key == ' '){
        if(recordFlag){
          saveStrings(dataPath("frames.txt"),split(frames+" ",' '));
          exit();
        }else saveONIToPLY();
      }
    }
    void saveONIToPLY(){
      frames = int(loadStrings(dataPath("frames.txt"))[0]);
      println("recording " + frames + " frames");
      int w = context.depthWidth();
      int h = context.depthHeight();
      noLoop();
      for(int i = 0 ; i < frames; i++){
        PrintWriter output = createWriter(dataPath("frame_"+i+".ply"));
        output.println("ply");
        output.println("format ascii 1.0");
        output.println("element vertex " + (w*h));
        output.println("property float x");
        output.println("property float y");
        output.println("property float z");
        output.println("end_header\n");
        context.update();
        int[]   depthMap = context.depthMap();
        int     index;
        PVector realWorldPoint;
        for(int y=0;y < h;y++){
          for(int x=0;x < w;x++){
            index = x + y * w;
            realWorldPoint = context.depthMapRealWorld()[index];
            output.println(realWorldPoint.x + " " + realWorldPoint.y + " " + realWorldPoint.z);
          }
        }
        output.flush();
        output.close();
        println("saved " + (i+1) + " of " + frames);
      }
      loop();
      println("recorded " + frames + " frames");
    }
    

    recordFlag 设置为 true 时,数据将保存到 .oni 文件中。 我没有在文档中找到任何内容来读取 .oni 文件中有多少帧,因此我添加了frame 计数器作为快速解决方法。如果点击空格,录制将停止,但也会将帧数保存在 txt 文件中,然后退出应用程序。这将在以后有用。

    recordFlag设置为false时,如果已经有录音,则会播放。 如果您在此“模式”中点击空格,绘图将停止,帧号将从 .txt 文件中加载,并且对于每一帧:

    1. 上下文将被更新(移动到下一帧)
    2. 深度图中的每个像素都将转换为一个点
    3. 所有点都将被写入 .ply 文件(您可以使用meshlab 处理)

    保存所有帧后,草图将继续绘制。由于没有 3D 绘图且草图相当简单,因此性能应该会更好,但请记住,大型 .oni 文件将需要大量 RAM。随意修改草图以满足您的需要(例如过滤掉您不想保存的信息等)。

    还要注意上面的内容,虽然应该将每个单独的帧保存到 PLY 中,但它保存的是相同的。调用 noLoop() 时,上下文似乎没有 update() 。这是一个使用 3s 的修改后的 hacky 版本。延迟(希望到那时 .ply 填充将被写入磁盘)。

    import SimpleOpenNI.*;
    
    SimpleOpenNI  context;
    boolean       recordFlag = false;
    boolean       saving = false;
    int frames = 0;
    int savedFrames = 0;
    
    void setup(){
      context = new SimpleOpenNI(this);
    
      if(! recordFlag){
        if(! context.openFileRecording("test.oni") ){
          println("can't find recording !!!!");
          exit();
        }
        context.enableDepth();
      }else{  
        // recording
        context.enableDepth();
        // setup the recording 
        context.enableRecorder(SimpleOpenNI.RECORD_MEDIUM_FILE,"test.oni");
        // select the recording channels
        context.addNodeToRecording(SimpleOpenNI.NODE_DEPTH,SimpleOpenNI.CODEC_16Z_EMB_TABLES);
      }
      // set window size 
      if((context.nodes() & SimpleOpenNI.NODE_DEPTH) != 0)
        size(context.depthWidth() , context.depthHeight());
      else 
        exit();
    }
    void draw()
    {
      background(0);
      context.update();
      if((context.nodes() & SimpleOpenNI.NODE_DEPTH) != 0) image(context.depthImage(),0,0);
      if(recordFlag) frames++;
      if(saving && savedFrames < frames){
          delay(3000);//hack
          int i = savedFrames;
          int w = context.depthWidth();
          int h = context.depthHeight();
          PrintWriter output = createWriter(dataPath("frame_"+i+".ply"));
          output.println("ply");
          output.println("format ascii 1.0");
          output.println("element vertex " + (w*h));
          output.println("property float x");
          output.println("property float y");
          output.println("property float z");
          output.println("end_header\n");
          rect(random(width),random(height),100,100);
          int[]   depthMap = context.depthMap();
          int     index;
          PVector realWorldPoint;
          for(int y=0;y < h;y++){
            for(int x=0;x < w;x++){
              index = x + y * w;
              realWorldPoint = context.depthMapRealWorld()[index];
              output.println(realWorldPoint.x + " " + realWorldPoint.y + " " + realWorldPoint.z);
            }
          }
          output.flush();
          output.close();
          println("saved " + (i+1) + " of " + frames);
          savedFrames++;
      }
    }
    void keyPressed(){
      if(key == ' '){
        if(recordFlag){
          saveStrings(dataPath("frames.txt"),split(frames+" ",' '));
          exit();
        }else saveONIToPLY();
      }
    }
    void saveONIToPLY(){
      frames = int(loadStrings(dataPath("frames.txt"))[0]);
      saving = true;
      println("recording " + frames + " frames");
    }
    

    我不确定帧和文件是否同步以及深度数据是否以中等质量保存,但我希望我的回答能提供一些想法。

    【讨论】:

    • 是的 - 这实际上是我开始我的草图的项目,不幸的是它使用了 Shiffman 库,这对于我想做的事情来说太慢了。我加载了您的示例(谢谢!) - 我没有从 kinect 获得任何视觉效果。这正常吗?
    • 我正在记录一个小物体,所以范围 (minZ,maxZ) 相当小。要么增加范围(w、s、a、d 键控制)或完全删除它(删除这条线if(realWorldPoint.z &gt; minZ &amp;&amp; realWorldPoint.z &lt; maxZ){ 和属于它的}。同样关于速度,分辨率越小预览速度越快(这可能与您保存的点集分开)
    • 啊-明白了-使用您提供的代码我无法弄清楚的最后一件事是,它似乎每秒只能吐出大约 1 个 csv / ply,而不是 30 个。我的系统是不是很慢,或者代码是这样设计的?
    • 每帧应该保存 1 个 csv/ply。帧率仍然受性能影响。在我的机器上,保存时我得到〜4fps。查看更新的答案
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2016-05-09
    • 1970-01-01
    • 2021-08-07
    • 1970-01-01
    • 1970-01-01
    • 2018-01-06
    • 2019-11-05
    相关资源
    最近更新 更多