Getting streams from Kinect using OpenNI2

OpenNI2 comes with improvement. There’s no need to self build openCV to get it work with OpenNI. 

Step by step to get Kinect data in a program is: ( those boiler plate codes are mostly adapted from that of FORTH HandTrackingLib provided sample )

1. Initialize OpenNI : 
   OpenNI::initialize();
2. Call a device  and open it :Device device;
   device.open(openni::ANY_DEVICE);
3. Align the depth and color images so they have the same vantage points. Note that this generates a shift so the further object is to Kinect, the bigger the shift is, therefore at some gaps value may not exist. (I got this explanation from OpenNI documentation, though I tried but failed =( )
   device.setImageRegistrationMode(openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR );
4. Define streams :
   VideoStream depth, color;
5. Check if sensor is OK : (similar to color)
   if(device.getSensorInfo(SENSOR_DEPTH) != NULL)
6. If OK, let the streams start, also check if it starts OK: if(depth.start() != STATUS_OK)
7. Now they start streaming, define frames and read data from stream to frames :VideoFrameRef depthFrame, colorFrame;depth.readFrame(&depthFrame);
   color.readFrame(&colorFrame);
8. Check if them are valid:
   if(depthFrame.isValid() && colorFrame.isValid())
9. If valid, convert to opencv::Mat and show:cv::Mat depth(480,640,CV_16UC1,(void*)depthFrame.getData(), 2*640);
   cv::Mat depthNorm;
   cv::normalize(depth, depthNorm, 0, 255, CV_MINMAX, CV_8UC1);
   imshow(“Depth”, depthNorm);
   cv::Mat bgrMat,rgbMat(480,640,CV_8UC3,(void*)colorFrame.getData(),3*640);
   cv::cvtColor(rgbMat,bgrMat, CV_RGB2BGR);// opencv expects the image in BGR format
   imshow(“Color”, bgrMat);
10. Remember to stop streams –> destroy streams –> close device –> shutdown openNIdepth.stop();
    depth.destroy();
    color.stop();
    color.destroy();
    device.close();
    OpenNI::shutdown();

However, in order for the program to work, add those in Property Pages of the project:

1. include folders of OpenNI2, OpenCV ( I use v2.4.3) under C/C++ –> General –> Additional Include Directories
   
2. lib folders of OpenNI2, OpenCV under Linker–>General–>Additional Library Directories
3. All libraries that project uses, e.g.:
   opencv_core243d.lib
   opencv_highgui243d.lib
   opencv_imgproc243d.lib (or any other opencv libraries that you want to use)
   OpenNI2.lib (mandatory)
4. Very important, copy all files in folder Redist of OpenNI , for example C:\Program Files\OpenNI2\Redist for 64bit, C:\Program Files (x86)\OpenNI2\Redist for 32 bit, into working directory of the project. [Working directory == place that contains files :  .vcxproj .vcxproj.filters .vcxproj.user ]

Evaluation:

• Easier to get data than with KinectSDK
• Have a lot of middlewares available to use
• Depth to Color aligment doesn’t work very well
• But KinectSDK has better skeletons
• new KinectSDK 1.7 now also provides hand detection, so everything in a package
• new KinectSDk 1.7 has KinectFusion, pretty cool. I’m wondering what I can do with it.

Thanks FORTH team!

About these ads