How to use Kinect with ABB robots

John Wiberg

Started on a draft for a blog post, so I thought I'd put it here first before fleshing it out.

 

How do you develop an application combining Microsoft Kinect with ABB Robotics IRC5 controllers?

First start by downloading and installing Visual Studio 2010/2012, express edition is free to use.
http://www.microsoft.com/visualstudio/eng/products/visual-studio-express-products
Then install the Kinect SDK & play with that a bit.
http://www.microsoft.com/en-us/kinectforwindows/develop/developer-downloads.aspx
When you have got the kinect bit going then you need to be able to communicate with the robot, so you need to download and install RobotStudio
http://www.abb.com/roboticssoftware
Then head over to the Developer Center and click Robot Communication (which is the ABB Robotics - PC SDK)
http://developercenter.robotstudio.com/
try out the samples and go through the chapter on Accessing the Controller:
http://developercenter.robotstudio.com:80/Index.aspx?DevCenter=RobotCommunication&OpenDocument&Url=../RobotCommunicationAppManual/doc29.html

What you probably want to do is to have a move loop with a couple of targets, then update the pos data in those targets depending on the input of the Kinect user.
http://developercenter.robotstudio.com:80/Index.aspx?DevCenter=RobotCommunication&OpenDocument&Url=../RobotCommunicationAppManual/doc30.html
Please do remember to be careful so test everything on the Virtual Controller in RobotStudio first.

Then when/if  there are questions or bragging rights about cool results then pop over to the user forum and the Developer Center forum.
http://forums.robotstudio.com/forum_topics.asp?FID=9

And remember if you get some fun/good results you should then upload a vid to youtube or something, I'm sure our youtube account would like to feature/link such stuff if you send them a note with link(s) and description(s).
http://www.youtube.com/user/ABBRobotics

stevemcatee

Possibly not quite what you had in mind, but...
For my Masters thesis I've been integrating Robot Studio with Kinect Scanning to perform unknown object detection and collision avoidance in real-time... well 2.2 Hz
Future developments will be faster and obviously better machine the faster it goes.

I've developed this as a plugin for Robto Studio.  
It uses the simulation created in Robot studio to compare to the real scan, so what ever you have designed gets used directly.
My Software  "Collider" compare the simulation to the 3D scan from the Kinect and examines the differences to look for "unknown" or unexpected objects. 
These objects are then compared to the path of the robot from the robot studio design simulation. 
If an object interferes with the path of the robot the robot can be rerouted around the obstruction and the process can continue...

So a few concepts demonstrated:

• 3D scanning with a Microsoft Kinect
• Simulation Scan Comparison 
• Unknown Object Isolation
• Collision detection
• Path planning
  

Have a look here at some results from the Swinburne University Youtube site.

http://youtu.be/4aacUzI8UOg

http://youtu.be/cmzsIx7i2OU

http://youtu.be/cBFWemREMKE

 
If anyone is interested check out some more details at www.splatconcepts.com.au

Steve

MechatronicsStudent

Hi John

I am doing exactly that for my Bachelors Project.

I have a while loop that goes through 3 different targets, all of which are copies of the original and I just update the original. Does this seem wise?

I have got the new pos data from the Kinect, but will I need to calculate the orient and config data for the robtargets too?

I know I can use OrientZYX to find the orient data but I do not know what angles it wants as inputs. Are these the angles between the origin and the new point in space?

as for the config data, i can find no easy way of calculating this. I can only think of ConfLOFF but that may cause some difficulties.

Kind regards

Mkanggg

Hi, I'm currently developing my final year project for my bachelor about how to control the ABB IRB120 robot using the Kinect Xbox 360 and I found out the one that is similar to my project herehttps://www.youtube.com/watch?v=rjeQYLaPbVM. The C# and RAPID code have been provided in the description but I need someone to guide me how to do since I'm not familiar with the programming. Anyone here could help me?