The Xbox 360 Kinect unit, which is designed to allow human motion to control on-screen games, has been hacked on several occasions since its debut last year, much to Microsoft's public annoyance. This time round it has been hacked to control a surgical robot.
A group of graduate students at the University of Washington have modified the firmware and software of the Kinect to guide the actions of surgical robots.
The technology is still in its embryonic stages."For robotics-assisted surgeries, the surgeon has no sense of touch right now", Howard Chizeck, the university's professor of electrical engineering told the newswire. The hack doesn't actually allow surgeons to use Kinect for hands free operations. That's surely some years off yet. What the researchers have done is add force feedback so surgeons have an added sense when performing robotics assisted surgery.
And this is where the Kinect enters the frame, as Chizeck notes that the hacked unit adds the sense of touch and feeds information to the surgeon. The team reckons the set up could save $50,000 on the standard rig used to perform non-invasive surgery. That requires an array of very complex remotely controlled surgical instruments, cameras in tubes and joysticks that are far more convoluted than your bog standard Xbox 360 controller.
That next generation equipment isn't cheap but the surgeons have to rely on tiny video displays to make sure they're not cutting through vital organs.
"We could define basically a force field around, say, a liver," said Chizeck. "If the surgeon got too close, he would run into that force field and it would protect the object he didn't want to cut."
Engineers have been working to integrate gaming's force feedback technology into the robots, translating the tiny bumps - via the Kinect - into larger forces that are felt on the surgeon/operator's end of the link. In order for 'force feedback' technology to work properly, it needs some sort of frame of reference to tell it when the robot is brushing against a bone.
"Originally the group planned CT scans to provide the data, but soon the group got the idea of using a depth camera to provide a more precise picture by measuring infrared light reflected off of the surface. Last month, the group decided to use Microsoft's Kinect, "for obvious reasons."
The news will please Steve Balmer, Microsoft's president, who was quoted on last weekend's edition of BBC Click, when presenter Spencer Kelly interviewed him, as saying that Microsoft is working on modifying the Kinect with firmware updates itself, following the various hacks that programmers have made in recent months.
This hasn't stopped Microsoft from 'getting legal' with some of the Kinect hackers, but it is reports like this one that could well push Microsoft into releasing a Kinect devkit soon.