Kinect Hacks
DUO Project from the NUI Group needs your help !
Hello,
it’s been a while since anything has been posted here.
And yet, the NUI Community has never been so productive and involved than today !
If you didn’t hear about the DUO Project, i like to think of that as the golden child of the Kinect and the Leap Motion.
Code Laboratories one again created the true Open Source product that is the result of the past few years research and development in the 3D sensing field.
Click here to view the embedded video.
Unfortunately, there is only 6 days left on the Kickstarter, and they only have done half of the goal… If you feel like you would enjoy such a device, feel free to back the project, and share it to all your friends !
Next Generation Kinect for Mobile Devices (Current Smartphones, Tablets & NBs)
According to ABI Research’s report, it is projected that over 600 million mobile devices will be produced with vision-based behavior recognition features in 2017. However, there are still several technical problems need to be solved with these features first:
1. Under Different Lighting Conditions
2. Interferences From Moving Backgrounds
3. High Power Consumption
4. Large Size of the Mobile Device Itself
5. Double BOM Cost for the 2nd Camera (For color video calling and color vision sensing)
This technology solves these problems above for tablet PCs and mobile phones.
Two Implemented Applications:
The team solves the mentioned problems for tablet PCs and smart phones and has implemented 2 applications as below:
1. It successfully replaces Slide-to-Unlock patent from Apple.
2-1. No interruptions from the power-saving setting (It will turn off the backlight frequently) for long-time using circumstances. Such as watching a movie, reading a book, checking the stock market or reading a map on the mobile device.
2-2. A great deal of power can be saved from our module, as the screen from the device will be turned off immediately once the user is away. It also saves the power from using the video applications that are with Auto-play settings.
(2-1. & 2-2. also like Samsung S4′s Smart Pause and Smart Stay combined)
Click here to view the embedded video.
█ Power Consumption: 0.1444mW (Mass product edition, not prototype in video; By the way, Kinect is 13200mW):
According to the 2 major applications: To read and to unlock, the module only will be turned on immediately once it detects the user (the operation rate is really low) and it will be always off for the rest of the time when the device is not being used.
Take reading for example. The backlight timeout preset time of the most popular 10-inch iPad is 5 minutes and the backlight timeout preset time of a 7-inch Samsung tablet is 30 seconds. Our module only takes 0.1 seconds to operate, the operation rate is only 0.0222%, and the power consumption is only 0.1444 mWh. Imagine how much power our module can save from those settings.
█ Power Consumption of the Display Backlight:
Take 7 to 10 inch tablets that are released in the market recently for example, there are about 20 to 36 backlight LEDs in those tablets. When the tablet is not being used but the backlight is still on, the power consumption from all those backlight LEDs can be assumed as how much power we can save by using our module. Furthermore, our module only uses 1 to 2 LEDs, it only detects in every 15 seconds or 2.5 minutes and it takes only 0.1 seconds to operate.
█ Power Consumption from the Power Button:
According to Sony’s research, people press the power button 76 times a day on their phones. If people use their phones 15 hours a day, the button would be pressed 5 times an hour. According to the regular using habits, if people forget to press the power button to turn it off after using their phone twice an hour, our module can save more than 923 times power that is consumed from a 10 inch LCD screen.
Our previous work: http://www.kinecthacks.net/using-1-camera-to-integrate-live-foreground-fake-background-in-real-time/
Sky Chen’s LinkedIn: tw.linkedin.com/pub/sky-chen/67/489/80/
Kinection — turns your Kinect into a game design device
Looking for something that will allow you to sue your Kinect to creat games with a 3D engine? Then you may want to check out Kinection.
Kinection is a free plug-in that will let you use your Kinect withe Shiva Engine 3D, giving you the opportunity to turn your Kinect’s powerful camera into a vital tool for your planned gaming masterpiece.
What’s more, Kinection works with both Kinect for Windows and Kinect for Xbox 360. All you have to do is download Microsoft SDK 1.6 for the latter (which most of you have probably done by now, anyway).
To learn more about this plug-in, be sure to visit the website using the link below.
moveIT — turn your body into an Autodesk Inventor modeling tool wit a Kinect
moveIT is an add-in application for Autodesk Inventor that allows you to pose and position a human body model, using your own movements and a Microsoft Kinect sensor and movements of your own body. For example, if you lift a hand or bend an elbow, the pose of the human body model open in Autodesk Inventor updates accordingly.
Here are some key features of the add-in:
- All major joints are supported: head, torso, arms, elbows, shoulders, hips, knees and leg
- Ability to control not only the pose but the position of the model as well: if you jump or make steps in any direction, the position of the human body model in Autodesk Inventor updates accordingly
- Ability to adjust size and proportions of the human body model to better fit specific needs
- Ability to interact with the virtual environment, ‘activating’ and ‘deactivating’ blocks with a touch
For more details, be sure to visit the moveIT website using the link below.
Imogen Heap talks Ableton-controlling gloves
This isn’t the first that we’re seeing these gloves but in this demo, Imogen Heap does a good job of explaining how the technology behind the Ableton-controlling gloves came about and evolved and what exactly it’s capable of.
“It’s really like playing an instrument in 3D space,” said Heap, and she certainly makes it look like that as you watch her use the gloves.
Give it a look and have a glimpse of where Kinect-powered performances are heading.
Skeletal hand and finger tracking in full 3D
One thing that KinectHackers have found really useful with the Kinect is its capability to make 3D and 3D-related applications not only a reality but also affordable. Today’s featured hack is an example of that.
Here are some tech details from the developer:
“Markerless articulated skeletal hand tracking research from 3D depth data providing pose (position and orientation) of all 17 bones in user’s hand and fingers. Tracking algorithm as well as graphics application and data gathering runs around 30 FPS running single threaded on Intel x86 processor. 60fps on faster desktop PC.”
According to the developer, their tracking technology is original and not based on any of the current methods out there now (i.e. its not particle swarm nor decision tree systems). It was also the same hand tracking technology that was used on the Snake demo that Intel used for CES 2013. You can watch on the video, courtesy of The Verge, at the 50-second mark.
Artec Studio launches $1000 Kinect Contest
Artec Group, developers of Artec Studio, a software that turns Kinect, Asus Xtion or Primesense sensors into a real 3D scanner have launched a “cool” winter-themed contest for Kinect owners worldwide.
The contest launched in late January 2013 and is ongoing until Feb. 28. According to Artec Group, “the main idea is to scan any object on winter theme and submit it to viewshape.com.”
The winning entry can win up to $1000 but there are consolation prizes as well. Not bad if you ask us, especially since you’ll get to try out something new and potentially fun fun.
The contest is free to enter and interested individuals can submit as many entries as they’d like. Winners will be announced on March 4.
For more details, be sure to check out the link we’ve included below.
Skanect — an affordable 3D capture tool
Skanect is a new affordable, easy to use tool that allows users to capture a full color 3D model of an object, a person or a room.
Skanect transforms your Microsoft Kinect or Asus Xtion camera into an ultra-low cost scanner able to create 3D meshes out of real scenes in a few minutes. It leverages consumer-grade 3D cameras, thereby limiting the hardware cost to a fraction of the cost of previous scanning solutions.
It’s also as simple to use as a point-and-shoot camera – just move around your 3D camera to capture a full set of viewpoints, and you will get a mesh at interactive speeds.
For a personal and hobbyist use, you can even download a free version of Skanect! Check it out using the link below.
control:mapper let’s you customize controls with a Kinect
Today’s featured shack shows how Reality Control’s new software, ‘control:mapper’ enables you to interact with existing games and applications using your Microsoft Kinect and a Windows 7 or 8 PC
‘control:mapper’ allows you to assign any combination of keys and mouse control to defined gestures, or virtual triggers located in the space around your body, allowing you to direct the action in your PC using nothing but your body.
One of the good things about ‘control:mapper’ is that it makes software more accessible for people who may be physically disabled. It does this by customizing control to suit each individual’s specific needs. The software also automatically detects and adjusts for whether you are standing or seated.
Learn more about ‘control:mapper’ by visiting the link below.
Equipment Training via Kinect
The Institute for Energy Technology in Norway has used the Microsoft Kinect to create a virtual reality interface to explore and interact with virtual equipment. This could be used for instance in training to understand and maintain equipment.
The interaction consists of a Kinect as well as a mouse button used for selecting virtual objects, which allows for a rich interaction repertory and simple interaction. With this setup, the user can manipulate virtual objects to move, rotate and take components apart.
These virtual equipment mock-ups lets users get hands-on experience with the equipment, and allows for practice of assembly and disassembly tasks.
It’s a great example of how the Kinect can be used to create more interesting learning tools that can keep an audience engaged. We’re looking forward to seeing more displays like this.
The Rangeinet — a self-contained multimedia instrument
We’ve noticed that performance artists have really connected with the Kinect. A lot of cool new music and dances are being created with the help of Microsoft’s little miracle machine. Today’s hack is no different.
The Rangeinet is a self-contained multimedia instrument. It uses an IR camera to sense the objects in front of it, assigns object positions and sizes to musical parameters, and then projects onto those objects as they are sonified.
The instrument can scan across the X, Y, or Z axes, and derives the bounds and centers of each object across all three dimensions. It can scan across a dimension for momentary values, or scan by whole discrete objects. The scan path can be smooth or jagged and quanitzed to create regularized rhythms. Both synthetic and sampled instruments are available, as are delay effects and percussive elements.
Learn more about The Rangeinet at developer Dan Iglesia’s website. We’ve got the link for you below.
Kinect-powered Google TV controller
Virtual controllers are always cool. They let you do stuff without having to worry about the hassle of of physical-based interfaces. And they’re always interesting to work with.
Today’s featured hack is another great example of Kinect-powered virtual controllers. This oen though is geared for use with Google TV. Here’s a description of the hack from the developer.
“I’ve written a program that allows you to use a Kinect sensor to control the pointer on a Google TV device. The Kinect sensor is used like the touch control of a Google TV remote control.
“The program is based on the simple OpenNI wrapper for Processing. The program also uses the Anymote-for-Java library”
For those interested in learning more about the hack, the code is open-source and can be accessed via the handy-dandy link below.
Synapse for Kinect
Synapse for Kinect can be a powerful tool for music-making when set up with a bit of care. It can become a way of performing and constructing complex tracks on the fly.
As with many new developments, the initial novelty of being able to make music with a Kinect is great fun to play with. But as with any musical instrument it takes time and patience to learn how to use it as a serious live performance tool.
This track uses Synapse to control five different synths/devices in Ableton Live. By carefully mapping different sections of the range of movements on x, y and z axes, two arms can simultaneously transmit MIDI notes, control effects (such as filters) and mix the volumes for each of the tracks. The drum loops are triggered using a Wii remote and OSCulator.
Its interesting to see how different approaches to the development of Kinect/music software create different possibilities for performance. Kinectar and Synapse use different approaches to turning movement into music – the kinds of music its possible to create are different for each. In the long run, it is the interaction between developers and musicians (and both these developers are musicians) that will lead to more exploration of the amazing potential for musical expression through gesture and movement that the Kinect offers.
As the hardware/software technologies mature (with lower latency and higher resolution than the current Kinect) there is plenty of potential for gestural performance to become as subtle and expressive as any traditional instrument.
Space Recall for Kinect
Space Recall is, the developer claims, an interactive image device for recalling the space in memories.
“The original static image which was pieced together will instantaneously switch to a dynamic image through the movement and the change of viewpoints of the audience. To the image, the audience is active and able to decide his viewpoints. In the meantime, it is also the memory recall and exploration from the space where the audience has ever been.”
According to the developer, when being in a certain space, the audience could take photos there from different viewpoints and upload to Space Recall. the app adopts Microsoft Photosynth technology, something which automatically pieces together the photos from the same space into a panorama.
In the future, the audience can move back to the space in the past via Space Recall. Microsoft Kinect’s depth camera calculates user’s body shifts and viewpoint changes to simulate the visual experience in the real world. It is true to life as if the space in the memory appears before the eyes.
Space Palette – Music and Visuals via the Kinect
The Space Palette is a musical and graphical instrument that lets you play music and paint visuals simultaneously by waving your hands in the holes of a wood frame. No prerecorded media, sequences, or loops are used – everything is generated in realtime by your hands.
The wood frame is a reference for the player, while the Microsoft Kinect is used to detect the position of whatever hands (or objects) appear in the holes of the frame. The depth of your hands matters as much as their left/right/up/down position – it’s like having multiple three-dimensional mouse pads in mid-air.
Any number of hands can be used. Musically, the large holes are like piano keyboards (left-to-right) on which you play individual notes, and hand depth controls things like vibrato and filters. Visually, the large holes allow you to paint with graphical shapes (heavily processed by visual effects), and hand depth controls their size. three dimensions, and hand depth controls the size of the graphics.
The 12 small holes in the corners of the Space Palette are used to select different sets of sounds and graphics. Each of the 4 large holes plays a different sound and paints a different graphic, simultaneously.
Kinect Penalty Game
Football fans (Soccer to all American Kinecthackers out there) should get a kick out of this one. Check out this penalty game from / El Triple/ medialab, a company dedicated to the development of games and applications for events and marketing.
The interactive development consists balls shot from the penalty spot. The user can shoot in the direction you want, with five opportunities to score a goal.
The company was in charge of developing the game for a an event sponsored by Club Atlético River Plate and Progressive Association, and wanted something that would utilize the Kinect’s amazing gaming and promotional potential.
The event was held in 2012 at the auditorium in Recoleta Buenos Aires, Argentina.
SkiRanger Beta v1.4 now out!
Some months back, we featured an interesting little game called SkiRanger. For those who ended up liking the game as much as we did, we’ve got great news for you — v1.4 is now out!
The developer promises the following enhancements following feedback from fans everywhere:
- The game now supports Kinect for Windows and OpenNI officially (all known hacks are still working as well). It doesn’t matter which sensor you are using and what driver you have installed, the game will work.
- SkiRanger now has multi-player support, up to 4 players
- The beta also includes an experimental stunt level to show off the new pipes and grind tricks (We wanted to know if it would be cool to have a motion-controlled Tony Hawk type of game).
- We have also included some funny AR apps to show off some Kinect use cases.
The developer says that they’re currently working to overcome some “Kinect licensing issues” and until that’s resolved, the game will probably stay in beta (and free). if any of our Kinecthackers out there can help them out with this conundrum, I think they’d really appreciate it.
To contact the guys behind SkiRanger, drop by their website and give them a shout. A handy-dandy link is available for you below!
3D stereoscopic game with gesture controls for 3D LG CINEMA TVs
Russian company United 3D Labs developed a 3D stereoscopic interactive game as part of a promo event for 3D LG CINEMA TVs.
The user controls the game via gestures, utilizing a Kinect sensor. The game was made using a 3D stereo format and demonstrates all the LG CINEMA 3D TVs advantages in an interesting and highly interactive way. It was also a good showcase of how emerging technologies, motion controls and 3D viewing in this case, can combine to create even better entertainment tools.
The promo, which was held at one of Moscow’s biggest cinemas, was a huge hit with customers and is another great example of how you can use the Kinect to make advertising campaigns more attention-grabbing and engaging for consumers.
Learn more about the developer and their projects using the link below.
A Kinect-powered interactive picture frame
We’ve seen how the Kinect can be used by different people to create various iterations of art but what about a Kinect hack that lets you display art? Check out this interactive picture frame:
- The xyz-position of the viewers head in the room is connected to the position of the virtual camera.
- Pacing back and forth from one wall to the other will change the image.
- The app uses maxmsp with a kinect sensor.
It looks like something galleries can use to spruce up their displays or maximize their limited space. What do you think?
To learn more about the project and the developer, be sure to visit the project website using the link below.
Have a spooky experience with Ghosthunter
Halloween may be a couple of months away but we thought we’d share something that could make your next Kinect gaming session a scary but fun experience — Ghosthunter!
In Ghosthunter, the player explores a haunted house to find and capture wandering spirits. To aid them in this quest, players have a ghost detector and capture unit, which they use to scan the room for poltergeists. As they scan, the ghost detector, which in this case is your Xbox 360 controller, vibrates when a ghost is near. Here are a little more details from the developers:
“The player moves the Xbox 360 controller around in the Kinect-visible space to act out the process of scanning the room, bringing them closer into the action. When a ghost pops out, players are so engaged that they literally jump in surprise!”
Ghosthunter, which won the UCSC Sammy Awards 2012 Practicum Prize, looks like a fun little title that brings us back to the Kinect’s gaming roots. Check it out and let us know what you think!
