Site-wide links

Glossary of Common Gestures

After examining multiple projects for common gestures, the following movements were agreed upon to be the most commonly defined gestures for specific actions. While certain middleware libraries and utilities may have pre-defined gestures available, many developers choose to define their own custom gestures based on skeletal data received from the Kinect. Users should refer to individual middleware documentation for details on whether pre-defined gestures exist within the system,  the extent of gestural/skeletal data available, and the possibility of creating additional/custom gestures from that data.

Forward

  • Step Forward (movement in the physical space)
  • Step Forward with distance (speeding up/slowing down)
  • Leaning Forward
  • Walking place
  • Both arms forward
  • Hand movement (up/down, positional)

Jump

  • Jumping
  • Hand up (raise the roof)
  • Flicking (hand flick)
  • Kick a leg out
  • Left single leg
  • Raise Elbows
  • Tip toes (possible false positives, slouching, etc)

Left

  • Leaning
  • Left Arm out
  • Left foot out
  • Rotating
  • Both arms left
  • Leftward motion
  • Turn head slightly to the left

Right

  • Leaning
  • Right Arm out
  • Right foot out
  • Rotating
  • Both arms right
  • Rightward motion
  • Turn head slightly to the right

Rotate

  • Rotation at the hips (hold, rotate, release)
  • one hand up, one hand swipe
  • Wax on wax off (circular motion)
  • arm out left/right for positive/negative rotation
  • Right arm to left – left rotation (hold, rotate, release)
  • Left arm to right – right rotation (hold, rotate, release)

Looking Around

  • Vertical/horizontal separation of hands (distance)
  • Head tracking
  • Mouse look (without the mouse)
  • Head mounted display
  • Clasped hands pointing

Interaction

  • Compression of hands
  • Alternate hand action
  • Sound/speech based

Additional Avatar Interactions

Note: The following gestures, while commonly used, are used for more specific applications than general gestural interaction. These gestures are usually specific to the project or application in question and may include more customized gestures than those listed here. These simply serve as a guideline for some of the more common specialized actions.

  • Aim gun – hand pointing outward
  • Attach two objects to each other – push with left hand to grab one object, push with right hand to grab another object, bring hands together
  • Attack with melee weapon – swiping motions
  • Cancel action – swipe with both hands in opposite directions
  • Close application – make X shape with arms
  • Eating/Drinking – holding hand up to mouth
  • Fire gun – arc arm upward at elbow joint
  • Grabbing something – compression of hands
  • Hold steering wheel – hands holding imaginary steering wheel, hands on 9 and 3
  • Keyboard control – swype texting using single hand, hold hand in place for a specified time to press a key
  • Mouse left click – left/right arm out, depending on what is comfortable for the user
  • Mouse right click – left/right arm out, depending on what is comfortable for the user
  • Moving a clasped object – dragging clasped hands
  • Multiple selection – place left hand at one point, right hand drags downward, push with both hands to complete selection
  • Opening something – open hands, as if holding a book
  • Rotate steering wheel – rotate hands in a circular motion
  • Rotate suspended object in x dimension – swipe left/right
  • Rotate suspended object in y dimension – swipe up/down
  • Rotate suspended object in z dimension – swipe diagonally
  • Scale suspended object in x dimension – separate/join both hands left/right
  • Scale suspended object in y dimension – separate/join both hands up/down
  • Scale suspended object in z dimension – separate/join both hands diagonally
  • Select item from horizontal toolbar / control slider – place left hand at left side of toolbar, use right hand as relative distance from left hand for selection, push to finalize selection
  • Separate two objects from each other – push with left hand top grab one half of an object, push with right hand to grab the other half of the object, pull hands apart
  • Sitting – crouching
  • Sleep – prop tilted head by both hands
  • Store object inventory – open imaginary trench coat and place inside
  • Switching weapons – taking hand over head to touch back
  • Throwing a clasped object – moving clasped hands in a direction the unclasping
  • Toggle interaction – hand push at location of object
  • Translate suspended object in x dimension – slide hand left/right
  • Translate suspended object in y dimension – slide hand up/down
  • Translate suspended object in z dimension – push hand forward, pull hand backward


References:
Kinect Gestural UI Tips
http://www.useit.com/alertbox/kinect-gesture-ux.html
Hand/arm positioning
http://kinecthacks.net/kinected-control-freak/
http://kinecthacks.net/taming-ar-drone/
Basic interaction techniques
http://kinecthacks.net/kinected-space-invaders-at-electroshock/
http://www.youtube.com/watch?v=8CTJL5lUjHg
http://www.youtube.com/watch?v=62wj8eJ0FHw
http://www.youtube.com/watch?v=jm10fwsTfw0
Compilation of various hacks
http://www.youtube.com/watch?v=y8VLnr5ylug
Holding an object
http://www.youtube.com/watch?v=4yp37U-YHv4&feature=related
Manipulating 3D space
http://www.youtube.com/watch?v=7agnTVMjSy4&feature=related
http://www.youtube.com/watch?v=Brpu30vjCa4&feature=related
Fingertip detection
http://www.youtube.com/watch?v=5nZSVjY_gNo&feature=related
http://www.youtube.com/watch?v=kf3G-DXqt6Y&feature=related
http://www.youtube.com/watch?v=PY_Ip1HChe4&feature=related
Object recognition
http://www.youtube.com/watch?v=cRBozGoa69s&feature=related
D. A Bowman, D. Koller, and L. F Hodges, “Travel in immersive virtual environments: An evaluation of viewpoint motion control techniques,” in vrais, 1997, 45.   
-Appears to cover some of the basics of various virtual world control schemes
3D User Interfaces: Theory and Practice (Boston: Addison-Wesley, 2005).  
( http://albert.rit.edu/record=b1909543~S3 )
 
A. Kulik, “Building on Realism and Magic for Designing 3D Interaction Techniques,” Computer Graphics and Applications, IEEE 29, no. 6 (2009): 22-33.  
( http://ieeexplore.ieee.org.ezproxy.rit.edu/xpls/abs_all.jsp?arnumber=5307640 )