This project aims to detect and classify emotional state from video sequences using natural body movement, pose and hand gestures. The original contribution of this project is to improve classification results by developing a novel technique for classifying features that use a three-dimensional feature cuboid and depth information. Motion constraints that directly affect the detection of emotion will be identified, resulting in a framework for detecting dynamic qualities of a gesture. Focus will be on developing an emotion processing engine to generate feature vectors for multiple modalties, and use classification techniques to detect emotional states.