Deceptive behavior expressed through nonverbal data is commonly very complex, multi-dimensional, and noisy, involving different kinds of data (numeric, symbolic, visual, etc.). Some of those behaviors could be person specific and may require temporal modeling with real-time performance.

Our approach is driven by automatically sensing and modeling the probabilistic dependencies of nonverbal channels (e.g. facial expressions, prosody, and body language) through development of an online interface to deploy our framework in the wild to collect and validate our generated model. Findings from this endeavor may inform a common model for conversational behavior dynamics.

By development of the framework we address how to automatically model the dependencies and interplay of human nonverbal behavior to predict deceptive intent; how to collect a large-scale dataset of spontaneous examples of deception; and how to design interfaces allowing a user to visualize, inspect and validate videos for deceptive intent.