Providing Contextual Assistance in Response to Frustration in Visual Analytics Tasks

This work proposes a method for helping users in visual analytic tasks by using machine learning to detect and respond to frustration and provide appropriate recommendations and guidance. I have collected an emotion dataset from 28 participants carrying out intentionally difficult visualization tasks and used it to build an interactive frustration state detection model which detects frustration using data streaming from a small wrist-worn skin conductance device and eye tracking. I have present a design exploration for interventions appropriate to different intensities of frustrations detected by the model. The interaction method and the level of interruption and assistance can be adjusted in response to the intensity and longevity of detected user states.


Detecting Negative Emotion for Mixed Initiative Visual Analytics

The work describes an efficient model to detect negative mind states caused by visual analytics tasks. I have developed a method for collecting data from multiple sensors, including GSR and eye-tracking, and quickly generating labelled training data for the machine learning model. Using this method I have created a dataset from 28 participants carrying out intentionally difficult visualization tasks. I have concluded the work by a testing the best performing model, Random Forest, and discussing about its future applications for providing just-in-time assistance for visual analytics.

Talking Hands

We investigate the utility of mobile accelerometer data to identify human hand gestures and recognize the sign language using Recurrent Neural Networks (RNNs). A set of unsupervised features are learn, to recognize the phrases from American Sign Language (ASL), using Restricted Boltzman Machines (RBM). We validate the efficacy of our method by comparing it with the best performing supervised feature set — the proposed unsupervised features outperform the traditional handcrafted features. We use a labelled dataset of 600 accelerometer readings collected from 50 users to validate our approach.

Continue reading

Gaze-Based Text Entry Systems

Text input is a common task in interaction research. The alternative text production systems are typically tested by usability studies and also by standardized performance measurements like word-per-minute, Error Rates etc. Text entry by eye gaze is quite similar to any screen-based text entry technique, such as the on-screen keyboards. The interface is more or less the same, only the interaction technique of pointing and selecting changes as eye gaze is used instead of a stylus or other pointing devices. However, this change in the interaction technique brings a number of design issues that make text entry by eye gaze a unique technique with its own set of research problems.

Continue reading


The concept of CampusConnect (an intranet social networking site) is based and keeping the simple motto – “How can I Help?” and not “What I can Get?” This is to create a positive impact on personal and professional level particularly in the ‘Way of Life in the Campus’, by means of exchange of information or services among individuals, Groups or even like-minded teams who ‘Care and Share’. Most importantly, cultivation of productive relationships within the campus by means of ‘Connecting’ with one another.

Continue reading