Research

This page lists the various research projects I have worked on at Carnegie Mellon University for graduate school, the Institute for Infocomm Research in Singapore, Cornell University during my undergraduate studies, and pre-college at the Nanyang Technological University, Singapore.

Thesis Defense: Improving Understanding and Trust with Intelligibility in Context-Aware Applications

I will be defending my thesis in late April 2012 about my work in providing intelligibility in context-aware applications.

When:   April 23rd, Monday @ 9.30am
Where:  Newell Simon Hall 1507

THESIS DEFENSE
Improving Understanding and Trust with Intelligibility in Context-Aware Applications

COMMITTEE
Anind K. Dey (Chair), Carnegie Mellon University, Human-Computer Interaction Institute
Scott E. Hudson, Carnegie Mellon University, Human-Computer Interaction Institute
Aniket Kittur, Carnegie Mellon University, Human-Computer Interaction Institute
Margaret M. Burnett, Oregon State University

DOCUMENTS
Flyer
Dissertation

ABSTRACT
To facilitate everyday activities, context-aware applications use sensors to detect what is happening, and use increasingly complex mechanisms (e.g., by using big rule-sets or machine learning) to infer the user’s context and intent. For example, a mobile application can recognize that the user is in a conversation, and suppress any incoming calls. When the application works well, this implicit sensing and complex inference remain invisible. However, when it behaves inappropriately or unexpectedly, users may not understand its behavior, and this can lead users to mistrust, misuse, or even abandon it. To counter this lack of understanding and loss of trust, context-aware applications should be intelligible, capable of explaining their behavior.

We investigate providing intelligibility in context-aware applications and evaluate its usefulness to improve user understanding and trust for context-aware applications. Specifically, this thesis supports intelligibility in context-aware applications through the provision of explanations that answer different question types, such as: Why did it do X? Why did it not do Y? What if I did W, What will it do? How can I get the application to do Y? Etc.

This thesis takes a three-pronged approach to investigating intelligibility by (i) eliciting the user requirements for intelligibility, to identify what explanation types end-users are interested in asking context-aware applications, (ii) supporting the development of intelligible context-aware applications with a software toolkit and the design of these applications with design and usability recommendations, and (iii) evaluating the impact of intelligibility on user understanding and trust under various situations and application reliability, and measuring how users use an interactive intelligible prototype. We show that users are willing to use well-designed intelligibility, and this can improve user understanding and trust in the adaptive behavior of context-aware applications.

Page last modified: July 6, 2012

Laκsa: Mobile App for Intelligible Control of Interruption

laksa-screenshots

Mobile phones allow people to keep in touch with others and be easily reachable. However, the increasingly intimate use of smartphones also risks more social disruptions (e.g., in meetings and movie theatres) and work interruptions. This is because current smartphones are not smart enough to comprehensively understand the context of where its owner is, what he is doing, what is socially appropriate, and with whom he can be connected to then, etc.

Therefore, we have developed Laκsa, a mobile app to automatically infer the user’s context for social availability. It uses the rich sensors in smartphones (e.g., GPS, microphone, accelerometer, calendar) together with sophisticated machine learning algorithms to infer contextual cues, such as whether the user is in an impromptu conversation at the office, on an evening run outdoors, or at home listening to music. With this, Laκsa can provide contextually relevant features such as automatically silencing or activating the phone’s ringer in an intelligent and appropriate manner.

Laκsa is also intelligible to communicate with users. Using algorithms to provide explanations, Laκsa helps users to understand what it knows and how it makes inferences, and enables users to share such situational and social understanding with friends and family. Hence, Laκsa uses location and activity to connect (κ) users for social awareness.

Publications

We have published several research papers on using Laκsa to investigate the design of intelligible visualizations of context-awareness and to evaluate the usefulness of intelligilibility.

  1. Lim, B. Y., Dey, A. K. 2011. Design of an Intelligible Mobile Context-Aware Application. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 157-166. DOI=10.1145/2037373.2037399
  2. Lim, B. Y., Dey, A. K. 2011. Investigating Intelligibility for Uncertain Context-Aware Applications. In Proceedings of the 13th international conference on Ubiquitous computing (UbiComp '11). ACM, New York, NY, USA, 415-424. DOI=10.1145/2030112.2030168 .
  3. Lim, B. Y., Dey, A. K. 2013. Evaluating Intelligibility Usage and Usefulness in a Context-Aware Application. In Human-Computer Interaction. Towards Intelligent and Implicit Interaction. Springer Berlin Heidelberg, 2013. 92-101.
    pdf-icon
Page last modified: May 23, 2016

Second Workshop on Intelligibility and Control in Pervasive Computing

I am co-organizing a Pervasive 2012 workshop on Intelligibility and Control in Pervasive Computing with Jo Vermeulen and Fahim Kawsar to be held on June 18. This is the second year of the workshop. The Call for Papers is out and more information on the workshop can be found at the workshop website.

Page last modified: April 16, 2016

Thesis Proposal: Improving Understanding, Trust, and Control with Intelligibility in Context-Aware Applications

I will be presenting my thesis proposal in early May 2011 about my work in providing intelligibility in context-aware applications.

When:   May 2nd, Monday @ 1.30pm
Where:  Gates-Hillman Center 6115

THESIS PROPOSAL
Improving Understanding, Trust, and Control with Intelligibility in Context-Aware Applications

COMMITTEE
Anind K. Dey (Chair), Carnegie Mellon University, Human-Computer Interaction Institute
Scott E. Hudson, Carnegie Mellon University, Human-Computer Interaction Institute
Aniket Kittur, Carnegie Mellon University, Human-Computer Interaction Institute
Margaret M. Burnett, Oregon State University

DOCUMENTS
Flyer
Proposal

ABSTRACT
To facilitate everyday activities, context-aware applications use sensors to detect what is happening, and use increasingly complex mechanisms (e.g., by using machine learning) to infer the user’s context. For example, a mobile application can recognize that you are in a conversation, and suppress any incoming messages. When the application works well, this implicit sensing and complex inference remain invisible. However, when it behaves inappropriately or unexpectedly, users may not understand its behavior, and this can lead users to mistrust, misuse, or abandon it. To counter this, context-aware applications should be intelligible, capable of generating explanations of their behavior.

My thesis investigates providing intelligibility in context-aware applications, and evaluates its usefulness to improve user understanding, trust, and control. I explored what explanation types users want when using context-aware applications in various circumstances. I provided explanations in terms of questions that users would ask, such as why did it do X, what if I did W, what will it do? Early evaluation found that why and why not explanations can improve understanding and trust. I next developed a toolkit to help developers to implement intelligibility in their context-aware applications, such that they can automatically generate explanations. Following which, I conducted a usability study to derive design recommendations for presenting usable intelligibility interfaces of a mobile application. In the remaining work, I will evaluate intelligibility in more realistic settings. First, I shall explore the helpfulness and harmfulness of intelligibility for applications with high and low certainties. Finally, I shall investigate how intelligibility, through improving user understanding, can help the users to more effectively control a context-aware application.

Page last modified: April 10, 2012

ContextToolkit.org

I’ve recently launched a website for the new Context Toolkit that I’ve adapted from the original one built by my advisor, Anind, years ago. Visit www.contexttoolkit.org to learn more. There you can download v2.0 of the toolkit, and learn how to use it from tutorials there. The Intelligibility Toolkit is also now available for download as part of the Context Toolkit. Tutorials for how to use its various components are also located on the website.

Page last modified: January 17, 2011

Workshop on Intelligibility and Control in Pervasive Computing


I am co-organizing a Pervasive 2011 workshop on Intelligibility and Control in Pervasive Computing with Jo Vermeulen and Fahim Kawsar to be held on June 12. The Call for Papers is out and more information on the workshop can be found at the workshop website.

Page last modified: April 18, 2016

Toolkit to Support Intelligibility in Context-Aware Applications.

With a design framework in place from [Lim & Dey 2009], this work makes a technical contribution by facilitating the provision of 8 explanation types (Input, Output, What, Why, Why Not, How To, What If, Certainty) generated from commonly used decision models in context-aware applications (rules, decision tree, naïve Bayes, hidden Markov models). The Intelligibility Toolkit extends the Enactor framework [Dey & Newberger] by providing more types of explanations and supporting machine learning classifiers other than rules. We validate the toolkit with three demonstration applications showing how the explanations can be generated from various decision models.


intelligibility toolkit - architecture

Read the rest of this entry »

Page last modified: April 18, 2016