Context-Aware Notifications for Wearable Computing

Design Space of Notification. We have identified the user's and the environment's interruptability as key factors for determining whether or not to notify the user and for selecting the best notification modality. Both factors have to be considered independently in order to select the best notification. Together they span the Design Space of Notification, depicted on the left. It allows to evaluate situations and notification modalities in a structured manner.
For example the situation "Boring Talk" means, that the user is highly interruptable, but the environment must not be interrupted, ie. its interruptability is low. For the situation "Driving a Car" the opposite is true: the driver must not be interrupted because he needs his attention to drive a car, but for the environment it would matter much, since there are few people around.

Additionally this space allows to classify notification modalities according to their interruptiveness. E.g. an HMD has a low social interruptability, but a high personal one, especially when blinking or motion is employed.

Inferring the Interruptability. When it comes to recognizing the current interruptability using sensors, we found it inappropriate to model entire situations as the ones depicted in the design space above. Considering for example the situation "working in your office". You could have either a meeting with your boss or you could be working alone – each time with a different interruptability. Increasing the number and granularity of situations would obviously help, but at the cost of specifying many special cases for a typically large number of contexts.

We therefore opted to infer the interruptability directly from the low-level sensor data. For every sensor we define a Tendency in which area the interruptability is likely to be, given that this sensor indicates the presence of a context. Since our sensors are in fact classification sub-systems, we also obtain a likelihood for every sensor reading. The tendencies are weighted with the recognition likelihoods and summed. The maximum in the space then defines the interruptability of the user.

Sensor Technology. We use three different sources of context information: body-worn acceleration sensors to classify the user's activity, a microphone to classify the auditory scene and a simple location sensor, based on the current Wireless LAN Access Point. We have built a recording system that allows to record these three contexts at the same time. It consists of a laptop with two attached smart-its, a microphone and a Wireless LAN PCMCIA card.

The acceleration sensors are partitioned in two sets of six 3D sensors each. Each set has a sensor for the shoulder, arm, wrist, hip, knee and ankle. It is attached to a smart-it, which is in turn attached to the laptop over a serial port. Data is sampled at 100 Hz.

We use a Bayesian Classifier to classify the data into one of sitting, standing, walking, stairs up, and stairs down. We currently use mean and variance calculated over the last 50 samples.

We use a HMM classifier to classify the audio into one Lecture, Street, Conversation, and Restaurant. We currently use six different features and 2-state fully connected HMMs. See Context-Aware Notifications for Wearable Computing for a complete description.

Results. Using the above setup we have conducted experiments to show the feasibility of our approach. Current results show that we're in about 95% of the time close enough to the actual user interruptability so that the correct notification modality would have been chosen. See Context-Aware Notifications for Wearable Computing for a complete description.

As future work, we plan to evaluate the existing system on long stretches of data, rather days than just hours. Furthermore, we're interested in evaluating different modes of user interaction with such a system. Therefore we're currently working on building a real-time system, for long data recordings.