Pei-Yu (Peggy) Chi

Users of complex software applications often learn concepts and skills through step-by-step tutorials. We hypothesize that a mixed tutorial with static instructions and per-step videos can combine the benefits of both formats. We present MixT, a system that automatically generates step-by-step mixed media tutorials from user demonstrations. MixT segments screencapture video into steps using logs of application commands and input events, applies video compositing techniques to focus on salient information, and highlights interactions through mouse trails.

A large community of users creates and shares how-to videos online. It is often difficult for the authors of these videos to control camera focus, view, and position while performing their physical tasks. To help authors produce videos, we introduce Kinectograph, a recording device that automatically pans and tilts to follow specific body parts, e.g., hands, of a user in a video. It utilizes a Kinect depth sensor to track skeletal data and adjusts the camera angle via a 2D pan-tilt gimbal mount. Users control and configure Kinectograph through a tablet application with real-time video preview.

A devastating earthquake of the magnitude 9.0 hit the Northeastern Japan on March 11, 2011, followed by tsunami, fire, aftershocks, and serious nuclear crisis. By collecting online media of the earthquake and related hazards presented in a map, we hope to help people learn more about this catastrophe, the serious damage it has caused, and the care and supports from the world. In this interface, users can navigate the map and see the photographs, videos, and more information provided by the mass and social media. Our goal is to encourage people to take actions to donate to the organizations and help people in need.

People who are not professional storytellers usually have difficulty composing travel photos and videos from a mundane slideshow into a coherent and engaging story, even when it is about their own experiences. However, consider putting the same person in a conversation with a friend – suddenly the story comes alive. Raconteur is a system for conversational storytelling that encourages people to make coherent points. It performs natural language processing in real-time on a text chat between a storyteller and a viewer and recommends appropriate media items from a library. A large commonsense knowledge base and a novel commonsense inference technique are used to identify story patterns such as problem and resolution or expectation violation.

The interactive narrative system, called "A Little Quiz for the Little Hare" (story based on S. McBratney and A. Jeram's "Guess How Much I Love You"), aims to teach young children the concepts of measurement and comparison through the conversation between two characters. It explores the design space of enhancing interactive narrative using a commonsense knowledge database to understand players' intention and generate relevant narration dynamically.

Consumer electronics devices are becoming more complicated, intimidating users. These devices do not know anything about everyday life or human goals, and they show irrelevant menus and options. Using common-sense reasoning, we are building a system, Roadie, with knowledge about the user's intentions; this knowledge will help the device to display relevant information to reach the user's goal. For example, an amplifier should suggest a play option when a new instrument is connected, or a DVD player suggest a sound configuration based on the movie it is playing. This will lead to more human-like interactions with these devices. We've constructed the Roadie interface to real consumer electronics devices: a television, set top box, and smart phone. The devices communicate over Wi-Fi, and use the UPnP protocols.

Technology enables us to possess increasing numbers of functional devices at home, but also brings potential concern of resource overflow that decreases the usability. Can a home select proper Ways to Think for users based on the higher-level knowledge about the properties of different devices? This project suggests a framework design to make a home “think” and utilize home devices using commonsense reasoning. The proposed system includes a Difference-Networks to describe devices, and a Critic-Selector to react to user needs.

Although modern ease of access to technology enables many of us to obsessively document our lives, much of the captured digital content is often disregarded and forgotten on storage devices, with no concerns of cost or decay. Can we design technology that helps people better appreciate captured memories? What would people do if they only had one more chance to relive past memories? In this paper, we present a prototype design, PY-ROM, a matchstick-like video recording and storage device that burns itself away after being used. This encourages designers to consider lifecycles and human-computer relationships by integrating physical properties into digitally augmenting everyday objects.

Stress OutSourced (SOS) is a peer-to-peer network that allows anonymous users to send each other therapeutic massages to relieve stress. By applying the emerging concept of crowdsourcing to haptic therapy, SOS brings physical and affective dimensions to our already networked lifestyle while preserving the privacy of its members. This paper first describes the system, its three unique design choices regarding privacy model, combining mobility and scalability, and affective communication for an impersonal crowd, and contrasts them with other efforts in their respective areas. Finally, this paper describes future work and opportunities in the area of haptic social networks.

This project addresses cultural, social, and spatial concerns, and promotes sustainability and local food cultures in the social housing community. We take the advantages of Slow Food movement that preserves the cultural cuisines and food ingredients, and combine with the efficiency of food transportation and knowledge exchange. We believe sustainability starts at home and in simple, everyday tasks, thus propose a connected sustainable kitchen design that includes a Community Storage Box, an interactive learning system, in a dynamic market-based distributed network.

During cooking process, family cooks are commonly unaware of how many calories go into their prepared meals. This work presents a smart kitchen with Ubicomp technology to improve home cooking by providing the number of calories of ingredients that are used in prepared meals. In doing so, family cooks can more effectively control the meal calories based on family needs. Our kitchen has sensors to track the number of calories in food ingredients, and then provides real-time feedback to users on these values.

Ubicomp technologies can assist parents and occupational therapists in modifying behaviors in young children. In occupational therapy, an effective mean to motivate child behavior change is by designing playful activities which leverages the desire of children to play to induce their behavioral change. By embedding digital technology into activities, Ubicomp technologies can be used to enhance the effectiveness of play-based occupational therapy. We demonstrated two playful activity designs targeting on slow eating and tooth brushing behaviors in young children.

We surveyed and classified smart everyday objects based on the relation between the object’s digital (new) and its traditional functions, and the relation between the object’s digital (new) object-human interaction and its traditional object-human interaction. Then, we attempted to map out a design space for digital function and interaction of smart everyday objects.

As technology enables new and faster modes of communication among people, it has become increasingly difficult to manage our social contacts effectively. Existing contact management tools lack the support for intelligent analysis of our social interactions. Thus, we developed an intelligent system of social network agents called "Déjà Vu" for managing and analyzing personal social contacts over the web.

By analyzing and transforming the processes into a simulating game, we developed a multi-modal, interactive and role-playing game for people to experience selling food in a disruptive night market environment.