Talk 1

Locating the Design Problem: Design and the Development of Smart Products

Abstract: The initial development of "smart graphics" has opened the way to a new generation of digital products for both specialized and general audiences. But to move such products from the early conception stage to fully realized products that will survive in the context of human use, we need to understand more carefully the differences among advances in software engineering, usability studies, and design thinking. The goal of this talk is to present some of the ways that the discipline of design particularly in the new form of interaction design can contribute to the creation of "total products" that employ smart graphics.

Delivering Concrete Data to MoMA and the New York Stock Exchange:
Perceptually/cognitively informed information design

Abstract: Abstract information can be more easily understood when it is cast in concrete visual representations. Two case studies of delivered, working systems are discussed: The design of a handheld wireless device for Goldman Sachs, and the interactive part of the Mind'space exhibit in the Workspheres exhibition, currently showing at the Museum of Modern Art, New York. A design approach based on the multiple abilities of the human brain is presented. This approach posits an "information processing pipeline" through which input is successively refined through sensory, perceptual, cognitive, symbolic, linguistic, and structural processes in the brain. The approach also suggests that behavioral, emotional, and social processes, while not strictly part of information processing, are equally valuable in the design process. This list of differing abilities is suggested not as mind science, but as a design checklist that can guide design and provide a framework for critique. The case studies are discussed in the context of this checklist: design features are related to the brain processes that inspired or explain them.

Tangible Interaction + Graphical Interpretation: A "Smart Graphics" Approach to 3D Modeling

Abstract: Artists using standard 3D modeling packages must specify precisely the geometric and material properties of the models they create, and therein lies much of the complexity and tedium of using those tools. By contrast, children playing with construction toys like Lego and K'nex make simple models easily, and use their imaginations to fill in the details. We would like to transform computer-based geometric modeling into that same kind of playful, tactile experience but without sacrificing the ability to create the interesting geometric detail and movement that make 3D graphics and animation compelling. To retain the tactile experience of model manipulation, we look to tangible-interface technology; and to create detailed, fully realized models, we use "Smart Graphics" methods for graphically interpreting a nascent model by recognizing and augmenting its salient features. This combination of tangible interaction and graphical interpretation is investigated in a pair of case studies. Tangible modeling can be approached in two ways: either by directly instrumenting the modeling medium with embedded computation or by using external sensors to capture the geometry. Our first system consists of computational building blocks assembled into physical structures that in their aggregate determine and communicate their own geometric arrangement. A rule-based system interprets these structures as buildings, parses their architectural features, then adds geometric detail and decorative enhancements automatically. Our second system uses simple and robust computer vision to capture volumetric scans of clay models of such common toy-like objects as people, animals, trees, houses, cars, and boats. A volumetric matching algorithm allows us to recognize, interpret, and bring the clay models to (virtual) life. Joint work with: D. Anderson, J. Frankel, A. Agarwala, P. Beardsley, J. Hodgins, D. Leigh, K. Ryall, E. Sullivan, and J. Yedidia.