Ph.D. Thesis

Title : Wearable Computing and Contextual Awareness
Date Completed: June 1999
Advisor: Dr. Alex Pentland
University: Massachusetts Institute of Technology

M.S. Thesis

Title : Visual Recognition of American Sign Language Using Hidden Markov Models
Date Completed: February 1995
Advisor: Dr. Alex Pentland
University: Massachusetts Institute of Technology

Paradiso, J., and T. Starner, "Energy Scavenging for Mobile and Wireless Electronics", IEEE Pervasive Computing, Volume 4(1), 2005, pp. 18-27.

Ashbrook, D., and T. Starner, "Using GPS to Learn Significant Locations and Predict Movement Across Multiple Users", In Personal and Ubiquitous Computing, Volume 7 (1), 2003

Starner, T., B. Leibe, D. Minnen, T. Westeyn, A. Hurst, and J. Weeks, "The perceptive workbench:Computer-vision-based gesture tracking, object tracking, and 3D reconstruction for augmented desks", In Machine Vision and Applications, Volume 14 (1), pp. 59-71, 2003

Leibe B., T. Starner, W. Ribarsky, Z. Wartell, D. Krum, J. Weeks, B. Singletary, and L. Hodges, "Towards Spontaneous Interaction with the PerceptiveWorkbench", In IEEE Computer Graphics and Applications, Volume 20 (6), pp. 54-65, November 2000

Starner, T. and Y. Maguire, "Heat Dissipation in Wearable Computers Aided by Thermal Coupling with the User", In ACM Journal on Mobile Networks and Applications (MONET), Special Issue on Wearable Computers, Volume 4 (1), pp. 3-13, 1999

Starner, T., J. Weaver, and A. Pentland, "Real-Time American Sign Language Recognition Using Desk and Wearable Computer-Based Video", In IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 20 (12), pp. 1371-1375, December 1998

Starner, T., J. Weaver, and A. Pentland, "A Wearable Computing Based American Sign Language Recognizer", In Personal and Ubiquitous Computing, Volume 1 (4), 1997

Starner, T., S. Mann, B. Rhodes, J. Levine, J. Healey, D. Kirsch, R. Picard, and A. Pentland, "Augmented Reality Through Wearable Computing", In Presence, Volume 6 (4), pp. 386-398, Winter 1997

Wren, C., F. Sparacino, A. Azarbayejani, T. Darrell, T. Starner, A. Kotani, C. Chao, M. Hlavac, K. Russell, and A. Pentland, "Perceptive Spaces for Performance and Entertainment: Untethered Interaction Using Computer Vision and Audition", In Applied Artificial Intelligence Journal, Volume 11 (4), pp. 267-284, June 1997

Starner, T., "Human Powered Wearable Computing", In IBM Systems Journal, Volume 35 (3), pp. 618-629, 1996

Azarbayejani, A., T. Starner, B. Horowitz, and A. Pentland. "Visually Controlled Graphics." In IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 15 (6), pp. 602-605, IEEE Computer Society Press, June 1993

Friedmann M., T. Starner, and A. Pentland. "Synchronization in Virtual Realities." In Presence, Volume 1 (1), pp. 139-144, MIT Press, Spring 1991

Starner, T. and Paradiso, J. "Human Generated Power for Mobile Electronics." In Low Power Electronics Design, C. Piguet (editor), CRC Press, In Press 2004

Starner, T. and B. Rhodes. "Wearable Computers." In Encyclopedia of Human-Computer Interaction, W. Bainbridge (editor), Berkshire Publishing, In Press, 2004.

Schiele, B., T. Starner, B. Rhodes, B. Clarkson, and A. Pentland. "Situation Aware Computing with Wearable Computers." In Fundamentals ofWearable Computers and Augmented Reality, W. Barfield and T. Caudell (editors), Lawrence Erlbaum Press, 2001.

Starner, T., B. Schiele, B. Rhodes, T. Jebara, N. Oliver, J.Weaver, and A. Pentland. "Augmented Realities Integrating User and Physical Models." In Augmented Reality: Placing Artificial Objects in Real Scenes, R. Behringer, G. Klinker, G. J. Klinter, and D. Mizell (editors), A. K. Peters Ltd., December, 1999.

Starner, T., J. Weaver, and A. Pentland. "A Wearable Computer Based American Sign Language Recognizer." In Assistive Technology and Artificial Intelligence, V. Mittal, H. Yanco, J. Aronis (editors), Springer Verlag, AI Series, pp. 84-96, 1998.

Starner, T. and A. Pentland. "Real-Time American Sign Language Recognition from Video Using Hidden Markov Models." In Motion-Based Recognition, M. Shah and R. Jain (editors), Kluwer Academic Publishers, Computational Imaging and Vision Series, 1996.

Friedmann, M., and T. Starner, and A. Pentland. "Device Synchronization Using an Optimal Linear Filter." In Virtual Reality Systems, Earnshaw, Gigante, and Jones (editors), Academic Press, 1993.

Pentland, A., I. Essa, M. Friedman, B. Horowitz, S. Sclaroff, and T. Starner. "The Thing-World Modeling System." In Algorithms and Parallel VLSI Architectures, pp. 425-434. E.F. Deprettere (editor), Elsevier Press, 1991.

Conference Presentations with Proceedings (refereed and archival)
[Refereed publications in respected conferences, with extensive reviewing, and appearing in archival proceedings.]

J. Clawson, K. Lyons, T. Starner, and E. Clarkson. "The Impacts of Limited Visual Feedback on Mobile Text Entry Using the mini-QWERTY and Twiddler Keyboards." ISWC 2005

T. Westeyn, X. Bian, K. Vadas, T. Starner, and G. Abowd. "Recognizing Mimicked Autistic Self-Stimulatory Behaviors Using Hidden Markov Models." ISWC 2005.

K. Lyons, C. Skeels, and T. Starner. "Providing Support for Mobile Calendaring Conversations: An Evaluation of Dual-Purpose Speech." Mobile HCI 2005.

S. Lee, V. Henderson, H. Brashear, T. Starner, S. Hamilton, and H. Hamilton. "User-centered Development of a Gesture-based American Sign Language Game." NTID Instructional Technology and Education of the Deaf Symposium, Rochester, NY, June 2005.

V. Henderson, S. Lee, H. Brashear, H. Hamilton, T. Starner, and S. Hamilton. "Development of an American Sign Language Game for Deaf Children." Interaction Design and Children (IDC), Boulder, CO, June 2005.

S. Lee, V. Henderson, H. Hamilton, T. Starner, H. Brashear, S. Hamilton. "A Gesture-Based American Sign Language Game for Deaf Children." CHI, Portland, Oregon, April 2005. pp. 1589-1592.

E. Clarkson, J. Clawson, K. Lyons, and T. Starner. "An Empirical Study of Typing Rates on mini-QWERTY Keyboards." CHI, Portland, Oregon, April 2005. pp. 1288-1291.

K. Lyons, C. Skeels, T. Starner, C. Snoeck, B.Wong, D. Ashbrook. "Augmenting Conversations Using Dual Purpose Speech." User Interface and Software Technology (UIST), Santa Fe, NM, November 2004. pp. 237-246.

C. Metzger, M. Anderson, and T. Starner. "FreeDigiter: a gesture interface for small devices." In IEEE International Symposium on Wearable Computers, Washington, DC, October 2004. pp. 18-21.

K. Lyons, T. Starner, and D. Plaisted. "Expert Typing Using the Twiddler One Handed Chord Keyboard." In IEEE International Symposium on Wearable Computers, Washington, DC, October 2004.

T. Westeyn and T. Starner. "Recognizing SongnBased Blink Patterns: Applications for Restricted and Universal Access", In IEEE Intl. Conference on Automatic Face and Gesture Recognition, Seoul, South Korea, May 2004. pp. 717-722.

R. McGuire, J. Hernandez-Rebollar, T. Starner, V. Henderson, H. Brashear, and D. Ross. "Towards a One-Way American Sign Language Translator", In IEEE Intl. Conference on Automatic Face and Gesture Recognition, Seoul, South Korea, May 2004. pp. 620-625.

P. Lukowicz, J. Ward, H. Junker, M. Stäger, G. Tröster, A. Atrash, T. Starner, "Recognizing Workshop Activity Using Body Worn Microphones and Accelerometers", In Pervasive Computing 2004. pp. 18-22.

T. Starner, C. Snoeck, B. Wong, and R. McGuire, "Use of Mobile Appointment Scheduling Devices", I n ACM Conference Human Factors in Computing Systems, Vienna, Austria, April 2004. pp. 1501-1504.

K. Lyons, T. Starner, D. Plaisted, J. Fusia, A. Lyons, A. Drew, and E. Looney, "Twiddler Typing: One-Handed Chording Text Entry for Mobile Phones", In ACM Conference Human Factors in Computing Systems Vienna, Austria, April 2004.

T.Westeyn, H. Brashear, A. Atrash, and T. Starner, "Georgia Tech Gesture Toolkit: supporting experiments in gesture recognition." In ACM International Conference on Multimodal Interfaces, Vancouver, BC, November 2003, pp. 85-92.

M. Stäger, P. Lukowicz, N. Perera, T. von Büren, G. G. Tröster, and T. Starner, "SoundButton: Design of a Low Power Wearable Audio Classification System." In IEEE International Symposium on Wearable Computers, White Plains, WA, October 2003, pp. 12-16.

H. Brashear, T. Starner, P. Lukowicz, H. Junker, "Using Multiple Sensors for Mobile Sign Language Recognition." In IEEE International Symposium on Wearable Computers, White Plains, WA, October 2003, pp. 45-53.

Minnen, D., I. Essa, and T. Starner, "Expectation Grammars: Leveraging High-Level Expectations for Activity Recognition" In IEEE Proceedings of Computer Vision and Pattern Recognition Conference 2003, Madison, Wisconsin, June 2003.

D. Ashbrook and T. Starner, "Learning Significant Locations and Predicting User Movement with GPS." In IEEE International Symposium on Wearable Computers, Seattle, WA, October 2002, pp. 101-108.

D. Krum, O. Omoteso, W. Ribarsky, T. Starner, and L. Hodges. "Speech and Gesture Multimodal Control of a Whole Earth 3D Visualization Environment." VisSym, Barcelona, Spain, May 2002.

K. Lyons and T. Starner, "Mobile Capture for Wearable Computer Usability Testing." In IEEE International Symposium on Wearable Computers, Zurich, Switzerland, October 2001.

B. Leibe, D. Minnen, J. Weeks, and T. Starner, "Integration of Wireless Gesture Tracking, Object Tracking, and 3D Reconstruction in the Perceptive Workbench." Intl. Conf. on Computer Vision Systems, Vancouver, Canada, July 2001, pp. 73-92.

T. Starner, J. Auxier, D. Ashbrook, and M. Gandy. "The Gesture Pendant: A Self-illuminating, Wearable, Infrared Computer Vision System for Home Automation Control and Medical Monitoring. " In IEEE International Symposium on Wearable Computers, Atlanta, GA, October 2000, pp. 87-94.

W. Rungsarityotin and T. Starner, "Finding Location Using Omnidirectional Video on a Wearable Computing Platform." In IEEE International Symposium on Wearable Computers, Atlanta, GA, October 2000, pp. 61-68.

G. Abowd, C. Atkeson, A. Bobick, I. Essa, B. MacIntyre, E. Mynatt, and T. Starner, "Living Laboratories: The Future Computing Environments Group at the Georgia Institute of Technology, " In ACM Proceedings of the 2000 Conference on Human Factors in Computing Systems (CHI 2000), The Hague, Netherlands, April 1-6, 2000.

K. Lyons, M. Gandy, and T. Starner. "Guided by Voices: An Audio Augmented Reality System." Intl. Conf. on Auditory Display (ICAD), Atlanta, GA, April 2000, pp. 57-62.

B. Leibe, T. Starner, W. Ribarsky, Z. Wartell, D. Krum, B. Singletary, and L. Hodges, "The Perceptive Workbench: Towards Spontaneous and Natural Interaction in Semi-Immersive Virtual Environments." IEEE Virtual Reality, New Brunswick, NJ, March 2000, pp. 13-20. Best Paper Award.

T. Starner, B. Leibe, B. Singletary, and J. Pair. "MIND-WARPING: Towards Creating a Compelling Collaborative Augmented Reality Game." In Proc. of ACM Intelligent User Interfaces, New Orleans, LA, January 2000, pp. 256-259.

Kidd. C., G. Abowd, C. Atkeson, I. Essa, B. MacIntyre, E. Mynatt, T. Starner, "The Aware Home: A Living Laboratory for Ubiquitous Computing Research", In Proceedings of Second InternationalWorkshop on Cooperative Buildings 1999, Editors, Streitz, J. Siegel, V. Hartkopf, S. Konomi, Pittsburgh. LNCS 1670. Springer: Heidelberg, 1999.

Starner, T., B. Schiele, B. Rhodes, T. Jebara, N. Oliver, J. Weaver, and A. Pentland. "Augmented Realities Integrating User and Physical Models." In IEEE International Workshop on Augmented Reality, San Francisco, CA, November 1998.

Starner, T., B. Schiele, and A. Pentland, "Visual Contextual Awareness in Wearable Computing." In IEEE International Symposium on Wearable Computers, Pittsburgh, PA, October 1998. pp. 50-57.

Starner, T. and Y. Maguire, "A Heat Dissipation Tutorial for Wearable Computing." In IEEE International Symposium on Wearable Computers, Pittsburgh, PA, October 1998.

Jebara, T., C. Eyster, J. Weaver, T. Starner, and A. Pentland, "Stochasticks: Augmenting the Billiards Experience with Probabilistic Vision and Wearable Computers." In IEEE International Symposium on Wearable Computers, Cambridge, MA, October 1997.

Starner, T., J. Weaver, and A. Pentland, "A Wearable Computing Based American Sign Language Recognizer." In IEEE International Symposium on Wearable Computers, Cambridge, MA, October 1997. pp. 140-149.

Starner, T. and A. Pentland. "Real-Time American Sign Language Recognition from Video Using Hidden Markov Models." In AAAI Fall Symposium on Disabilities. Cambridge, MA, November 1996.

Rhodes, B. and T. Starner. "Remembrance Agent: A continuously running automated information retrieval system." In Proc. of Pract. App. of Intelligent Agents and Multi-Agent Tech (PAAM), London, April, 1996.

Wren C., A. Azarbayejani, T. Darrell, M. Johnson, K. Russell, T. Starner, and A. Pentland. "Hierarchical Classification for Human Body Tracking." In Proc. SPIE Integration Issues in Large Commercial Media Delivery Systems, 2615, pp. 99-106, October, 1995.

Starner, T. and A. Pentland. "Real-Time American Sign Language Recognition from Video Using Hidden Markov Models." In IEEE International Symposium on Computer Vision, pp. 265-270, Coral Gables, FL, 1995.

Russell, K., T. Starner, and A. Pentland. "Unencumbered Virtual Environments." In IJCAI Workshop on Entertainment and AI/Alife, Montreal, Quebec, August 1995.

Starner, T. and A. Pentland. "Visual Recognition of American Sign Language Using Hidden Markov Models." In IEEE International Workshop on Automatic Face and Gesture Recognition, pp. 189-194, Zurich, Switzerland, 1995.

Wren, C., T. Darrell, T. Starner, M. Johnston, K. Russell, A. Azarbayejani, and A. Pentland. "Pfinder: A Real-Time System for Tracking People." In SPIE Conference on Real-Time Vision, M. Bove (editor), Philadelpia, PA, July 1995.

Pentland, A., B. Moghaddam, and T. Starner. "View-Based and Modular Eigenspaces for Face Recognition." In IEEE Proceedings of Computer Vision and Pattern Recognition Conference, pp. 84-91, Seattle, WA, July 1994.

Starner, T., J. Makhoul, R. Schwartz, and G. Chou. "On-line Cursive Handwriting Recognition Using Speech Recognition Methods." In IEEE Proceedings International Conference on Acoustics, Speech, and Signal Processing, pp. 125-128, April 1994.

Pentland, A., T. Starner, N. Etcoff, A. Masoiu, O. Oliyide, and M. Turk. "Experiments with Eigenfaces." In Proceedings of the Looking At PeopleWorkshop, IJCAI'93, pp. 1-6, Chamberry, France, August 1993.

Friedmann, M., T. Starner, and A. Pentland. "Device Synchronization Using an Optimal Linear Filter." In ACM Proceedings of the Symposium on Interactive 3D Graphics, Volume 26 (2), pp. 57-62, ACM Press, March 1992.

Pentland, A., M. Friedman, B. Horowitz, S. Sclaroff, and T. Starner. "The ThingWorld Modeling System," In Proceedings of International Workshop on Algorithms and Parallel VLSI Architectures, pp. 168-172, Pont-a-Mousson, France, June 1990.

Conference Presentations with Proceedings (refereed and invited paper)
[Publication (as a full paper, a short paper, or a technical note) in peer refereed conference/workshop proceedings or publication of an invited paper (as marked) in a conference/workshop proceedings.]

T. Starner. (invited paper) "Ubiquitous Mobile Interfaces." NIDRR Wireless RERC State of Technology conference. Atlanta, GA. May 2005.

K. Lyons and T. Starner. "Augmenting Cognition with Wearable Computers" Human Computer Interaction International (HCII), Las Vegas, NV, July 2005.

T. Westeyn, P. Pesti, K. Park, and T. Starner. (invited paper) "Biometric Identification using Song-Based Eye Blink Patterns." Human Computer Interaction International (HCII), Las Vegas, NV, July 2005.

D. Minnen, T. Starner, J. A. Ward, P. Lukowicz, G. Tröster. "Recognizing and Discovering Human Actions from On-body Sensor Data", ICME, Amsterdam, The Netherlands, July 2005.

T. Starner and D. Ashbrook, "Augmenting a pH Medical Study with Wearable Video for Treatment of GERD" In IEEE International Symposium on Wearable Computers, Washington, DC, October 2004.

K. Lyons, N. Patel, and T. Starner. "KeyMenu: A Keyboard Based Hierarchical Menu." In IEEE International Symposium on Wearable Computers, White Plains, NY, October2003, pp. 240-241.

T. Starner. (invited paper) "Wearable Computers as Intelligent Agents." Proc. of Interaction, Tokyo, Japan, March 2002.

B. Wong and T. Starner. (extended abstract reviewed) "Conversational Speech Recognition for Creating Intelligent Agents on Wearables." Human Computer Interaction International (HCII), New Orleans, LA, August 2001, pp. 823-827.

D. Ashbrook, J. Auxier, M. Gandy, and T. Starner. (extended abstract reviewed) "Experiments in Interaction Between Wearable and Environmental Infrastructure Using the Gesture Pendant." Human Computer Interaction International (HCII), New Orleans, LA, August 2001, pp. 818-822.

B. Singletary and T. Starner. (extended abstract reviewed) "Symbiotic interfaces for wearable face recognition." Human Computer Interaction International (HCII), New Orleans, LA, August 2001, pp. 813-817.

B. Singletary and T. Starner. "Learning Visual Models of Social Engagement." Intl. Work. on Recognition, Analysis, and Tracking of Faces and Gestures in Realtime Systems, Vancouver, Canada, July 2001, pp. 141-148.

T. Starner, B. Leibe, B. Singletary, K. Lyons, M. Gandy, and J. Pair. (invited paper) "Towards Augmented Reality Gaming." Proc. of IMAGINA, Monte Carlo, January 2000.

K. Lyons, L. Harvel, and T. Starner. "A Context-based Document System for Wearable Computers." In IEEE International Symposium onWearable Computers, Atlanta, GA, October 2000, pp. 187-188.

T. Starner, D. Kirsh, and S. Assefa. "The Locust Swarm: An Environmentally-powered, Networkless Location and Messaging System." In IEEE International Symposium on Wearable Computers, Cambridge, MA, October 1997, pp. 169-170.

A. Pentland, A. Pentland, M. Petrazzuoli, A. Gerega, and T. Starner. "The Digital Doctor: An Experiment in Wearable Telemedicine." In IEEE International Symposium on Wearable Computers, Cambridge, MA, October 1997, pp. 173-174.

Submitted Journal Papers

J. Ward, P. Lukowicz, G. Tröster, T. Starner. "Human Activity Recognition Using Body-Word Microphones and Accelerometers." Pattern Analysis and Machine Intelligence (PAMI), May 2005.

Submitted Conference Papers

K. Park, H. Brashear, S. Lee, V. Henderson, H. Hamilton, and T. Starner. "American Sign Language Recognition for Language Development of Deaf Children." I CCV Workshop on Analysis and Modeling of Faces and Gestures 2005.

Scientific Magazine Articles

D. Dagon, T. Martin, and T. Starner. "Mobile Phones as Computing Devices: The Viruses are Coming!" IEEE Pervasive Computing. 3(4), December 2004.

T. Starner. "Keyboards Redux: Enabling mobile e-mail and other services with fast mobile text entry." IEEE Pervasive Computing. 3(3), September 2004.

T. Starner. "Powerful Change Part 1: Batteries and Possible Alternatives for the Mobile Market." IEEE Pervasive Computing. 2(4), December 2003, pp.86-88.

M. Gandy, D. Ross, and T. Starner. "Universal Access and Wearable Aids." IEEE Pervasive Computing. 2(3), September 2003, pp. 19-23.

T. Starner. "The Enigmatic Display." IEEE Pervasive Computing, 2(1), January 2003, pp. 133-135.

T. Starner. "Attention, Memory, and Wearable Interfaces," IEEE Pervasive Computing, 1(4), October 2002, pp. 88-91.

T. Starner. "The Role of Speech Input in Wearable Computing." IEEE Pervasive Computing, 1(3), July 2002, pp. 89-93.

T. Starner. "Wearable Agents." IEEE Pervasive Computing, 1(2), April 2002, pp. 90-92.

T. Starner. "Wearable Computers: No Longer Science Fiction." IEEE Pervasive Computing, 1(1), January 2002, pp. 86-88.

T. Starner. "Thick Clients for Personal Wireless Devices." IEEE Computer, 35(1), January 2002, pp. 133-135.

T. Starner. "The Challenges of Wearable Computing: Part 2." IEEE Micro, 21(4), July 2001, pp. 54-67. (These articles were written to be part of a special issue on wearable computing but then split due to the length so as to be printed over two issues. However, an unexpected retraction in the second issue caused both parts to be printed in the same issue.)

T. Starner. "The Challenges of Wearable Computing: Part 1." IEEE Micro, 21(4), July 2001, pp. 44-52.

T. Starner, B. Leibe, B. Singletary, K. Lyons, M. Gandy, and J. Pair. (informal review) "Exploring Mobile and Desk-based Gaming Appliances." Sidebar in "Survey on Information Appliances." IEEE Computer Graphics and Applications (CGA), 20(3), May 2000, p. 22.

M. Billinghurst and T. Starner. "Wearable Devices: NewWays to Manage Information." IEEE Computer, 32(1), January 1999, pp. 57-64.