Anhong Guo wearing a light blue shirt standing in front of a glass wall with drawings.

I design, develop, and deploy hybrid human-AI intelligent interactive systems to provide access to visual information in the real world. By combining the advantages of humans and AI, these systems can be nearly as robust and flexible as humans, and nearly as quick and low-cost as automated AI, enabling us to solve problems that are currently impossible with either alone.

I am looking for great people to join my team HAIL (Human–AI Lab) at University of Michigan CSE!

Anhong Guo is an Assistant Professor in Computer Science & Engineering at the University of Michigan. Anhong completed his Ph.D. in the Human-Computer Interaction Institute, Carnegie Mellon University. He is also a Forbes' 30 Under 30 Scientist, a Google Research Scholar, an inaugural Snap Inc. Research Fellow, and a Swartz Innovation Fellow for Entrepreneurship. Anhong has published in many top academic conferences and journals on interface technologies, wearable computing, accessibility and computer vision. Before CMU, he received his Master’s in HCI from Georgia Tech, and Bachelor's in Electronic Information Engineering from BUPT. He has also worked in the Ability and Intelligent User Experiences groups in Microsoft Research, the HCI group of Snap Research, the Accessibility Engineering team at Google, and the Mobile Innovation Center of SAP America.

News

May 8, 2023
VizLens and ImageExplorer are released and publicly available on the iOS App Store!
Jan 13, 2023
Our study on DIY Assistive Technology Design by Blind People accepted to CHI 2023
Apr 21, 2022
Received a Google Research Scholar Award to make collaborative AR accessible
Mar 28, 2022
Project CollabAlly awarded Best Paper Honorable Mention at CHI 2022

Publications

BrushLens is a phone case with actuators that recognize screen elements, and use actuators to touch the screen on behalf of the user. It shows a user uses BrushLens to ``Brush'' on the touchscreen, and the actuator will touch if it is precisely on top of the button.
Chen Liang, Yasha Iravantchi, Thomas Krolikowski, Ruijie Geng, Alanson Sample, Anhong Guo
(To Appear) UIST 2023
pdf · ACM DL · video · 30s preview
AI-based assistive technologies tend to assume 'universal' needs of BVI people and are thus one-size-fits-all, rather than accounting for unique differences and desires. This figure shows an example where a blind user uses Seeing AI to read a letter and trying to find out about the sender name Rebecca. However, it'll read many other unnecessary and distracting details.
Jaylin Herskovitz, Andi Xu, Rahaf Alharbi, Anhong Guo
CHI 2023
pdf · ACM DL · full video · 30s preview · talk · dataset
An illustration of VRGit. A History Graph (HG) that represents non-linear version history is anchored on the user’s left arm, where each node is a 3D miniature of that version. Inside each miniature, objects are highlighted using color coding if they are changed compared to the previous version. Mini avatars are anchored in the HG to represent which version users are in. Users can also create portals to monitor other users’ first-person views. A shared history visualization facilitates group discussion by anchoring the HG on a surface and allowing users to preview a version and reuse objects collaboratively.
Lei Zhang, Ashutosh Agrawal, Steve Oney, Anhong Guo
CHI 2023
pdf · ACM DL · full video · 30s preview · talk
This figure illustrates the scenario of a human support robot designed to perform tasks for the elderly or those with mobility impairments. The human is looking for his book, The Future of Ideas, which is the top book in the stack located on his desk, next to a computer monitor. However, 'The Future of Ideas' may be ambiguious, as the phrase is also displayed on the computer monitor. Using deferred inference, the robot may ask the human to reframe their query, 'Top Book', at which point the robot is confident what the user is referring to.
Stephan J. Lemmer, Anhong Guo, Jason J. Corso
IUI 2023
pdf · ACM DL · code
As an overview of XSpace's components, this is a sketch of a person using an AR application in their living room. There is a virtual avatar on a virtual chair, showing the mesh crop and overlay method. There is shared virtual content around the space.
Jaylin Herskovitz, Yi Fei Cheng, Anhong Guo, Alanson Sample, Michael Nebeling
ISS 2022
pdf · ACM DL · video · talk · code
OmniScribe to make 360° videos accessible. A describer is using the OmniScribe web authoring interface, which supports them to better understand the 360° content to author standard audio descriptions and create immersive labels, in order to enable BVI people to interact with 360° content immersively using smartphones and headphones. The picture shows a man wearing wireless headphones, holding a mouse in his right hand to operate a computer and use our OmniScribe for audio description. On the computer is the OmniScribe interface.
Ruei-Che Chang, Chao-Hsien Ting, Chia-Sheng Hung, Wan-Chen Lee, Liang-Jin Chen, Yu-Tzu Chao, Bing-Yu Chen, Anhong Guo
UIST 2022
Screenshot of the four step CustomizAR process: Step 1: Select Object, Step 2: Select Design, Step 3: Adjust & Measure, and Step 4: Finished. Currently showing Step 3: Adjust & Measure, where the user roughly indicates the measurement location. The selected design of the bicycle water bottle cage with zip-tie attachment is displayed on the top left corner. The measurement guide is in the center of the screen. On the bottom, it's showing that bottleDiameter is the parameter that's currently being measured, as well as controls and instructions for taking a photo.
Chen Liang, Anhong Guo, Jeeeun Kim
DIS 2022
pdf · ACM DL · 30s preview · talk
CollabAlly system overview: (i) Extracts visual cue information from Google Doc's HTML DOM tree, including collaborators, cursors, comments, highlighted text, and text differences; (ii) Parses this data into a readable format and display it in three pop-up dialog boxes on-demand: (a) enhanced collaborator announcements, (b) comment tracking and navigation, (c) real-time and asynchronous text changes; (iii) Conveys contextual information using audio features, including spatial audio and voice fonts. The figure shows a Google doc page and 3 interfaces of CollabAlly dialog boxes. The annotations also include different visual cues (collaborator, cursor, comment, highlighted text, and text changes) and audio features (spatial audio and voice fonts).
Cheuk Yin Phipson Lee*, Zhuohao Zhang*, Jaylin Herskovitz, JooYoung Seo, Anhong Guo
Honorable MentionCHI 2022
pdf · ACM DL · full video · 30s preview · talk · code
Left: An image of a bed with a white outline around it. A speech bubble reads: Bed. Double tap to explore. Right: The same image of the bed, now with a variety of rectangular bounding boxes around the white pillow and blue sheets. Two speech bubbles read: A white pillow and The bed is blue.
Jaewook Lee, Jaylin Herskovitz, Yi-Hao Peng, Anhong Guo
CHI 2022
pdf · ACM DL · full video · 30s preview · talk
TutorialLens records the finger location relative to device marker location in 3D for authoring mode, and then reproduces that based on detected device marker location in the access mode (shown in picture).
Junhan Kong, Dena Sabha, Jeffrey P. Bigham, Amy Pavel, Anhong Guo
SUI 2021
pdf · ACM DL · supplemental · video
A decision tree to predict the instances on which the Face API gender classifier is likely to make a mistake based on meta-attributes of the data. Each node of the tree is labeled by the counts of correct and wrong instances belonging to the clusters. Nodes are colored to represent the relative error rate, green shades for lower error rates and red shades for higher error rates.
Solon Barocas, Anhong Guo, Ece Kamar, Jacquelyn Krones, Meredith Ringel Morris, Jennifer Wortman Vaughan, W. Duncan Wadsworth, Hanna Wallach
AIES 2021
pdf · ACM DL · arXiv · poster · slides · talk
Leila, a black, non-binary person with a filtering face mask walks down a neighborhood street with one hand in their pocket and the other hand on their cane. They have a short mohawk and are wearing a jacket, shorts, tennis shoes, and glasses.
Cynthia L. Bennett, Cole Gleason, Morgan Klaus Scheuerman, Jeffrey P. Bigham, Anhong Guo, Alexandra To
Honorable MentionCHI 2021
pdf · ACM DL · slides · talk
Image: "Walking in neighborhood with face mask", by Disabled And Here, licensed under CC BY 4.0
Two example prototypes for making AR apps accessible. A: Foundational Accessibility. Screenshot of a virtual chair with a voice over target around it, a speech bubble shows the app announcing "Back of chair with blue cushion". B: Scanning. Screenshot of AR grid overlaid on a coffee table. Speech bubbles show the app announcing "Found a new horizontal surface" and "Scanned 2 surfaces totaling 2.3 square meters".
Jaylin Herskovitz, Jason Wu, Samuel White, Amy Pavel, Gabriel Reyes, Anhong Guo, Jeffrey P. Bigham
ASSETS 2020
pdf · ACM DL · video · slides · talk
Image of a person in a wheelchair in front of a swing gate
Shaun K. Kane, Anhong Guo, Meredith Ringel Morris
Best Paper NomineeASSETS 2020
pdf · ACM DL · slides · talk
Image: "Swing Gate - SL 931 - Belgium", by Automatic Systems, licensed under CC BY 2.0
Screenshot of a tweet by @CDCgov from April 1, 2020 3:55pm: Actions to reduce spread of the virus, such as social distancing, are key to #FlattenTheCurve. 2 of 3 (original tweet link: https://twitter.com/CDCgov/status/1245439600472084486) The tweet contains an image of the common public health infographic about “flattening the curve”, but the tweet did not include alt text for the image. The image shows an example of a common flatten the curve info-graphic. A tall peak indicates the height of the pandemic if left unchecked, and a shorter spread out curve depicts the effects of social distancing efforts.
Cole Gleason*, Stephanie Valencia*, Lynn Kirabo, Jason Wu, Anhong Guo, Elizabeth J. Carter, Jeffrey P. Bigham, Cynthia L. Bennett+ Amy Pavel+
ASSETS 2020
pdf · ACM DL · supplemental · slides · talk
The user is holding the phone in landscape mode with one hand, and aiming the camera towards a touchscreen coffee machine. The user’s other hand is wearing a fingercap exploring on the screen. The StateLens iOS app is providing audio guidance to the user.
Anhong Guo, Junhan Kong, Michael Rivera, Frank F. Xu, Jeffrey P. Bigham
UIST 2019
pdf · ACM DL · arXiv · full video · 30s preview · slides · talk · UIST live talk
Two Blocks users are collaboratively creating a table in augmented reality.
Anhong Guo, Ilter Canberk, Hannah Murphy, Andrés Monroy-Hernández, Rajan Vaish
Ubicomp 2019
pdf · ACM DL · arXiv · video
On the left: a screenshot of Android App Drawer taken using X-Ray. On the right: a user holding the phone showing the same screenshot in X-Ray image viewer. Talk back cursor is visible.
Sujeath Pareddy, Anhong Guo, Jeffrey P. Bigham
Best Artifact AwardASSETS 2019
pdf · ACM DL · video · code
A word cloud composed of words including AI fairness, accessibility, artifical intelligence, inclusion, and bias.
Anhong Guo, Ece Kamar, Jennifer Wortman Vaughan, Hanna Wallach, Meredith Ringel Morris
ASSETS 2019 AI Fairness Workshop
Three images in the VizWiz-Priv dataset, including an image of a wall of photos containing faces, an image of a credit card, and an image of a pregnancy test. The private information regions in the images are highlighted and inpainted.
Danna Gurari, Qing Li, Chi Lin, Yinan Zhao, Anhong Guo, Abigale Stangl, Jeffrey P. Bigham
CVPR 2019
pdf · supplemental · CVF · vizwiz.org · poster
Interaction scenario of Minuet: after returning home, the user points at the Roomba and then the dirty area to ask Roomba to clean it up.
Runchang Kang, Anhong Guo, Gierad Laput, Yang Li, Xiang 'Anthony' Chen
SUI 2019
pdf · ACM DL · video
An example question sensor created in Zensors++ asking 'Is someone using a printer?' with a bounding box focusing on the printer area.
Anhong Guo, Anuraag Jain, Shomiron Ghose, Gierad Laput, Chris Harrison, Jeffrey P. Bigham
Ubicomp 2018
pdf · ACM DL · video · slides
A table with many boxes covered with white paper showing text such as glasses, butter, jam, etc. A user is holding and targeting his phone at one object, while touching the object with the other hand. This is showcasing the window cursor interaction technique that supports non-visual attention to items within a complex visual scene, in which the user moves the device itself to scan the scene and receives information about what is in the center of the image.
Anhong Guo, Saige McVea, Xu Wang, Patrick Clary, Ken Goldman, Yang Li, Yu Zhong, Jeffrey P. Bigham
ASSETS 2018
pdf · ACM DL · video · slides
Example printed overlays and legends generated by Facade. (a)-(d) demonstrate the different material combinations we tested in the design iterations (NinjaFlex with Braille, Flex+PLA Braille label, Flex+PLA Braille cover, and Flex+PLA embossed letter cover). Facade users can choose to print a legend for the abbreviations (e).
Anhong Guo, Jeffrey P. Bigham
IEEE Pervasive Computing 17(2), 2018, Maker Tech column
Distribution of the first six words for all questions in the VizWiz dataset. The innermost ring represents the first word and each subsequent ring represents a subsequent word. The arc size is proportional to the number of questions with that word/phrase.
Danna Gurari, Qing Li, Abigale J. Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, Jeffrey P. Bigham
Spotlight PresentationCVPR 2018
pdf · arXiv · website · poster · video · tech review
A user accessing the microwave augmented with tactile overlays generated by Facade.
Anhong Guo, Jeeeun Kim, Xiang 'Anthony' Chen, Tom Yeh, Scott E. Hudson, Jennifer Mankoff, Jeffrey P. Bigham
CHI 2017
pdf · ACM DL · full video · 30s preview · talk
A user holding a 3D-printed cup holder augmented with a flexible ring generated using our flexible buffers technique.
Jeeeun Kim, Anhong Guo, Tom Yeh, Scott E. Hudson, Jennifer Mankoff
DIS 2017
pdf · ACM DL · video
The user is holding the phone in portrait mode with one hand, and aiming the camera towards an inaccessible microwave control panel. The user’s other hand is exploring on the panel. The VizLens iOS app is providing audio feedback and guidance to the user.
Anhong Guo, Xiang 'Anthony' Chen, Haoran Qi, Samuel White, Suman Ghosh, Chieko Asakawa, Jeffrey P. Bigham
UIST 2016
pdf · ACM DL · full video · 30s preview · talk
Two tilt-based interaction techniques for enabling no-touch, wrist-only interactions on smartwatches. Left: AnglePoint, which directly maps the position of a virtual pointer to the tilt angle of the smartwatch. Right: ObjectPoint, which objectifies the underlying virtual pointer as an object imbued with a physics model.
Anhong Guo, Tim Paek
Honorable MentionMobileHCI 2016
pdf · ACM DL · video
A user working on a document on a smartwatch using WearWrite, by leveraging a crowd to help trans­late their ideas into text.
Michael Nebeling, Alexandra To, Anhong Guo, Adrian A. de Freitas, Jaime Teevan, Steven P. Dow, Jeffrey P. Bigham
CHI 2016
pdf · ACM DL · full video · 30s preview · talk
A system architecture diagram of an order picking system augmented with weight checking error detection.
Xiaolong Wu, Malcolm Haynes, Anhong Guo, Thad Starner
ISWC 2016
pdf · ACM DL · video
Four BeyondTouch interaction techniques, including tapping on a phone in the pocket, tapping on the back of a phone while holding it with two hands, tapping and sliding on the back of the phone while holding it with one hand, as well as tapping and sliding next to the phone on the table to control the device.
Cheng Zhang, Anhong Guo, Dingtian Zhang, Yang Li, Caleb Southern, Rosa I. Arriaga, Gregory D. Abowd
TiiS 6(2), 2016
pdf · ACM DL
Pick-by-head-up display system using a Google Glass with a opaque display to show the pick order instructions.
Anhong Guo, Xiaolong Wu, Zhengyang Shen, Thad Starner, Hannes Baumann, Scott Gilliland
Computer 48(6), 2015
Four BeyondTouch interaction techniques, including tapping on a phone in the pocket, tapping on the back of a phone while holding it with two hands, tapping and sliding on the back of the phone while holding it with one hand, as well as tapping and sliding next to the phone on the table to control the device.
Cheng Zhang, Anhong Guo, Dingtian Zhang, Caleb Southern, Rosa I. Arriaga, Gregory D. Abowd
IUI 2015
pdf · ACM DL · video
Image of a set of order picking bins with LED displays and buttons.
Xiaolong Wu, Malcolm Haynes, Yixin Zhang, Ziyi Jiang, Zhengyang Shen, Anhong Guo, Thad Starner, Scott Gilliland
ISWC 2015
pdf · ACM DL
Image of the experimental setup, including 24 pick bins (on two shelving units with four rows and three columns each) and three order bins on the right. An example pick list is annotated with superimposed labels.
Anhong Guo, Shashank Raghu, Xuwen Xie, Saad Ismail, Xiaohui Luo, Joseph Simoneau, Scott Gilliland, Hannes Baumann, Caleb Southern, Thad Starner
Honorable MentionISWC 2014
pdf · ACM DL · video · talk
Assistant Professor
Computer Science & Engineering
College of Engineering
University of Michigan
Curriculum Vitae
Google Scholar
Email: anhong@umich.edu
Twitter: @AnhongGuo
YouTube Channel

Apps Released

App icon of VizLens
VizLens
App icon of ImageExplorer
ImageExplorer

Attending

9/18-19, 2023
CRA LEVEL UP
10/17, 2023
AI Symposium
10/22-25, 2023
ASSETS
10/29-11/1, 2023
UIST

People

Ph.D. students (mentoring plan)
Ruei-Che Chang
Ruei-Che Chang
Gregory Croisdale
Gregory Croisdale
Jaylin Herskovitz
Jaylin Herskovitz
Chen Liang
Chen Liang
Lei Zhang
Lei Zhang
Master's and Undergrad students
Minyu Cai
Minyu Cai
Jacob Gettig
Jacob Gettig
Dier Hou
Dier Hou
Yuxuan Liu
Yuxuan Liu
Tao Lu
Tao Lu
Jin Pan
Jin Pan
Haocheng Ren
Haocheng Ren
Linfeng Oscar Song
Linfeng (Oscar) Song
Muzhe Wu
Muzhe Wu
Andi Xu
Andi Xu

Application Materials