Earbuds That See

Each earbud has a rice-grain-sized camera that captures low-resolution images and sends them to a phone or other nearby device. There, an AI model verbally answers questions about the scene in about one second (watch in action). The prototype—called VueBuds—achieved over 80% accuracy in identifying objects and 93% accuracy in identifying book titles and authors. Due to camera size constraints, the images are in grayscale, so the system cannot yet answer color-related questions. Future studies will explore how VueBuds could help people with low vision read books and assist travelers with translations.

The project team hopes VueBuds will be adopted more widely than smart glasses and VR headsets, since people already wear earbuds.