Interactive Image Description: In Digital/Audio Books for Individuals with Visual Impairments

Existing image descriptions often lack context or detail and fail to meet the diverse needs of people with visual impairment. Preferences vary widely based on factors such as the type and onset of visual impairment, age, and personal interests. A one-size-fits-all approach to image descriptions is inadequate, highlighting the need for customizable, context-aware solutions. This research shows the approach, insights and suggestions for future works that might help in making images in books accessible, interactive and personalised for people with visual impairment. This was done as a part of internship with Koninklijke Bibliotheek. The project was supervised by Ted van der Togt (Researcher, Koninklijke Bibliotheek, The Hague) and Jeff Love (Assistant Professor and Data Steward, Industrial Design Engineering, TU Delft) during MSc Design for Interaction at TUDelft, Netherlands.

In this blogpost you can read the executive summary of the project. The full report can be found on Zenodo as audiobook, PDF and epub!

Prasun, P. (2025). Interactive Image Description: In Digital/Audio Books for Individuals with Visual Impairments. Zenodo. https://doi.org/10.5281/zenodo.14796675

Executive summary

Individuals with visual impairments face significant challenges accessing visual information in books, particularly digital formats like e-books and audiobooks. Visual details are essential for understanding narratives and contexts, yet their accessibility remains insufficient. As mentioned in the article “The Accessible Books Consortium: what it means for publishers” by Catherline Jewell in WIPO magazine in 2018, globally, less than 10% of published works are available in accessible formats, leaving a significant gap in inclusive for people with visual impairments. This project investigates how artificial intelligence (AI) and interactive technologies can bridge this gap.

Research Problem

Existing image descriptions often lack context or detail and fail to meet the diverse needs of users. Preferences vary widely based on factors such as the type and onset of visual impairment, age, and personal interests. A one-size-fits-all approach to image descriptions is inadequate, highlighting the need for customizable, context-aware solutions.

Methodological Approach

A mixed-methods approach was employed, including desk research, user interviews, surveys, generative co-creation sessions, and prototype testing. A survey of twelve participants revealed differences in user preferences based on their age and the onset of their visual impairment. Younger users preferred detailed, customizable descriptions, while older users were more neutral about personalization. Co-creation sessions further emphasized the importance of integrating descriptions seamlessly into the reading experience.

Proposed Solutions

The research explored AI-driven, interactive solutions to enhance image accessibility without disrupting the reading flow. Key proposals included: 

• Haptic Feedback: Subtle vibrations alert users to an image, with optional prompts for additional details.
• Voice-Activated Interaction: Users can ask specific questions about an image, such as “Describe the setting” or “What are the people wearing?”
• AI-Driven Personalization: Customizable descriptions tailored to user preferences and book context, adjustable in depth and length.

Findings and Key Insights

Prototype testing highlighted a strong user preference for simple and flexible solutions. Participants valued the ability to control the level of detail in image descriptions, allowing uninterrupted reading. Privacy concerns associated with AI use emerged as a notable issue. The usability assessment, using the System Usability Scale (SUS), yielded an average score of 80, indicating excellent usability. However, the small sample size underscores the need for further validation. 

Future Directions 

This research identifies the potential for context-aware, interactive descriptions, emphasizing simplicity, flexibility, and privacy. Future work could explore integration into e-readers, mobile apps, or everyday portable devices. Key development areas include customizable image descriptions and the incorporation of context-aware AI functionalities. 

Conclusion 

This study highlights the critical need to improve the accessibility of visual content in books for individuals with visual impairments. Leveraging AI, haptic technologies, and user-centered design principles can create a more inclusive and engaging reading experience. The research lays a good foundation for further exploration and broader validation.