AI design
Designs how people use, understand and correct products powered by artificial intelligence, whose answers can vary and be wrong.
What it deals with
An AI system does not always answer the same way or get it right, and it is hard to know in advance what it will be able to do. You have to design for error, doubt and trust that matches how often the system is right.
Principles
Say what it can do
Also explain how reliably it does it, from the first use.
Design the error
Whatever the system suggests can be corrected, undone or ignored.
Keep people in control
The person decides and can tell the system what went wrong.
Applications in design
Use it like this
Test the idea with real outputs from the system, including the bad ones, not just the ideal ones.
Avoid
Promising more than the system does well.
Don’t fall for it
A system that does well in testing won’t necessarily do well with real data: keep measuring after launch.
Models
- ModelGuidelines for Human-AI InteractionEighteen guidelines for designing AI products, ordered by the moment of use.
Methods and tools
- ApproachValue sensitive designTaking into account the values of everyone affected throughout the design of a technology.
- MethodWizard of OzTesting a system that doesn't exist yet, with a hidden person acting as the machine.
- ApproachPrivacy by designProtecting personal data from the first sketch, with the most private option already set.
Connections
Authors and works
- 2019
Saleema Amershi et al.
Proposed 18 guidelines for human-AI interaction and tested them with 49 practitioners on 20 products.
- 2019
Google PAIR
Published a guide for teams: user needs, data, mental models, explanations and trust, control and errors.
- 2020
Qian Yang, Aaron Steinfeld, Carolyn Rosé and John Zimmerman
Explained why designing with AI is so hard: it is unclear what it will be able to do, and its outputs can be very complex.
- 2024
European Union, AI Act
Since 2 August 2026, its Article 50 requires telling people when they are talking to an AI system, unless it is obvious, and marking the audio, images, video or text it generates in a machine-readable way.
- 2026
European Union, Digital Omnibus on AI
Gave generative AI systems already on the market before 2 August 2026 until 2 December 2026 to comply with the marking requirement.
Sources
Original sources
- Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan, J., Kikin-Gil, R., & Horvitz, E. (2019). Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–13). ACM. View source: Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan, J., Kikin-Gil, R., & Horvitz, E. (2019)
- Google PAIR (2019). People + AI Guidebook. Google. View source: Google PAIR (2019)
- Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020). Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM. View source: Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020)
- Reglamento (UE) 2024/1689 del Parlamento Europeo y del Consejo, de June 13, 2024, por el que se establecen normas armonizadas en materia de inteligencia artificial (Reglamento de Inteligencia Artificial). Diario Oficial de la Unión Europea, July 12, 2024. View source: Reglamento (UE) 2024/1689 del Parlamento Europeo y del Consejo, de June 13, 2024, por el que se establecen normas armonizadas en materia de inteligencia artificial (Reglamento de Inteligencia Artificial)
- Reglamento (UE) 2026/1744 del Parlamento Europeo y del Consejo, de July 8, 2026, que modifica el Reglamento (UE) 2024/1689 (Ómnibus digital sobre IA). Diario Oficial de la Unión Europea, July 24, 2026.
Last reviewed: