Tangible Co-Ideation
Prompting an AI with your hands instead of your keyboard. Published at DIS’26
DIS ‘26 Student Design Competition | MIT Media Lab | Co-first authors (equal contribution): Yuqing Lucy Li, Quincy Kuang, Paul-Peter Arslan, Yuhan Wang
My role: concept, hardware, and capacitive sensing, AI Mentor pipeline
Chat is a narrow doorway into AI.
Today, nearly every conversation with an AI happens through a text box that favors incremental refinement over exploratory thinking. We introduce Tangible Co-Ideation, a system that supports AI-assisted ideation by manipulating physical objects representing different ideation actions, allowing people to navigate and shape a space of emerging ideas.
So We Gave Prompts A Body
Ideation Personas
We created Ideation Personas: a multimodal RAG knowledge base built from image and text training data, allowing the agent to generate both visual and textual idea outputs based on selected datasets, e.g., publications, books, and interviews of experts from specific perspectives.
By placing an Ideation Persona onto an Idea room, the agent will generate 3 ideas from a persona’s perspective.
You can also ask custom questions e.g., create a sketch of an idea.
Or engage in debate mode, where two ideation personas can be placed face-to-face to start a discussion and allow users to engage in multiple perspective thinking.
Lighthouse
Lighthouse situates new ideas. It guides users by showing related work and shining light on unknown areas. We created a pipeline where an LLM identifies three clusters of grounding work: main themes, adjacent contexts, and technical foundations. Each cluster is then populated with at least five papers retrieved through the Semantic Scholar, arXiv, and Crossref APIs.
A set of tangible vocabulary externalizes diverse choices during the ideation process and enables users to generate alternatives, redirect exploration, combine ideas, and eliminate unpromising directions.
Making it work took two kinds of iteration.
- Sensing. An overhead camera needed each object’s top visible, which got in the way of natural handling. We switched to 3D-printed objects with three conductive points on the base, so any standard touch screen recognizes them by their footprint.
- Integration. Software built without the objects in mind, and objects built without the software, broke the experience. Physical objects need a clear link to the screen, like a digital shadow, and legs so they look like they’re stepping out of it.
Canvas
The interface begins as a blank canvas. Users initiate a brainstorming session by drawing a circle, a "room" that represents a discussion space. Inside the room, users sketch and describe the design problem they wish to solve. Ideas are represented as bubbles that can either be AI-generated or manually added by the user. Every idea bubble remains connected to its originating room through visible branches, preserving the complete derivation history. In this way, users can continually refer to the original problem as they iterate on ideas.
Users develop ideas through four ideation vocabularies: Branch, which generates alternative ideas; Step, which extends an idea into a chain of developments; Merge, which combines two ideas into one; and Prune, which removes ideas from the workspace. Together, these interactions form the foundation for organizing, comparing, and developing both human- and AI-generated ideas.
Why it matters.
Chat is not a neutral interface. It favors people comfortable creating through language, in response to a continuous stream of input. We set out to examine a different one: Thinking through tangibility and spatial arrangement, and what such an interface contributes to collaborative, multi-agent brainstorming. We found that it reshapes the relationship between a pair and the model. Rather than passively receiving generated content, pairs using physical objects became more active participants: they spoke more, took more turns, produced more of their own ideas, and settled into a burstier rhythm of ideation, in contrast to the flatter cadence of the screen-only condition. Lastly, we believe that the choice of interaction vocabulary determines which parts of thinking are delegated to LLMs and which are retained by the human - question in today’s AI landscape is more important than ever.
