Drawing Intelligence

by James A. Landay PhD at Carnegie Mellon University (CMU)

SILK is an interface sketching system where you can draw UI elements like buttons or sliders by hand, and the system recognizes them and makes them interactive. It helps designers stay in the sketching phase longer before moving to programming the UI elements.

by Karl Toby Rosenberg

DrawTalking is an interface where sketches, speech, and text labels are correlated with one another. It learns to associate sketches with objects and verbs with specific movements, allowing users to animate scenes using spoken commands.

Stable Diffusion with ControlNet allows you to generate images by providing sketches. The model identifies the lines in the input image, understands the structure you’ve drawn, and generates detailed, styled images based on it. It demonstrates how AI can respond meaningfully to sketch input.

by a team of researchers from KAIST (Korea Advanced Institute of Science and Technology)

RoboSketch is an interactive tool that lets you draw directly in 3D space along different planes. The sketch creates a kind of skeletal framework, which can then be animated to explore motion and form.

by Anthropic

Claude's AI agent chatbot is connected to a canvas to recieve drawing input as a part of the prompt. It assists with coding user interfaces and interaction design.

by McNeel

allows designers to use a stylus to sketch and annotate directly over Rhino viewports.