Paint 3D Multimodal

The Brief

How can we allow users to interact and create in Paint 3D in a smart and natural way?

Introduced in 1985 with the advent of the earliest GUIs, Paint, everyone's first love with digital art and design, demanded an update that didn't just do it justice but also meant it could again become everyone's introduction to a new digital frontier. This time, Paint is paving the way for a 3D future by rethinking previously complex 3D software and bringing it to everyone. It's 3D for everyone.

When creating, we very rarely restrict ourselves to just a single hand. So why restrict ourselves digitally when the opportunities are endless? I set up user research tests to understand the gestures people use when interacting with real pen and paper. Using this as the springboard for digital interactions for pen and touch, and extending this into the Surface Dial. As part of this, I began looking at how the ruler might beat the real world equivalent.

Back to Basics

I set up a camera and invited users to come along to draw a set task. The instructions were deliberately vague to ensure they created as naturally as possible. What was super interesting was how much users manipulated a piece of paper with their hands as they drew, twisting and turning it, at some points even lifting it up to take a closer look.



I explored a series of designs that looked at using pen and touch to help manipulate a 3D object as they painted, attempting to take it as close to the real world as possible. As part of this, I looked at how the UI and canvas might be controlled by the less dominant hand to adapt the marks being made by the pen in the dominant hand.



MultiModal Video Concept

I progressed onto on-screen tools such as the ruler and how the ruler, more than just being an aid for straight lines, might double up as a zoom layer guide and even throug hteh use of smart interactions might help the user to create more flexibly.

Surface Dial

With Surface Dial I was concerned about making the most out of a great tool that had a whole new menu set that allowed even greater flexibility, particularly on larger screens such as the Surface Studio. This was a tricky task given that the Dial is effectively a one dimensional, linear tool.


Multi Modal Video Concept

Windows 10 Insider Announcement

Part of my designs were also implemented elsewhere throughout Windows 10 Creators Update, specifically using the ruler.