How can we let kids and novices explore 3D and have the most fun creating in Paint?
Introduced in 1985 with the advent of the earliest GUIs, Paint, everyone's first love with digital art and design, demanded an update that didn't just do it justice but also meant it could again become everyone's introduction to a new digital frontier. This time, Paint is paving the way for a 3D future by rethinking previously complex 3D software and bringing it to everyone. It's 3D for everyone.
The challenge of representing 3D on a 2D screen is enormous. Add to this, letting complete beginners create in 3D and the challenge can suddenly evolve into a riddle about as challenging as fitting an elephant through a keyhole. I worked on several key features through Paint3D working as part of a large team of incredible talent within the Windows division.
Communicating 3D in a friendly and approachable manner, requires creating a whole new language that bypasses much of the 3D terminology found in more professional 3D software packages. A problem not just restricted to Paint3D but, as a forward looking operating system defining itself as the platform for 3D creation and consumption, this was a challenge across the whole of Windows. To ease the challenge, I looked towards the real world to understand how people interacted with and spoke about 3D everyday.
To accomplish this, a colleague and I began a series of simple tests asking users to create an object using PlayDoh. Setting up a camera, I asked users to communicate their process as they created their model defined by the pictoral instructions we'd given them. Through this, we recorded the language they used to describe 3D in the real world. As the tests progressed, I decided to do a control, for further exploration where I asked users to create models using a different material, in this case, Lego to see how the language was used.
'Interactions inform the language. Therefore defining these interactions specifically with language / cues / clues / signifiers in mind may be a way to increase the usability and understandability of the system / product / software’.
- Rateau & Grisoni, Mimetic Interaction Spaces
The research indicated that the language chosen to describe actions was directly related to the context and the material being used. In the study, if participants were using PlayDoh, their words generally tended towards softer words more commonly associated with organic forms 'meld, merge, cut, pull, squash'. With Lego, the words became harder and more angular: 'click, snap, flip, plonk'. Paint and the Windows group, therefore, needed to define precisely what their 3D material was to be able to communicate tangibly and simply to mass users new to 3D.
Selection in 3D on a 2D screen is a tricky challenge to communicate simply, especially for users new to 3D. Add to this, with Paint's old selection style now needing an update, a whole rethink around selection was now required. The new selection model had to allow for complex tasks such as applying textures or 'stickers' to 3D objects, and elegantly 'magic selecting' or 'grab cutting' specific objects from their surroundings (see second image below for detailed user flow).
Any selection method had to remain consistent across 2D and 3D. To achieve this, I conducted an exploration of selection methods looking closely at how simple interactions such as mini animations and subtle visual cues could help, and as ever, always attempting to refer back to the real world to bridge the knowledge gap.
With Paint, we wanted to make sure that we made use of the new Universal Windows Platform system and adopted the standard behaviours to ensure cross platform compatibility. This presented a problem for Paint 3D, as traditionally, on attempting to close Paint after having made changes made to an image, a save dialog would appear giving the option to either 'save', 'save as' or 'cancel'. With the UWP architecture, there were no save dialogs.
I began a series of explorations into how the problems with auto saving might be alleivated using a combination of concepts such as 'paint project files' and simpler interim solutions such as internal app dialogs based on time appearing warning the user of rewriting over the project.
What began as a separate viewer project that looked into how a user might open a 3D object without any specialist software using Windows, out of the box, developed into a project that helped define the motion and interaction paradigms for 3D in Paint as a whole.
I looked at various real world motions and movements and adapted those to inform the behaviour on screen, keeping the focus of movement on the camera rather than the object, with strong limits to the world movement.
I wanted the interactions and motions to be as natural and fun as possible, with movement such as 'shaking' on long press to give a 3D feel through a 2D screen for movement of objects. I also explored the use of accelerometers to allow the user to 'peep over' the object as the camera titled with the motion of the tablet or device. What was crucial, however, limiting the scope of movement to certain axis points to prevent the disorientation, a common problem with 3D software for novice users.