How can we use HoloLens to solve some of our key training challenges for both engineers and pilots, enabling more scalable training programmes?
HoloLens is Microsoft's vision of a 3D future where, no longer limited by the physical constraints of a 2D screen, users can at last explore the magic of the digital world within the real, a mixed reality. To help companies see the full potential of HoloLens, Microsoft embarked upon an innovation lab programme that helped companies envision the possibilities and then realise them as proofs of concepts.
Japan Airlines approached Microsoft with a vision for a more scaleable training programme that would help them train both pilots and engineers and adapt their training programmes more quickly and cost effectively than existing methods. As a proof of concept, Japan Airlines asked Microsoft to show how HoloLens might train pilots to set up the flight deck and engineers, the parts of an engine.
The end demo was a success, a happy client and a great presentation by our team and Satya Nadella at WPC 2016.
We wanted to create an experience for the engine tutorial that would help break down the parts of the system that helped them in a fun and interactive manner whilst still maintaining functionality and usability. To do this I began a series of sketches looking at how the user would 'travel' immersively through the different parts of this complex structure, and use a series of hotspots to reveal information about key parts of the engine.
I was interested in using floating menus along with an x-ray reveals of the engine to show the different layers.
Taking this further, I developed a set of UI to help the user navigate through the experience. Designing for HoloLens presents an extra challenge, requiring designers to think first about the colours they use for UI. Being a projector based device, dark colours simply don't show up or become very transparent. A further constraint was consideration for the narrow view port. The narrow field of view meant that small moving indicator icons had to be used to hint to the user to look in a certain direction.
During exploration of the UI styling, I made a concept test using an image of a stripped back engine to explore how a UI might layer over.
I explored animating the engine in as an 'old school' television set to add a sense of fun, and how the engine might show an element of transparency to hint to the possibility of exploring the layers beneath. On clicking a hotspot about airflow, the user would then see air flow animating around and through the engine. Ultimately, I was keen to have a grounding point with a 'shadow' giving the user a frame of reference for the position of the engine in respect to the real environment, and also a focal point for global UI or menu buttons not specifically related to the engine.
The flight deck experience was designed as a lighter exploration of how HoloLens could help trainees. We focused specifically around the set up for flight, guiding the user through a voice UI as they 'flicked' the switches, with audio feedback.
As before with the engine, a key part to exploring the potential of the UI was prototpying it with an image, in this case a flight deck, and layering over exploratory UI to see how it might work.
The end demo was a success, a happy client and a great presentation by our team and Satya Nadella, showcasing our efforts.
Presented at Microsoft's Worldwide Partner Conference, 2016