Since Xerox’s PARC and Apple’s Lisa user interfaces hit the scene, the WIMP (windows-icons-menus-pointers) interaction model has become a universal standard. This past Hackweek, fellow designer and Chartcorps member Sam Chieng, front-end developer Aliya Robinson, and I decided to dive into UI’s history and potential future.

Here’s where our week’s investigation took us:

Sam

I was particularly intrigued by how we’ve arrived at the certain interfaces that we’re so accustomed to today. How did we decide on specific UI elements as “the best”? How does this innately connect to how humans learn and perceive information? Are windows and pointers the best representation of how we interact with a system?

As I began researching, I came across a few papers published in the early ‘90s by Jakob Nielsen on Human-Computer Interaction. Nielsen analyzed the dramatic change in interaction when computers transitioned from the command line to a modern GUI. The latter allowed for object-oriented structuring, or direct manipulation, which was based on childlike primitives of hand-eye coordination—for example: dragging a file icon into your trash icon, as opposed to typing ‘rm foo’ in a command line. In the end, this switch made new user adaptation significantly easier.

Direct Manipulation is a concept with which we’ve become so familiar today (particularly in WIMP models), but Nielsen explains that there are many interaction levels involved in a single task.

The graph above illustrates the multiple layers that can be applied to something as simple as deleting a few sentences in a text editor. It begs the question: can we make this interactions even more “direct”?

This is where newer interactions come in. Touch screens and wearables have made interacting with technology more intuitive and user-centric. The focus is now on controlling the task, not the computer, to get to your goal.

Another paper in 2000 by Michel Beaudouin-Lafon, a professor of Computer Science at University of Paris-Sud, introduced methods to compare the effectiveness of different interactions with direct manipulation. Evaluating how classic WIMP elements (Menus and Scollbars) compare with newer interaction techniques (Graspable Interfaces), Beaudouin-Lafon argued that the latter had better Indirection, Integration, and Compatibility—3 properties necessary to evaluate efficiency.

Most remarkable of all was how accurately Nielsen and Beaudouin-Lafon had predicted future interactions and interfaces we now use today; all their papers were published before the iPhone was introduced in 2007! I’m excited to see the new possibilities and developments we’re working on today in the discipline of Human-Computer Interaction, and how this extends to better post-WIMP design.

Aliya

My goal for this project was to learn about the Leap Motion controller and how it works. If you don’t know, the Leap Motion controller is a device that responds to hand motion, manipulating objects on your browser, visualizer, or device. Think of it as a Wii without a remote… and in 3D! As I dug deeper into Leap Motion code, I found that there are many plausible avenues for utilizing the technology. Ultimately, I found that the most accessible route, IMO, is to take DOM elements and animate the CSS transform properties, after creating a 3D object with CSS3 transform properties.

For example, to create a cube you can simply use the following HTML/CSS code.

Or in HTML:

Which gets you:

Then to manipulate the object with Leap Motion, you will use the following code snippet.

In this configuration, the circleElem is the cube and setTransform method takes the hand position and hand rotation from Leap Motion. Then, the programmer can dynamically change the transform style of the cube inside of this function.

All in all, it’s fairly simple to get started with plain DOM elements as opposed to finding or creating 3D models with any of the popular 3D modeling softwares currently available. It’s promising, with CSS3 Transforms, CSS Shapes, and modern browsers’ SVG, – we’re embarking on a nearly real 3D interactive experience for the web.

Ashley

My personal interest was in connecting skeuomorphic digital UI with the real-life counterparts (for example, a button online that looks three dimensional is visually mimicking a physical button). I decided to focus on hover states. Partly because unlike things like scrolling or clicking, they stand out as an interaction that in real life would really appear to be magic. To bring hover states into the real world I constructed a panel covered with a grid of squares cut from thick board.

Under roughly half of the squares were magnets. Half of these had northern polarity, while the other half had southern polarity. To interact with the board users put a magnetic controller (what you might call a digital era finger puppet) on their finger, and then hovered their hand across the board.

This demonstration let the user experience both the standard visual feedback of interactive elements ‘hovering’ to show their interactivity, and also an entirely new sensation when passing their hand over repellent magnets (the feeling was similar to running one’s hand over a bump). The experience was pleasant and novel, and my hope would be that hardware technology in the future can start to incorporate more instances of tactile feedback.

Although the three of us each took three different lenses for examining UI, we all reached a similar conclusion: the future of UI is rife with possibility. That means understanding the past, but always challenging its conventions. That means embracing new technology, but critically evaluating its use and construction. And that means building bold, new stuff.

This post is just a short overview of all the crazy and wonderful things we learned this past Hackweek. To see more of Hackweek in action, be sure to check out chartbeat.com/labs.

P.S. Want to join our team? We’re hiring: Chartcorps, Product Design, and Marketing Design. Check out the openings here.