Pupeteer

Presenting Puppeteer in jQuery Conference.

 

My Role

  • Conceptualization & Visualization
  • Interaction design 
  • UI design
  • Developing the Chrome Extension

 

Status

Worldwide release (March 1, 2016)

 

Judges and mentors

 

Libraries Used

  • gest.min.js
  • objectdetect.js
  • HTML, CSS, Javascript, jQuery

 

 

 

The Problem

Lack of simple, inexpensive, unobtrusive, intelligent, and OS independent tools to control web through hand and body gestures.

 

The Goal

To create a simple, scalable, extendable and intelligent framework which improves web accessibility using gestures

 

Summary

With a motive of a bringing a novel spin to everyday actions, we conceptualized a gesture based system to handle browser actions. Since chrome can operate on multiple OS, our solution as a chrome extension is OS independent. Leveraging existing hardware in laptops and open source gesture recognition libraries, we implemented Puppeteer. 

Missing parts of a video due to interruption is a common problem all of us face. Puppeteer aims to solve this by tracking the user’s eye. A simple action like looking away from the screen can pause a video and looking back again will cause it to resume. Other use cases involved scrolling and sliding through presentations through gestures.

 

Modes

eyetrack.png

Puppeteer currently supports three modes of operation.

 

Eye-Tracking: When a user looks away any video being played is paused. The video is resumed when the user looks back on the computer. 

 

Scrolling: User can scroll the page up/down and left/right using hand gestures.

 

Music Controls:  Up gesture pauses video, down gesture resumes, left gesture plays previous video and right gesture plays next video.