Presenting Puppeteer in jQuery Conference.


My Role

  • Conceptualization & Visualization
  • Interaction design 
  • UI design
  • Developing the Chrome Extension



Worldwide release (March 1, 2016)


Judges and mentors


Libraries Used

  • gest.min.js
  • objectdetect.js
  • HTML, CSS, Javascript, jQuery




The Problem

Lack of simple, inexpensive, unobtrusive, intelligent, and OS independent tools to control web through hand and body gestures.


The Goal

To create a simple, scalable, extendable and intelligent framework which improves web accessibility using gestures



With a motive of a bringing a novel spin to everyday actions, we conceptualized a gesture based system to handle browser actions. Since chrome can operate on multiple OS, our solution as a chrome extension is OS independent. Leveraging existing hardware in laptops and open source gesture recognition libraries, we implemented Puppeteer. 

Missing parts of a video due to interruption is a common problem all of us face. Puppeteer aims to solve this by tracking the user’s eye. A simple action like looking away from the screen can pause a video and looking back again will cause it to resume. Other use cases involved scrolling and sliding through presentations through gestures.




Puppeteer currently supports three modes of operation.


Eye-Tracking: When a user looks away any video being played is paused. The video is resumed when the user looks back on the computer. 


Scrolling: User can scroll the page up/down and left/right using hand gestures.


Music Controls:  Up gesture pauses video, down gesture resumes, left gesture plays previous video and right gesture plays next video.