About
DrawMotion is a personal prototype that turns hand gestures into strokes on a canvas. Inspired by a demonstration by Amy Plant, it started with a simple question: could I build a usable drawing experience without a keyboard, mouse, or touchscreen?
I designed the interaction, interface, and architecture, then handled the code, tests, documentation, and deployment. It was not built as a commercial product, but as a technical challenge focused on gesture-led interaction.
The experience at a glance
- The index finger moves the pointer.
- Pinching draws; releasing the pinch lifts the pen.
- A closed fist activates the eraser.
- A held peace sign opens and controls the gesture menu.
Conventional controls remain available for settings and export, but drawing itself can be completed entirely by hand.
Under the hood
The interface combines React, TypeScript, Tailwind CSS, and shadcn/ui components adapted to the product. Tracking and drawing remain isolated from that interface layer.
MediaPipe Hand Landmarker analyses the camera in a Web Worker. The model and its WebAssembly runtime ship with the application, so neither video nor hand landmarks are uploaded or stored.
- Translate hand landmarks into stable gesture intentions.
- Send those intentions to a Canvas 2D engine outside React rendering.
- Add history, PNG export, stabilized drawing, and shape assistance.
Two canvas layers separate persistent drawing from interactions. Only one frame is processed at a time, with one replacement frame waiting, which prevents an ever-growing computation queue.
Result and limitation
The prototype meets its goal: drawing, erasing, choosing commands, managing the canvas, and exporting a result through gestures. More importantly, it taught me to design an interface from the body and movement instead of conventional input devices.
Its main limitation comes less from tracking than from human motion: drawing in the air remains unfamiliar and imprecise. Stabilization and shape assistance reduce that problem, but a public-facing version would need to interpret intent more deeply. The prototype has not yet been tested by other people, so I do not claim user feedback or adoption metrics.
Publication
I presented the prototype and its gesture-led interaction in this LinkedIn post.