Mudras in Motion

Kinetic sculpture

Year: 2023
Place: London
Context: Group of four, Cyber Physical Systems module, Royal College of Art and Imperial College London
Role: Concept design, Arduino code, making
With: Hsintzu (Cindy), Harith, Sonia

4 mudras: read by a camera, sorted into four classes
3 servos, 1 stepper: moving the sculpture

A sculpture that answers hand gestures.

01A language of hands

Bharatanatyam, a classical Indian dance, speaks through hand gestures called mudras. Each one carries a meaning. We chose four.

Four painted hands showing Pataaka, Kapittha, Kartari-Mukha and Alapadma mudras on orange — Pataka, Kapittha, Kartari-Mukha, Alapadma.

Pataka, ‘flag’: The beginning of the dance
Kapittha, ‘wood-apple’: Holding flowers gracefully
Kartari-Mukha, ‘scissors face’: Opposition, or overturning
Alapadma, ‘fully opened lotus’: Yearning for the beloved

02Reading the hand

A webcam watches the dancer’s hand. Handpose OSC turns it into 63 numbers. Wekinator, trained on the four mudras, decides which one it is seeing.

Screenshots of hand tracking software beside a photo of the acrylic sculpture — Training each gesture, and the sculpture that answers.

Four screenshots of a hand gesture beside tracking software — Tracking, gesture by gesture.

03Moving the peacock

The sculpture takes its form from a peacock. An Arduino turns each mudra into its own movement of three servo motors and one stepper.

Drawing of the sculpture's internal frame with motors 1 to 4 labelled — The internal skeleton and its four motors.

1 Track: Handpose OSC reads the hand from a webcam and sends 63 values to port 6448
2 Classify: Wekinator, trained on the four mudras: 63 inputs, one output, four classes. It sends the result to port 12001
3 Forward: A Processing sketch passes Wekinator’s OSC messages on to the Arduino
4 Move: The Arduino turns each class into its own movement of three servo motors and one stepper motor
Power: The motors run on their own supply, above 5 volts. The driver’s temperature has to be watched
Tools: Rhino, Arduino, Handpose OSC, Wekinator, Processing

Flow of four steps from Handpose OSC and Wekinator through Processing to the Arduino and motors — The whole chain, with the course material behind each step.

04Shown

Shown at Interplay, a kinetic art exhibition at the Royal Society of Sculptors, London. I designed the concept, wrote the Arduino code and built it with Hsintzu (Cindy), Harith and Sonia.