Learning to play music is a challenge not only for beginners. Even an experienced musician has to invest much time and effort in order to imprint their hands, muscles and brain with a sequence of movements.
Musicians are practicing and repeating that sequence until it is embedded in their body memory.
Shay Almog’s project seeks new and intuitive ways to learn music, focused on the motor aspect. Shay is inspired by video games such as Guitar Hero and Beat Saber, wherein the player is required to perform predefined movement sequences, in sync with a song playing in the background and with the help of visual elements guiding the movement. The project explores the use of such visual aids as an alternative for musical notes. Shay created an application that connects the physical world to the virtual world - the user sits in front of a piano, puts on a VR headset and sees a virtual piano overlayed precisely over the physical one, and virtual hands. Visual aids are then displayed, guiding the fingers. Later, the piano was replaced by a percussion instrument and the ten fingers were replaced by two mallets, after realizing that the piano is too complicated of an instrument for this interaction. Long, complex pieces were replaced with short, simple loops. In this later version the visual aids were replaced by virtual mallets. The mallets move around in space and if the users follow them they succeed in playing along. The name of the project arose from the double meaning of “play” – play music and play a game – and from the insight that one of the most effective forms of learning is playing.
