Learn how to play the Piano in Mixed Reality

For this innovative project, our team developed an Oculus headset app designed to help users learn to play the piano in mixed reality. The app tracks finger movements, allowing users to play on a virtual piano. Alternatively, you can play a real piano as the app detects the notes you play. Notes appear and move towards the piano in mixed reality. The player has to play each note on the piano accordingly, providing a hands-on experience for learning the piano.

Our work focused on programming the app’s core functionality, integrating real-time finger movement tracking, and designing an intuitive user interface and much more. We also implemented several interactive features such as Wait Mode, Loop Mode, Slow Mode, and more, to cater to both beginners and advanced players. Whether using an electronic or acoustic piano, users can learn their favorite songs quickly and efficiently. With features like virtual music sheets, real-time feedback, and a composing mode, learning piano has never been this immersive and accessible.