A mixed-reality shoot 'em up game for VR and MIDI Keyboard. The game is an experiment on playing video games by interacting with real-world interfaces. Here, as a proof of concept, a 25-key MIDI keyboard on the table is aligned with a white box in VR as its avatar. The player tries to shoot incoming enemies by using the white keys of the MIDI keyboard. The projectiles are fired in VR from the position of the keys in real life. The original idea was to overlay the game on top of the passthrough feature of Oculus Quest (i.e. a real-time view of your surroundings). This way the projectiles would fire right away from the keys you press on the MIDI keyboard. However, we need to run the game using Oculus Link for getting the MIDI input, and passthrough is not available when running Oculus Link. So we kept visuals within VR and kept the interaction with outside world.
Be Visionary - (Sponsored by Sony Interactive Entertainment)
Virtual Reality platform
Tools and Technologies:
Unity (any product), Virtual reality head mount display (Oculus Rift, HTC Vive, etc.)
- Minis plugin by Keijiro Takahashi (allows getting MIDI inputs into Unity) - 25-key MIDI Keyboard (You can also use your own controller mapping and emulate the experience. You can do so by creating and assigning a new" Input Action Asset" in Unity.) - Oculus Link (Required to use MIDI inputs, otherwise it would require networking to pass MIDI input from PC to Oculus. It is also worth trying a Bluetooth MIDI keyboard.)
- Connect a 25-key MIDI keyboard and an Oculus Quest to PC.
- Download the Unity project from GGJ or Github.
- Open the project in Unity.
- Switch to Android target in Build Settings.
- Enable Oculus Link from your Oculus Quest headset.
- Start to play in Unity.
- In VR, look down and align the white box to match the MIDI keyboard
- Clicking white keys of the MIDI keyboard should fire projectiles.
Game Design & Development:
Semih Bertug Benim
Semih Bertug Benim