HoloShot
Augmented Reality Weapons Training & Simulation
by
Ever wanted to play Duck Hunt with an M4?
Shoot holographic zombies as they climb through the windows of your home?
Re-enact the Bin-Laden raid in your own personal combat HoloDeck?
All of this is now possible with HoloShot.
The HoloShot system sits at the unique intersection of tactical training and video game simulation. It is a simple solution to an emerging need that the US Army has only recently identified as a "technological insertion point" within their Synthetic Training Environment (STE).
Dry-fire training with guns has long been hailed as a useful and safe alternative to live-fire exercises. There are certain things you can only safely do in a simulator (e.g., crashing in a flight simulator). The education and training benefit of such "safe failure" can not be overstated, especially with dangerous activities like shooting and flying. By layering augmented reality targets and other digital artifacts on top of the shooter's natural field of view, HoloShot takes dry-fire practice to a whole new level of immersive interactivity.
HoloShot is an Instrumented Weapon solution the US Army's Squad/Soldier Virtual Trainer Weapons Optimization (S/SVT WO) problem, part of the larger STE effort. An Adapter kit (our simulator bolt + magazine) connects the Controller (the rifle) to the Game Console (HoloLens/HMD) via an SDK/API (which we have developed in parallel with the Adapter). The user supplies their own M16/M4/AR-15 platform rifle, and our system:
1) converts the rifle into a functioning dry-fire weapon simulator,
2) communicates the physical movements and actions of the rifle to the HoloLens, and
3) provides a programmer's toolkit to render the desired holographic display elements and build interactive applications.
The M4 rifle is the actual “game controller” here – and all normal operator controls are preserved. Our system is a controller-adapter (or controller interface) that converts existing rifles into game controllers using easily insertable devices, our e-Bolt and e-Mag. Designed precisely around the internal dimensions of the M4, the conversion kit works with all M16/M4/AR-15 platform rifles (and pistols) that accept standard BCGs and magazines. Installation of the kit replicates the standard procedure for replacing the M4 BCG and magazine, which preserves the operator's manual of arms and avoids introducing negative habit transfer. This design approach allows the system to be used in the latest modern rifles, yet is also backward compatible with older equipment (e.g., M16A1).
Our API packages enable development of engaging, interactive shooting scenarios and apps. Our prototype “Steel Shooter” app for HoloLens allows users to realistically “shoot” holographic steel targets at various distances within their augmented field of view. The app leverages our core game engine API packages, enabling 3D animations, simulated physics, collision detection, visual and sound effects, etc. This is as fun as it sounds... aim, pull the trigger, and ring that gong!
Our prototype HMD features a Microsoft HoloLens affixed to a modified standard combat helmet (PJ/FAST-style). By carefully adjusting the placement of the HoloLens relative to the shooter's eye, we have developed a wearable system that allows natural shouldering of the rifle, comfortable cheek weld and eye position, the ability to use the rifle's iron sights without damaging the HMD, and the ability to keep all targets in the rifle sights also in the shooter's augmented FOV. The HMD has good ergonomics, weight, and feel, and since it is HoloLens-based, it leverages the platform's integrated Bluetooth, 3D stereoscopic sound (which adds significantly to perceived realism), and its excellent existing development tools and workflow (Unity + Visual Studio). HoloLens 2 also has user eye-tracking, further empowering developer innovation and creativity.