Augmented Reality Visors
- This page is a transcription of an unclassified document. For the original, see ARV-401-S3.PDF
Augmented Reality Visors (ARVs) are a family of prototypes and working models of bullet-resistant visor-type attachments specifically made for military-style ballistic helmets. Their design is owned and steadily developed upon by the ASC Department of Defense.
ARVs are used to greatly enhance a soldier's spatial and special awareness through various forms of info displays inserted directly in but not obscuring their field of view. This, coupled with the ability to be connected directly or wirelessly to a practically limitless field of other digitally-capable military equipment, can be immediately and tremendously beneficial to both the individual and the group. Besides overlaying useful information onto the surroundings, wearing ARVs can prevent flash blindness, and give the user 100% useful vision in partial and total darkness and all other conditions.
As previously stated, ARVs have limitless potential in the kinds of beneficial information they can display cleanly in a soldier's field of view, given other equipment to interface with. However, even when external equipment is unavailable or otherwise not in use, many models of ARV can still be a major boon to personnel, due to having an extensive suite of sensors and imaging devices built into the visors themselves. Early models of ARV had these sensors and devices externally exposed, which gave a bulky and uneven weight and look. Later models took an effort in concealing these exposed devices behind a sleek durable slate of metallic shielding, and formed the visor design around a reduction in eye fatigue and an increase in field of view.In terms of physical shape and aesthetic, the latest ARV models offer a smooth, slightly spherically curved internal and external surface akin to snow goggles, enough distance from one's face to easily wear glasses under the visor, and a completely opaque and matte view from the outside looking in, meaning bystanders cannot see in. These models are most commonly seen in active use by the Riot Control Corps, a component of the Department of Defense operating majorly within the civilian sector.
Current models of ARV attach directly to hinges on ballistic helmets that are made to accept a visor, or otherwise are adapted to accept a visor. Stemming from these hinges, flat metal extensions set the bulk of the visor far enough away from the face as to not interfere with most small-sized corrective lenses. The visors proper are opaque; having a VLT of 0%. They are a spherically curved slate of smooth metal, inside of which is housed a host of electronics, most notable of which are multiple spectral imagery cameras, a central processing unit which receives from the cameras, and a screen which visually outputs what the cameras can see, plus other information.
The major layers of ARVs ordered from external to personnel-facing are:
- 1. External metal housing
- 2. Sensor & imaging suite
- 3. Data buses, memory, & central processing unit
- 4. Very high refresh-rate (VHR) screen
1. Metal Housing
The housing of ARVs is made of military-grade composite metal all around for the express purpose of extreme durability, electromagnetic resistance, and very high bullet resistance. The housing is not specifically made to be scratchproof nor fingerprint resistant, however both of these are present in varying capacities, and neither of these wear types are detrimental to usability.
The outside panel of the visor housing is removable, and has a non-constant thickness on the inner face, as is necessary for the placement of camera lenses and certain other electronics. The housing comes default with a drab silver look to it by virtue of its manufacturing process. By its userbase, it is often given a special designation marking like a number on its front face, or less commonly: on the hinge extensions.
At the left hand edge of the visor, there exists two ports which are used for direct connection with the ARV. They are often protected by a rubber plug with the same color as the housing itself. Furthermore, there are physical buttons located on the inner upper edge of the right hinge extension, which are placed in such a way because they are only meant to be used as a backup menu navigation interface.
2. Sensor & Imaging Suite
Full up ARV models are packed with a great deal of electronic equipment, most of which are imagery devices necessary for viewing the surroundings through the visor; and in multiple spectrums either one at a time or at the same time. Any of these devices can be automatically and/or manually turned on or off at the individual level. In the context of the wireless transceiver, it can be variably powered from 100% to 0% and anything in between.
The ARV sensor package includes the following, and each will be expanded upon below:
- 2.1 eXtreme Accuracy Accelerometers & Gyroscopes (XA2Gs)
- 2.2 Laser rangefinder
- 2.3 Visible Light Cameras (VLCs)
- 2.4 Ultraviolet-Sensitive (UVS) "Starlight" cameras
- 2.5 Forward-Looking Infrared (FLIR) cameras
- 2.6 Visible Light Intensification (VLI) "Starbright" cameras
- 2.7 Global Positioning System (GPS) receivers
- 2.8 Acoustic Level & Positioning Sensors (ALPSes)
- 2.9 Wireless transceiver
2.1 eXtreme Accuracy Accelerometers & Gyroscopes
- Accelerometers and gyroscopes embedded in ARVs are used for head tracking and inertial guidance purposes. Multiple of both are used for redundancy and error checking purposes. The CPU (3) of the visors receives continuous input from these accelerometers and gyroscopes to continuously update ARV's position and direction in 3D space, which is paramount to many of its key functions, particularly allowing it to provide the user with coordinates, elevation, heading, and map directions.
- In the context of coordinates and elevation, an ARV initializes its current position in 3D space once by GPS (2.7) whenever it is switched on from a cold start. It will then continuously update this 3D position through the output of the XAG2s, until it either encounters a fatal coordinate error, is manually GPS (2.7) updated, or is turned off completely.
- On an individual level, XA2Gs can withstand upwards of 300 Gs of acceleration without reporting erroneous data, and are accurate to within five millimeters of their real position per hour of continuous movement; this means that an ARV may report a maximal drift of five inches (120 millimeters) off their real position per 24 hours without a new GPS (2.7) update.
2.2 Laser Rangefinder
- Usually located in the upper centerline of the visor, there is embedded a laser rangefinder which can be toggled on or off. The laser's beam is invisible to the naked eye and is pulsed in one of a grand number of settable patterns or "channels" as to minimize interference between individual ARVs, vehicles, and other equipment. This rangefinder is accurate out to 2.5 km (1.6 mi), beyond which it can no longer be safely deemed reliable.
2.3 Visible Light Cameras
- VLCs are embedded in various places, hidden beneath semi-transparent portions of the metal housing (1), behind which they can see the outside yet the outside cannot see the cameras. Two are used at the front, for the purpose of maintaining an appreciable field of view for everyday use in the visible light spectrum, and on the latest models, an additional two cameras are embedded at the extreme edges of the visor, just behind the corners, to give users a greater-than-natural field of view. All models also employ yet smaller cameras on the inner face of the ARV, which point towards the face and are used only for eye-tracking purposes.
2.4 Starlight Cameras
- Ultraviolet-sensitive "Starlight" cameras (as opposed to "Starbright" cameras (2.6)) are used mostly to augment the VLCs' (2.3) vision; in that the image data from the Starlight cameras can be used as a clipping mask over the VLC (2.3) image data, highlighting areas of more intense ultraviolet glow to uncover points of interest in a viewed person or object.
- It is also possible for an ARV user to switch completely over to Starlight vision in order to view into the ultraviolet spectrum from 400 nm to 200 nm wavelengths. These Starlight camera lenses are immediately surrounded by intentionally diffused UV lasers to act as a powerful source of UV light that is invisible to bystanders. This active UV source can be toggled on or off.
2.5 Forward-Looking Infrared Cameras
- High-resolution FLIR cameras embedded in ARVs can be used similarly to Starlight cameras, in that passive IR pickup can be overlayed on VLC (2.3) imagery to potentially highlight points of interest. Users can also switch completely to viewing in the IR spectrum at any time.
- An ARV's FLIR sensors pick up wavelengths between and including 12 µm and 2.5 µm, which for viewing purposes are normalized to a pixel brightness of 100% to 0% respectively. Unlike Starlight functionality, FLIR is completely passive and does not require separate IR illumination to best view the environment.
2.6 Starbright Cameras
- Image intensification cameras, otherwise known as visible light intensification or "Starbright" cameras (as opposed to "Starlight" cameras (2.4)), are the familiar night vision cameras. Specifically, Starbright cameras are embedded in ARVs to multiply the luminance of low- and very-low-light environments. Starbright cameras can intensify color imagery such that light fields much too weak for the user's naked eye to perceive are clearly viewable.
- Starbright cameras are useful in contrast to FLIR (2.5) cameras due to the fact that Starbright outputs clear color imagery in otherwise low light conditions, as opposed to FLIR (2.5) outputting a spectrum of high to low IR pickups.
2.7 Global Positioning System Receivers
- One GPS receiver, or more than one if necessary, is embedded in an ARV primarily for initializing its position and elevation in 3D space upon boot up or otherwise a necessary coordinate update, after which navigation is then continuously updated through inertial guidance with the use of multiple accelerometers (2.1).
2.8 Acoustic Level & Positioning Sensors
- Sophisticated miniaturized microphones are embedded in an ARV which are sensitive to a specific range of auditory frequencies between and including 15 Hz and 150 kHz, and capable of picking up sounds at 0 to 180 dBSPL without clipping. At least two microphones are embedded in an ARV, likely at opposite horizontal ends of the visor, for the express purposes of sound localization.
2.9 Wireless Transceivers
- Wireless data antennae are fitted horizontally along the internal top edge of an ARV, offering the capability of sending and receiving data to and from other ARVs, as well as to and from other wirelessly interfaceable devices like routers, base stations, or general purpose computers.
- There are multiple of these transceivers embedded for the purpose of utilizing beamforming strategies; directional constructive interference for efficient signal deliverance, signal strength, and also the localization of incoming signals. The transceiver array operates in the V band and can send and receive multiple gigabytes of information per second.
3. Data Buses, Memory, & CPU
Directly underneath the layer of electronic equipment lies the next major layer which contains all high-speed circuitry necessary for connecting said equipment to a central processing unit, the CPU itself, and a permanently installed bank of random-access memory. The CPU operates on a reduced instruction set, and also has embedded in it a significant amount of cache.
All electronic equipment (2.) are attached directly to the CPU, and the CPU controls everything in an ARV from boot-up to shutdown. The software which it runs on and what the software is programmed to do is detailed in Firmware Specifications. The CPU proper is an integrated circuit whose design is proprietary to the ASC Department of Defense. This processor includes sixteen cores for efficient parallel processing. It does not have a base clock speed that is then multiplied. Rather, all components and buses operate at the same synchronized clock rate of 1.21 GHz.
Also in this layer, there is a high-capacity battery powering mostly the CPU that can power the other peripherals of the ARV on its own at the same time as a continuous nine hours without recharging, and recharging a discharged battery to full capacity takes about one hour. Along with this, there is a significant amount of data storage used for the recording of imagery in all available spectrums individually, but at the same time. This is what allows the ARV to double as a body camera. The storage is also available to record many other types of data alongside visuals. This storage can be accessed wirelessly or by direct connection to the visor housing (1).
In regards to the data transfer rate of all data buses between layers, the maximum throughput is several times higher than is necessary for the existing components to operate at optimum levels for comfort and stability. Similarly, the CPU is capable of operating stably at a much higher clock speed than it is originally set at during manufacturing, without detrimental effects on the current electronics suite. This is to make ample room for improvement and/or addition of new or more software and/or hardware.
4. Very-High Refresh-Rate Screen
The screen in which ARV users view their surroundings and all overlays set upon it, spans nearly the entire diameter of the innermost personnel-facing layer. The screen itself has the exact same spherical curvature as the front panel. There is a small half-inch bezel between the top and bottom edges and the screen itself. At the left and right sides, the screen extends cleanly to the hinge extensions of the metal housing (1) with no bezel.
The screen has a pixel density of 250,000 pixels per square inch (38,747 px/cm2), and a native resolution of 4000×1750px, giving it a 16:7 aspect ratio. The screen operates at a very high refresh-rate of 1000 Hz, and an average pixel-level response rate of no more than 1 ms for the purposes of minimal eye fatigue and maximum fidelity of even the quickest movements, near-indistinguishable from viewing said movements with the naked eye.
Users' "field of view" to the edges of the screen itself is about 120 degrees horizontally; vertically, this is about 70 degrees. With the screen on and viewing with a four-VLC (2.3) model ARV, a user can actually observe a maximal horizontal field of view of 280 (two-eighty) degrees. This virtual field of view is adjustable in realtime by increments of one degree in the menu system or externally by other interface.
It should also be noted that the screen has a transparent anti-spall coating on its face in order to prevent any potential shrapnel from passing through to the user's face in the event of an impact.
Driving the bulk processing of raw sensor input are several independently R&D'ed software programs (hereinafter referred to as "firmware") that are stored in specific non-volatile memory components, of which cannot be written to nor erased wirelessly nor by external connection. Upon bootup, firmware is loaded into much faster random-access memory. From the RAM, the CPU can swiftly access banks of instructions as to what to do with all sensors' relatively continuous streams of data.
The basic systems as made available from ARV firmware in order of operation per clock cycle are:
- 1. Encryption/decryption system
- 2. Targeting system
- 3. Recognition system
- 4. Eye tracking system
- 5. Menu system
- 6. External input/output system
- 7. Display handler
- 8. Recording system
1. Encryption/Decryption System
An EDS is crucial in ensuring that all communication is secure between itself and any other connected equipment; or arguably more importantly, all communication between itself and the outside world, in the context of wireless connection. Encryption used is end-to-end utilizing the supersingular isogeny variant of the Diffie–Hellman method of key exchange with 2688-bit numbers.
2. Targeting System
This part of the firmware is responsible for all things target related, this mostly includes spotting points of interest within the field of view of the cameras, such as areas of extraordinarily intense infrared radiation as picked up by FLIR. When the TGS spots an object of interest, it will relay that information to the display handler (7.), which then overlays the appropriate reticle over and/or around the target in question. The targeting system works in close conjunction with the recognition system (3.) in the identification of individuals as well as the identification of certain objects such as a gun or knife.
3. Recognition System
Whereas the TGS (2.) points out a target to the DH (7.), the RCS can describe the target to the DH (7.). The recognition module of the firmware is responsible for reading a grand number of small and subtle details about a person or object. It is programmed primarily to read the facial structure of a person, the face's expressions, and the person's body language, and can relay this information to the user through the DH (7.). Secondarily, it is also programmed for the recognition of many different object types, not only including firearms and other weaponry but also mundane items such as license plates, badges, and papers. The RCS can read very small details such as 8-point font text on pages a few feet away, and much farther away given more normal- or larger-sized text.
4. Eye Tracking System
Almost regardless of the angle of the ARV, it can and will track the user's exact line of sight to the screen. That is, the ARV can track exactly where the user's gaze is on the screen, through tracking where each eye is individually pointing. In general everyday use, this is mostly used in conjunction with the menu system (5.) in order to navigate it hands-free.
5. Menu System
The main interface between user and ARV is the menu system, inside of which is housed all options built upon all the parameters necessary for sensor operation. Often, users do not dive deep into these advanced settings, however they more often use the menu system to switch between spectral imaging. The MUS works almost entirely in conjunction with the eye tracking module to allow users to swiftly select things on the fly, incredibly fast with practice. In the event that eye tracking is unavailable, the backup menu-navigating interfaces are the physical buttons on the housing.
6. External IO System
All IO between the ARV and the environment outside of itself is considered to be external, and is handled entirely by the EIOS. The EIOS connects an ARV to various other equipment and/or computer systems through either direct connection to the housing ports or wirelessly through the transceiver array.
7. Display Handler
Driving the screen entirely on its own, the DH constantly updates the screen at the set frames per second (default 1000) which is tied to the screen's refresh rate, meaning the refresh rate will change with the frame rate unless uncoupled in advanced settings. The DH collects all information sent to it and outputs it at the requested place and appropriate time on the screen. The DH "locks" its frame rate to the selected value, and completely writes a singular frame to the screen before writing the next frame to it, thus preventing any tearing of the frame in order to maintain proper visibility.
8. Black Box Recording System
Many different things are recorded all at once by the BBRS. The BBRS packages and records individually the raw data output of all systems and sensors, as well as keeping an action record of all minor and major actions such as button presses and movement, and stores the prepared files on the on-board storage component(s) or outputs to the EIOS (6.) for an attached external device. All of these recorded systems and sensors are put into individual files in order to provide a clear picture for each, in the event that the files are inspected later. This means, for example, a scene can investigated further by looking deeper into separated streams, or a scene can be recreated by overlaying say the targeting system video file atop the visible light camera video file, after having used the action log to find what sensors were actively in use by the user at the time.
The full up ARV system provides a tremendous positive impact on the individual, especially in an urban setting, however it is certainly not restricted to urban environments, as arrangements can be made to alter the ARV during or after manufacture in order to suit the needs of the userbase. Not only does the system treat the individual, but utilizing more than one can increase the effectiveness of the group just as much.
So far, the ARV system has seen a great deal of exceedingly successful useful usage by the RIOT CONTROL CORPS of the Department of Defense who, as many know, have been and still are consistently operating within the civilian population. Large strides have been made in quick and accurate identification of individuals by facial recognition even in the presence of anti-recognition measures being taken by individuals in question. The same can be safely said for other identifiable objects like license plates and photographs.
It should also be noted that a supplement to the ARV system came to RC in the form of simple gun accelerometers used to track weaponry in 3D space and thus display on the ARV exactly where their weapon is pointing and where their bullets will land. This is only a single item of a potential limitless supply of extremely helpful equipment besides the ARV itself that the DOD/R&D Division can provide, or can be suggested to or requested of the DOD/R&D Division.
Outside of the needs of RC, ARVs can be used effectively by any government group finding use in overlaying information and graphics onto their field of view for extended periods of time. The ARV system can be used as-is or it may have parts removed and/or added to suit the needs of the division giving it to troops. One such division is potentially the Public Security Service (PS) of the Department of Civilian Affairs (DCA), who are similar in operation to that of RC. The ARV system is also in consideration for deployment to combat wings such as the National Guard Reservist Forces (NGRF) and Interior and Border Patrol Agencies.