The IronVision situational awareness system from Elbit Systems in 2017 was tested by the Israeli army in an armored vehicle with closed hatches
The modern battlefield for armored vehicles is becoming increasingly lethal. In addition, the experience of modern combat operations shows that there is a great need for advanced armored vehicles capable of maneuvering and fighting with fully closed hatches, especially in combat conditions in populated areas. Take a look at some potential solutions in this area.
Traditionally, working with closed hatches significantly complicated the tracking of your forces and forces of the enemy, and especially threats at short distances. Developments in the field of active protection systems have partially solved the problem of combating such threats in terms of survival, but such systems should work with closed hatches. In this regard, there was a need to develop new technologies in order to restore the level of combat situation or situational awareness, which decreased as a result of the transition to hostilities with closed hatches.
In order to alleviate the severity of the problem, concepts originally developed for aviationbegin to be applied in land vehicles. Through the use of simulated-environment technologies, the traditional levels of Situational Awareness (SD; the quality of the integrated perception of heterogeneous information in a single spatio-temporal volume [tactical, navigational, general geographical, etc.]) in a machine, even when working with your head out, are not only comparable but in some cases increase. Further development can go through the integration of augmented reality (DR; adding imaginary objects to the images of real-world objects, usually of auxiliary informative properties), in other words, layering various visualized data on real-world images in order to qualitatively transform the CO image and present it to the commander.
However, it is important to determine how to present this information and how to dispose of it. The increased amount of information should help the commander to command, and the driver to drive the car, and not be an obstacle that must be overcome.
Helmet of the pilot of the latest American fighter F-35
Since there is a need to present information in an accessible form to the user, the developers have proposed several solutions. First of all, we should mention helmet-mounted displays (English HMD - helmet-mounted display), which allow users to observe the situation around the car without looking out, looking at the seamless images displayed with a negligible delay. Many consider this decision as the “gold standard” in the field of displaying the CO image.
For example, in the F-35 fighter, the helmet is integrated with aircraft avionics and sensors, and all information previously displayed on the cockpit windshield is now projected onto the helmet visor. The position and orientation of the pilot’s head are monitored, thus, in his field of view, each target is identified and highlighted or assigned to attack.
Elbit Systems used its HMD aviation technology to develop an analog system designed for use in armored vehicles. IronVision is a helmet-mounted display (powered through a single cable) that feeds images from three or six day / night optical-electronic cameras. Installed around the perimeter of the machine body, the cameras provide a conformal display of the surroundings - horizontally all-view 360 ° and vertically 90 °. The multispectral image from the cameras is displayed on the video channel of high quality directly to the biocular instruments of one or two users located in front of their eyes.
In addition, IronVision can interact with the on-board combat control system (SMS), displaying information on the combat mission being performed on the image projected to the commander. The vice president of the ground systems division, Boaz Cohen, confirmed that in 2017 the Israeli army tested this system on the main battlefield. tank Merkava Mk 4. The Ma'anak battle control system of the Merkava Mk 4 tank is part of the Elbit TORC2H information and control system, and thus IronVision can overlay information about tracking its forces and other data.
Due to the fact that the IronVision system is compatible with the NGVA STANAG 4754 (NATO Generic Vehicle Architecture - the standard NATO architecture for vehicles) - mandatory for all NATO programs on existing and future habitable and uninhabited vehicles and subsystems - and can work with a battle management system , it is also capable of combining data from other means on the battlefield, for example, UAVs, and then presenting the information in the PIP format. Similarly, the commander can choose to view the image from any sight or sensor on the machine and monitor the corresponding system. When controlling an armament complex in the same mode, the IronVision system is able to synchronize the armament with the commander's line of sight, simplifying work in search and impact mode. If the operator needs to look at the situation inside the car, he can stop the video signal and observe through a completely transparent lens.
However, if the aircraft cabin or habitable armored vehicle compartment provide stable conditions for tracking the position of the head inside a certain volume, then the electromagnetic situation inside the combat armored vehicle is much more complicated than in the aircraft cabin. Elbit Systems has developed several proprietary algorithms to solve this problem, which allows to obtain an absolutely clear image.
The British company BAE Systems has developed its “through view” system through armor. The BattleView 360 digital imaging system tracks and arranges elements of the environment surrounding the machine before presenting the user with a fully processed conformal image (used in cartography, when part of the surface of the globe is to be drawn on a plane (on a map) with all the angles preserved) with tactical information, taken from the SMS and superimposed on the HMD.
Stefan Thelin, Chief Technologist at BAE Systems Hagglunds, said that five different DR helmets were tested with the BattleView 360 system, with the result that it was determined that the most functional product is a self-designed Q-sight monocular, which is installed in front of the right eye. It was also confirmed that the new Q-sight with a wide field of view can provide not only more situational information, but also provide better image clarity.
According to the company, the technology used in Q-sight is based on a unique method of transmitting light through the use of holographic technology and the concept of optical waveguides. This configuration minimizes losses from traditional optics and eliminates the need for any intermediate lenses, which allows for a lightweight, compact, brighter optical display that does not produce distortion. The wide field version of 40 ° x30 ° has a display resolution of 1024x768 pixels, while the collimated images displayed on it are focused to infinity.
An image from the IronVision system of Elbit Systems with superimposed icons of the location of its own and enemy forces
The need for the BattleView 360 system was determined in accordance with the requirement of the so-called “CV90 Armored User Group” to increase the level of CO. Lead Engineer for CV90, Dan Lindell, said that BAE Systems decided to implement technologies that are already at a high level of technological readiness before combining them into one system that will have significantly greater capabilities.
However, Lindell also explained that they are considering a system that would not be just a “transparent” armor technology. “In addition to the fact that BattleView 360 is a circular vision system for all 360 °, it also includes a three-dimensional map system, the ability to connect to external systems, such as UAVs and ground mobile robots, to provide the crew with information while on the move. In addition, before presenting the image to the crew, it imposes an augmented reality on the simulated situation by connecting to the SMS at a higher level. ” Joint work with the SMS allows BattleView 360 to analyze, for example, data from the UAV, and combine them with tactical signs of the disposition of their own and enemy forces. Further, the final picture is presented to users on various means of displaying images.
The BattleView 360 system also significantly reduces the workload on users by ensuring that the information and capabilities that they are provided with are consistent with the task they perform. In practice, this means that when the shooter uses BattleView 360, the system communicates with its subsystems on the platform, providing the best information for this task. If an active protection complex is installed on the machine, for example, then a very high image quality is needed to distinguish the attacking ammunition. The BattleView 360 system provides a high degree of detail to objects, being perhaps the best option for identifying targets. Using its sensors to identify targets dramatically increases the efficiency of the gunner.
BattleView 3 360D card developed by BAE Systems
In addition, despite the fact that, if necessary, the BattleView 360 system allows the user to manually add or remove information about the situation, its software is intelligent enough to automatically respond to a situation in which the user finds himself. Lindell explained: “While the shooter searches for targets, certain information will be added to his sighting system from the SMS. Further, when it goes into target capture mode, the information will automatically change to something more accurate in order to support it in this mode. Finally, after the shot, the system will automatically return to the search mode. ”
BAE Systems claims that modularity and scalability underlie this system. While the system comes with three cameras that provide full-length coverage and provide a high-resolution image in day and night (infrared) modes, BattleView 360, being compatible with NGVA, allows different users in the machine to add data to any image from any sensor. , whether it is a rear-view camera, an armament camera or a commander's sight. Situation information is also distributed between users through conformal inserts that show the commander where the shooter is looking, and vice versa. In addition, images from platform sensors and external sources can be displayed in the “picture in picture” format, while the image from individual sensors can be combined to obtain such features as, for example, a thermal image on the background of the day image.
2D card BattleView 360 system developed by BAE Systems
However, despite the fact that HMD is an important element of the system, BattleView 360 can work with existing displays of vehicles. The image from the sensors can be displayed on monitors in order to increase the level of controllability of the system and to conduct more in-depth data analysis. As a result, paratroopers in a car that do not use HMD can analyze and add necessary information, such as the recommended route and coordinates of dangerous areas, to the general picture of the crew’s CO on standard displays.
Lindell also confirmed that BattleView 360 as a complete system is at the level of technological readiness of 6 (technology demonstration), and the function of "transparent" armor and integration of DR at the level of 7 (readiness of subsystems), and that it is currently being tested by the customer.
While the development process of this technology has made it more common, the development of such “gold standard” systems from scratch is still extremely expensive. Instead, companies are introducing options for the second and third row, which allows obtaining alternative systems for working with closed hatches. One of the options is to borrow technologies in the civilian market in order to introduce ready-made solutions and systems into which special software can then be loaded and which can be integrated with the SMS platform and other subsystems and sensors.
In line with this trend, Rheinmetall developed its PanoView HMD system and is now ready to market it. PanoView is essentially an armored vehicle imaging system based on the DR. The system uses a ready-made commercial helmet DR of an unnamed manufacturer, hardened in accordance with the MIL-SPEC standard and received a certificate of operation inside armored vehicles.
A representative from Rheinmetall explained that PanoView is “a functionally complete solution for commanders of infantry combat vehicles. It not only provides a seamless panoramic image of the environment, but also integrates information from various sensors and control information systems. ” The commercial helmet in the PanoView works as a matter of fact as a component of the system visualization, which integrates information from various sensors directly into the field of view of the commander.
Further, focusing on how PanoView interacts with the SMS platform, he explained that “it is able to track its own and enemy forces, working in conjunction with the information management system. It takes the data and, through the Augmented Reality function, PanoView puts it in the field of view of the commander, overlaying it on the main picture. ”
In addition, since PanoView is compatible with NGVA and is based on a network infrastructure, the system allows for a modular approach when summarizing data from various sensors. This means that it works inside an essentially scalable system. Consequently, most of the functionality of modern SMS, for example, operational management, geographic information system and target identification, can be built into the system.
In addition, by complying with the standards of Joint Dismounted Soldiers System (STANAG 4677), PanoView goes beyond just a panoramic visualization tool, allowing the commander to access other sensors on the machine, such as the arrow location and laser irradiation systems or even actuators, for example, 40 -mm Raphe Obscuring System smoke curtain system from Rheinmetall. However, if the user does not have a system compatible with STANAG 4677, then an adapter is needed to visualize this information.
The open architecture of the system means that the operator can essentially integrate his own sensors into it. An open architecture PanoView system was tested on the Marder armored vehicle together with the Rheinmetall Dual ЕО optical electronic system with IR and day cameras, allowing the commander to switch between them. Depending on the external contours of the machine, six to eight cameras can enter the standard system, although the system of high-resolution 10 cameras is currently being tested.
Helmet Q-sight monocular from BAE Systems
The system can combine video signals from other platforms: the company also confirmed that up to four video streams can be received simultaneously, and the commander can switch between them and, if necessary, display video from his and an external source simultaneously. If the commander needs to look at the situation inside the car, then the display simply leans up. In addition, the company noted that Rheinmetall discussed with the Bundeswehr the installation of the PanoView system on its Puma BMP, as they expressed a desire to include more than one helmet in its structure so that the paratroopers could supplement the CO commander. For its part, Rheinmetall has the technical ability to solve this problem.
The Ukrainian state-owned foreign trade company Ukrinmash went the same way as Rheinmetall by purchasing the Microsoft HoloLens license for the production of the LimpidArmor helmet system, which, using optical and thermal imaging sensors, provides a full viewport at distances up to 300 meters. Four cameras, each with a 90 ° coating, are installed on the body of the vehicle, and the LimpidArmor system renders one frame with a delay and stitches the images in order to obtain a conformal picture of the situation surrounding the car. Working in conjunction with the SUB machine, LimpidArmor can complement the user's picture by overlaying the identification data of their own and the enemy forces, as well as embed the image from the drone. In addition, the system has the functionality that the developer calls "look-capture-shot", that is, the target tracking system and weapon system can work through the LimpidArmor system.
According to reports, the company Ukrinmash, which is part of the state company Ukroboronprom, received preliminary orders from the Ukrainian government for LimpidArmor systems
A big problem for all HMD systems, according to Cohen, is the intense electromagnetic field that is always present in an armored vehicle. The problem lies in the accuracy of measurements, while tracking the movements of the operator’s head inside the machine, it is necessary to ensure that the operator sees exactly what he is looking at. Failure to cope with this problem leads to a blurred image due to a large spatial delay. A delay of more than 200 milliseconds can also lead to operator motion sickness.
Cohen said Elbit Systems used its experience to develop HMD for pilots to cope with the problem and provide a fully calibrated system. A tracking unit was installed on the back of the helmet, which monitors the position of the user's head in inertial space and then displays the video in accordance with the direction of gaze with a delay of the entire 60 milliseconds. Rheinmetall could not confirm the technology underlying this system, saying only that a high level of image processing inside the frame was achieved.
Mr. Thelin noted that due to the use of gyroscopes you can track not only the movement of the head, but also take into account unexpected movements, that is, the image is not deformed and does not stop from the sudden movement, because it is stabilized in the same way as the main instrument is stabilized. Thelin also stated that the BattleView 360 has an “extremely low latency”.
The image from the IronVision system of Elbit Systems, where the image from the UAV is displayed in picture-in-picture format
View from above
One of the very useful functional features of almost all such systems is the ability to integrate the video signal from the drone into the CO picture at the local level.
The ability to see a picture from a bird's eye view significantly increases the commander’s awareness of the situation, but the possession of this information with superimposed tactical conventions raises the level of its CR WITH exponentially. The representative of Rheinmetall explained that “in fact, the video signal from the drone camera is just another sensor that the SMS works with in order to integrate its data into the overall picture, that is, if large specialized platforms can work as part of the systems system, then why do not use small and cheap UAVs or ground vehicles. ”
Jamie MacDonald, head of platform integration at BAE Systems, said that "the use of UAVs has traditionally been a rather specific solution - a special ground control station, specially trained personnel from a special unit assigned to headquarters ... unlike when the crew is issued for surveillance, which he manages directly. "
The main driver of the integration process of unmanned systems is understanding how in a modular open system compatible with NGVA, you can use developments from the civilian market to accelerate the introduction of improved capabilities into the combat space while at the same time excluding most of the existing infrastructure and transferring capabilities directly in the armored car itself.
MacDonald noted that by operating unmanned systems like any other system, the commander can set tactical tasks, for example, “follow me” or monitor a given area. At the same time, the ability of these systems to work independently and provide relevant situational data only increases their demand as a means of increasing the level of CO.
MacDonald noted that “due to the integration of commercial equipment, you can quickly move from equipment that costs many thousands of dollars, such as a panoramic sight, to a situation where you launch a drone from the stern of a car and instantly get a picture of combat space in a radius two kilometers away. The advantage here is that you don’t need to put your soldiers at risk, because you can send technology forward ... if you lose the unit costing just 200 dollars, then it’s no problem, get a new one. "
In addition, by expanding these capabilities by owning numerous NGVA-compatible machines that deploy multiple monitoring tools, it is possible that the system can recommend, for example, which tools to use to achieve a specific goal, based on data such as battery life or range.
However, an important part of any situational picture is the level of confidence of the commander in the apparatus and the information that is sent from him. Rory Bryn of BAE Systems explained that working on the quality of information that the device can provide is the basis for moving forward. He also said that BAE Systems had demonstrated these capabilities to the British Army in order to draw attention to the benefits of using technology originated in the commercial sector.
Teggeh 2 armored drivers control panel, into which the ARSS and CHDSS systems can be integrated
The Singapore company ST Engineering has developed its own systems for working with closed hatches and has now integrated them into the BMP Teggeh. These are the ARSS (All Round Surveillance System) and the CHDSS (Closed Hatch Driving and Surveillance System) system. Both of these systems are also expected to be integrated into the next generation NG AFV (Next Generation Armored Fighting Vehicle) combat vehicle. ARSS is a mixed system of cameras, thermal imaging and for low illumination sensitivity 11 suites installed around the perimeter of the case, images from which can be displayed either separately or overlaid.
Such a combination of sensors means that both types of images can be displayed on one monitor, allowing the commander to view thermal images superimposed on a high-resolution color video of the surrounding terrain. In addition, in the NG AFV armored vehicle, the ARSS system provides real-time images from the cameras also to the troop compartment, that is, the paratroopers receive additional information about the situation and, as a result, are tactically better prepared for landing, while the commander can use other functions provided by the system.
The CHDSS system consists of three flat-panel displays installed in the driver’s compartment, to which the image from day and infrared cameras is displayed, which makes it possible to maneuver with closed hatches. The system receives real-time information about the location of the vehicle from the sensors and displays the coordinate grid and intermediate points of the route on the driver’s displays, so the driver can rely on the vehicle commander when choosing the direction of travel.
In the Russian T-14 tank, the Armata also implemented a driving solution with closed hatches similar to the BMP systems Tegtech and Teggeh 2. The technology of “armor” for MBT T-14 provides all-round CO by stitching images from different cameras installed along the perimeter of the hull. The picture from them is displayed on three sensory monitors of the commander, to which other information on the combat mission can also be displayed. Sensor technology allows the commander to switch between cameras or other incoming information by touching the screen, while the system's algorithms provide image stitching and conformal image representation.
Although it is not possible to confirm precisely the additional functionality, it is quite possible that software for identifying people is integrated into the Russian tank system. It is not clear whether this function works simply as a warning system when approaching a threat, or whether the system is able to archive images for intelligence gathering.
Despite the obvious advantages of integrating elevated levels of CO into ground vehicles, one of the main problems today is the problem of information overload, to which their crews are exposed.
However, the information itself does not necessarily bring a problem; rather, the need to prioritize, classify and process huge amounts of data can potentially cause cognitive overload for the commander. In addition, excessive “pumping” of the commander with data about the task increases the risk that the critical part of the information may be missed, which in terms of CO means a possible general defeat. Instead, the greater amount of information available should be simplified and presented in a timely and appropriate manner.
In order to avoid cognitive saturation and at the same time ensure that the commander has access to as much information as possible, there are two realistic options: reduce the cognitive load or move it to another location.
The use of PD by the methods described earlier is an attempt to reduce the cognitive load on commanders by summarizing data from sensors and contextualizing information before it is displayed in their field of vision. Merging the digital and physical worlds in a way that allows users to interact with both of them at the same time means that the digital perception of situational data influences decision making in the real world. Thus, a feedback loop is created from analog to digital form and vice versa, which gradually reduces cognitive load.
However, the battlefield is a unique environment regarding the amount of available information and the severity of the consequences of losing or incorrect analysis of situational data.
One of the possible solutions to reduce the load is the use of artificial intelligence. The ability of artificial intelligence to analyze certain types of information means that the stage of analysis of the decision-making process can be separated from the workflow of the commander, and then reintroduced into the circuit if necessary to make a final decision.
For example, the SUB FINDERS C2 developed by the French company Nexter was updated by integrating artificial intelligence, which significantly reduced the cognitive load on machine commanders. By integrating into the windmill (vehicle electronics) of each specific machine, the FINDERS C2 SUB is able to continuously analyze signals from the platform sensors in order to determine the characteristics that the system has been trained. Artificial intelligence is able to learn to recognize the types of weapons, models of machines and human faces and warn the commander in the event of a positive match.
One of the passive elements of the system is the range finder, which detects the presence of an object, classifies it as a potential threat and determines the size and distance to it. If an object crosses a predetermined perimeter, a warning is issued to the vehicle commander. In addition, the passive system for determining changes determines the changes in the surrounding area during movement.
In this regard, the commander deliberately places himself outside the contour until the artificial intelligence system warns about crossing a certain border, that is, in this case he has to analyze only one threat that deserves attention instead of several potential threats.
The problem of cognitive load is also another aspect associated with different generations. Cohen said that the experience of Elbit Systems shows that young servicemen who come to the service are able to master new advanced equipment much faster and get used to it.
Recognizing this fact, companies also seek to reduce cognitive load by synchronizing with the commercial market of interfaces and platform management tools. For example, in November 2017, the British army used a XBox-style controller to control the MRZR 4x4 and the Terrier engineering vehicle.
Xbox-style controller is used in test tasks for remote control of the engineering vehicle Terrier and the MRZR of the British Army
Augmented reality has every opportunity to revolutionize traditional levels of situational awareness available for armored vehicles. The successful development of technologies for tracking the movements of the head of an operator in a complex electromagnetic environment of an armored vehicle means that in the future the defense industry can successfully take advantage of all the achievements of the commercial market in the field of augmented reality.
Not being a problem from a technological point of view, the focus is now beginning to shift towards improving the quality of the information available in the combat space while reducing the burden on those who must process it.
On the materials of the sites: