Technical vision and augmented reality: new research by the US Army
Experienced robot on the ground and visualization of information for the operator
Modern robotic systems are able to perform some tasks in an autonomous mode, for example, to move along a given route, taking into account the terrain and overcoming obstacles. Also, new systems are being developed that can conduct surveillance and reconnaissance, process data and provide ready information to a person. Experiments of this kind have recently taken place in the United States.
Latest Developments
New experiments and tests were carried out on August 24 by the Combat Capabilities Command and the Army Research Lab (ARL). Together with a number of other organizations, they are implementing the Artificial Intelligence for Mobility and Maneuver Essential Research Program, the goal of which is to create artificial intelligence for terrestrial RTKs and ensure its interaction with humans.
To date, within the framework of this program, several experimental models of equipment have been created. The last of them are equipped with advanced surveillance devices that provide autonomous reconnaissance and the issuance of data to a person through an augmented reality system.
For testing, two experimental RTKs with different equipment and software, as well as a set of equipment for a person, were manufactured. All these products worked on a real test site and generally confirmed their capabilities. Robots successfully identified suspicious objects on the ground, and the operators literally saw the results of intelligence.
Experienced Technique
In recent tests, two RTKs of a similar appearance were used. They were built on the basis of the Warthog UGV platform from Clearpath Robotics. It is a four-wheel drive vehicle of compact dimensions with electric motors, capable of carrying loads or special equipment, incl. integrated into control loops. Such a platform is capable of operating on operator commands or in standalone mode.
Scanning lidars of two types were used to observe the situation in the experiments. One Warthog received a VLP-16 LiDAR system from Velodyne; the second was equipped with an Ouster OS1 LiDAR product. Technical vision facilities were connected with computing units and data transmission equipment.
The principle of operation of the two types of robotic scouts is quite simple. Working on the ground, the RTK must "inspect" the environment and create a three-dimensional baseline. Then the lidars continue scanning, and the electronics compares the new data with the baseline. If any changes are detected, the automation should instantly determine their nature, assess the risks and notify the person.
For the operator of such a complex, augmented reality glasses and some related equipment are intended. Information about suspicious objects and changes is displayed graphically in real time. The glasses show the mark on a specific point on the terrain and accompany it with a short description - range, degree of danger, etc. Having received the most complete information, the operator can determine further actions, both his own and the robots.
test data
Recent trials have confirmed the overall potential of the new technology. In addition, the requirements for the components have been clarified. Thus, it turned out that even simple and relatively inexpensive low-resolution lidars can effectively complement human vision. Accordingly, the need for more complex and expensive products disappears - without loss of efficiency and with some increase in productivity.
Two prototypes with different equipment
During the tests, the lidars updated the picture up to several dozen times per second. Resolution - 10 cm. This was enough for efficient work, detection of conventional targets and data delivery to the operator.
In open publications, the presence of two experimental platforms with different complexes of equipment is mentioned, but the differences that they could show during tests are not disclosed. In addition, the features of the passed tests are not specified - the type of terrain, conditional goals, the speed of the entire complex and individual systems, etc.
Directions of development
Experienced RTKs based on Warthog UGV are exclusively technology demonstrators. With their help, ARL checked the functionality of the new hardware and software for it. At the same time, the foundation is laid for the creation of fundamentally new systems that can be used in the army. On the basis of current developments in the future, new complexes of equipment can be created, suitable for implementation in practice.
The developers of the project believe that such technologies must definitely reach the army. The joint work of people and technology will allow more effective surveillance and identify more dangerous objects. RTKs will be able to find enemy ambushes, explosive devices and other threats - without endangering living soldiers.
Feedback questions need to be worked out. The operator, having received data about the found object, will be able to make his own comments and corrections, and the robot will have to process and take them into account. In addition, the creation of learning systems is not ruled out - in this case, the RTK will be able to use the accumulated experience and more correctly respond to the situation.
Another promising idea, which is being worked out in the current program, is the creation of eye protection devices with augmented reality function. The need for such technologies is considered obvious and indispensable in the context of the further development of combat equipment. Glasses can give out a wide variety of information, and not only data from a scout robot.
It should be recalled that the current work is part of the larger Artificial Intelligence for Mobility and Maneuver Essential Research Program. Its goal is to create a RTK capable of independently determining all the features and parameters of the terrain, adapting to them and freely moving and maneuvering. The proposed technical vision means can be used as a universal component for collecting all data, for driving, reconnaissance, shooting, etc.
Step by step development
To date, in the United States and other countries, robotic systems for military purposes have been created, equipped with technical vision equipment with various functions. So, they have already mastered independent movement across different areas or monitoring the situation with the identification of targets. Some of the tasks so far have to be solved using the operator's console.
One of the urgent tasks is to increase the efficiency of work in already mastered areas - to improve the "skills" for driving or more accurate target recognition. Technologies are also being created for effective interaction of the RTK with people, which is now being done by the US Army Laboratory.
In the longer term, current and future projects should lead to very interesting results. A wide range of robots of different classes and purposes will be able to appear in army units. They will accompany the soldiers on the battlefield and take on some of the tasks, as well as instantly exchange information with them. RTKs have already reduced the load on the fighters, and in the future this trend will continue.
Thus, the achievements of science and technology can again change the appearance and capabilities of the army. Apparently, the American army will be the first to master the new reconnaissance means. However, in the future, similar systems will have to appear in other countries. Time will tell what such events will lead to.
Information