Just recently the role of video systems in robotized platforms was rather a continuation human-machine interface than a full-function device working in the circuit of automatic control. Example of such systems is numerous developments for video monitoring, video recording or semi-automatic regulation with obligatory presence of a man – operator. Embedded devices didn’t have enough computing power for deep analysis of video data flow and most of computational operations were limited by digital filtering and stabilization of images. It’s not surprising since amount of information produced by video sensors is big and usually significantly exceeds total amount of data from all other sensors.
Recently it has become possible to perform more deep analysis of video data and to embed video sensors into the automatic control systems without necessary presence of a man in the circuit. First of all, this is because of constantly growing computational capabilities of embedded systems, second of all, the latest successes of convolutional neural networks (CNN) in the images classification tasks can be considered as revolutionary. One should note that work principle of convolutional neural networks was inspired by nature, the specifics of functioning of primary visual cortex are reflected in the architecture of convolutional neural networks.
Together with that, the control functions objectives are not limited by the classification tasks. For dynamic objects control it’s important to get information not only about class of an object but also about its position, speed and other parameters. Solution of such problems as self-localization or estimation of position and orientation of any other remote objects is one of the most essential areas of research in control functions area at the moment. Beside this, real practical applications of technical vision are often characterized by the lack of possibility to tag learning samples of images. Thus, application of the supervised-learning approach is impossible in the part of practical tasks and it’s advisable to adapt architectures of neural networks for application of reinforcement learning methodology.
In our work we focus on the adaptation of current achievements in the area of convolutional neural networks and in sphere of reinforcement learning for their practical applications in dynamic objects motion control.
Fig.1. Technical vision.
SYNTHESIS OF NEURAL CONTROLLERS
There are a lot of approaches to the construction of neural controllers, these approaches differ by the principles of functioning. There are approaches like mimic based neural control, inverse neural control, backpropagation through time, approximated dynamic programming etc. Together with that there is a great amount of types of neural networks and all of this significantly increases number of possible variants of construction of neural controllers.
Beside this, having decided on the structure of neural controller and type of applied neural network, the developer faces problem of setting of network parameters (hyper-parameters) and ways of its training. All this makes the task of rather optimal control based of artificial neural networks (ANN) even for simple practical tasks almost unfeasible. It’s possible to highlight big amount of publications where the authors stop on that or another approach but by their own words, using just intuitive suggestions.
Optimization of hyper-parameters of neural network gives opportunity to formalize approach to the construction of neural controllers to make it more close to optimal for the particular applications. At the same time it's suggested to make an accent on the methods where it's advisable to optimize all parameters including structure of controllers, morphology and parameters of neural network learning. One of the approaches of this task resolution is evolution optimization by numerous parameters. At the same time it is noted that simultaneous optimization of combined networks of different type is possible. Meanwhile it is necessary to note that not the “phenotype” of the network is a subject for optimization but it’s the “genotype”, i.e., receipt of its construction. This approach allows maintaining area of search in reasonable limits, not limiting resultant network by the complexity.
For example, it is possible to consider the synthesis of a part of non-fully connected recurrent spiking network where its spatial structure is described by only few parameters. Neurons connection mechanism, learning methodology, functioning parameters is determined similarly. With such an approach, even with the synthesis of relatively complex network, list of parameters determining the procedure of its creation, can be composed of a few dozen values which are easily yielded to optimization.
Fig.2. Synthesis of neural controllers.
VISUALIZATION OF NEURAL NETWORKS
One of the features of working with artificial neural networks (ANN) is their functioning on the principle of the black box. I.e. after the ANN is formed based on its recipe, its operation is hidden from researcher's view, except that the signals on the input and output of the network can be monitored, and, if the ANN is integrated into the control system and participates in the control of objects, the quality of its performance can be evaluated by analyzing the behavior of controlled objects. In some cases this may be enough, especially in cases where the amount of the hyper-parameters of the network is small and acceptable results can be achieved by making iterative experiments on the network-object or network-model interactions. But if the number hyper-parameters is large, then conducting a large number of iterative experiments is ineffective. This is due to the fact that the searching area is necessarily limited by the number of experiments that can be performed by a researcher in a reasonable time. It is a kind of "curse of dimensionality" of search area during the construction of neural network architectures.
One of the possibilities to increase the efficiency of a neural network structures is using of an approach borrowed from neurophysiology. Neuroscientists have been already using 3D visualization in their work for a certain time. A striking example of this is the international project called The Blue Brain Project. Neuroscientists together with researchers from the field of computer simulation and visualization recreate an accurate model of the mammalian neocortex column. The main focus is on obtaining of information during the monitoring the simulation of 3D models. The researcher, in this case, is in constant interaction with simulated model. This allows generating and testing hypotheses more quickly and efficiently. The approach allows us to move away from methods of working with a black box and improve efficiency through continuous interaction with the test model.
The problems nature during the synthesis of neural controllers for control systems differs substantially from the problems being solved by neurophysiologists. Neuroscientists are trying to study the principles of natural biological neural networks and memory mechanisms in order to subsequently apply this knowledge to develop new medicines and to solve other tasks from the fields of medicine and biology. On the other hand, for the control functions field requires the ability to build effective neural controllers to solve specific engineering practical tasks. In addition, the neural network models should be adequate to the problem being solved in controls area and should be free from superfluous details.
We borrow from neurophysiologists their approach to 3D visualization, at the same time we exclude from the network model those items where there is reason to believe that they do not directly affect the processing and transmission of information. Simplification of the complexity of the models enables the development of tools for 3D analysis of neural networks on personal computers without the involvement of computing clusters and supercomputers. At the moment we are working on a set of software for visualization of neural networks based on OpenGL technology and a set of extensions for the popular 3D platforms for better detailed offline visualization. Working with 3D models established itself as a powerful tool for the analysis and development of neural network models.
Fig.3. Visualization of neural networks.