MULTIMODAL DATA PROCESSING BASED ON ALGEBRAIC SYSTEM OF AGGREGATES RELATIONS

Abstract

Context. In many engineering tasks where the monitoring of changes in the characteristics of an observation object, subject, or process is required, it is necessary to process multimodal data recorded with respect to time moments when these characteristics are registered. In this paper, the author presents a new approach to solving the problem of multimodal data structures timewise processing, which allows to simplify the processing of such data by using the mathematical apparatus of an algebraic system of aggregates and thereby reduce the requirements to computing resources. The algebraic system of aggregates operates with such specific data structures as aggregates and multi-images. These complex data structures can be obtained as a result of data measuring, generating, recording, etc. The processing of such multimodal data can also require discrete intervals processing.

Objective. The goal of the work is to formalise the relations between basic mathematical objects defined in the algebraic system of aggregates, such as elements, tuples and aggregates, as well as the data structures based on these mathematical objects, namely, discreet intervals and multi-images.

Method. The research presented in this paper is based on both the algebraic system of aggregates and the concept of multi-image which enable multimodal data timewise processing. A carrier of the algebraic system of aggregates is an arbitrary set of specific structures – aggregates. An aggregate is a tuple of arbitrary tuples, elements of which belong to predefined sets. Aggregates can be processed by using logical, ordering, and arithmetical operations defined in the algebraic system of aggregates. A multi-image is a non-empty aggregate, the first tuple of which is a tuple of time values. Such tuple of time values represents a certain discrete interval. To process discrete intervals and multi-images, a set of relations is defined in the algebraic system of aggregates. This set includes relations between tuple elements, relations between tuples, and relations between aggregates. The relations between tuples enable arithmetical comparison, frequency comparison, and interval comparison. This mathematical apparatus can be used for both complex representation of object (process) multimodal characteristics and further timewise processing of data represented as multi-images.

Results. The approach to discrete intervals and multi-images processing based on relations, which are defined in the algebraic system of aggregates, has been developed and presented in the paper. The author provides examples of the developed approach practical implementation.

Conclusions. The results obtained in the research presented in this paper has shown that the relations defined in algebraic system of aggregates enable processing of complex data structures named multi-images in data modelling, prediction and other tasks. To allow data processing with respect to time scale, discrete intervals can be employed. A discrete interval is a tuple of time values. In the paper, the author shows how relations for discrete intervals comparison can be used for solving practical tasks. Besides, the author presents the software tools which can be used for practical implementation of the given theoretical approach by employing the domain-specific language ASAMPL.