Machine learning – Development of new pattern recognition methods

FHM – Fuzzy Hypermatrices

Figure 1: The structure of the FHM classifier for a 2-dimensional, 3-class problem – (a) location of known samples belonging to given classes 1…3 in the problem space, (b)-(d) fuzzy function values assigned to the given classes (0 and between 100)

  • A very easy to implement and fast classifier that directly assigns fuzzy values to the inputs (thereby minimizing the time required for pattern recognition).

  • In return, the entire problem space must be kept in memory (in the form of multidimensional matrices), which can be very large.

  • Ideal for problems with a low number of input parameters, such as finding pixels of a given color tone in images.

SFIT – Sequential Fuzzy Indexing Tables

Figure 2: The structure of the SFIT classifier for an N-dimensional problem

  • An similarly fast classifier that narrows down the search area in the problem space step by step, using the input parameter values.

  • In the case of N parameters, it finds the model closest to it in N steps, or the most similar model estimated on a statistical basis.

  • It is a bit slower than the FHM classifier, but it can also be applied to more complex (higher dimensional) problems.

  • Several versions have been developed, e.g. ESFIT (Extended SFIT) that can recognize patterns of variable length, while SiSFIT (Simplified SFIT) is a newer, rotation-invariant version.

Fuzzy Shape Recognition

Figure 3: Construction of the contour of a hand – (a) Canny edge detector output, (b) simplified sequences created by back-pruning trees built from pixels, (c) the combined contour.

  • A fuzzy shape recognizer was also developed, which is used to analyze the outlines of specific detected areas (e.g. marked by FHM).

  • The contour lines are marked with a Canny edge detector.

  • A pixel tree is created from the pixels that make up the contour lines, and the contour of the shape is created by simplifying them and trimming back unnecessary branches, and finally, merging them.

Figure 4: Determination of points of interest (green) on a hand contour with different coefficient settings.

  • After that, the contour of the shape can be analyzed.

  • A point of interest (PoI) is a point where a significant change occurs in the direction of the outline (e.g. a corner). In order to ensure an even spread, PoIs are selected for certain (artbitrary) lengths.

  • The PoI pixels are analyzed based on the relative position of the pixels preceding and following them:

    • The ratio of the distance from the previous PoI to the distance from the next PoI

    • The ratio of the distance from the previous PoI to the diameter of the entire shape

    • The angle between the section from the previous PoI and the current PoI, and the section of the current PoI and the next PoI.

  • They receive labels ranging from 0 to 149, accordingly.

  • The label series are processed with the SFIT classifier in order to recognize shapes (e.g. hand gestures)

A practical application – identification of churches and buildings based on their roof based on drone photos

Figure 5: The process of identification based on shape

The system works as follows:

Figure 6: An input image (Bíňa rotund church)

During the training, first the FHM color models are trained based on the colors of the objects (roofs) to be recognized (manually annoted), as well as the SiSFIT classifier with the shape sequences.

Figure 7. – The output of the FHM classifier

1. Using the FHM classifier, pixels with the desired color shade can be found at high speed (1 operation/pixel) in the input image. The typically noisy image thus obtained is denoised with a density-based filter and morphological operators.

Figure 8. – The results of the noise filtering step

2. For the found regions, their contours are extracted with the fuzzy shape recognizer.

Figure 9: The contour of the shape with the identified points of interest (shape descriptors)

3. Go through the sequence of pixels representing the contour of the shape and find the points of interest. The SiSFIT classifier identifies the shape by searching for a known model that is the most similar to the resulting label sequence.

Figure 10: Another example: shape description sequence determined for the roof of Klížska Nemá church

Publications related to the research:

CONFERENCE PAPER: Shape Recognition in Drone Images Using Simplified Fuzzy Indexing Tables

B. Tusor, O. Takac, A. Molnar, S. Gubo, A. R. Várkonyi-Kóczy: Shape Recognition in Drone Images Using Simplified Fuzzy Indexing Tables. In IEEE 18th World Symposium on Applied Machine Intelligence and Informatics : proceedings. 2020. pp. 129-134. ISBN 978-1-7281-3149-8 link

BOOK CHAPTER: An Indexed Rule-Based Fuzzy Color Filtering Method

B. Tusor, A. R. Várkonyi-Kóczy, J. T. Tóth: An Indexed Rule-Based Fuzzy Color Filtering Method. In Recent Advances in Intelligent Engineering (Topics in Intelligent Engineering and Informatics 14). Cham, CH : Springer, pp. 299-307, 2020. ISSN 2193-9411 link

CONFERENCE PAPER: A Fuzzy Data Structure for Variable Length Data and Missing Value Classification

A. R. Várkonyi-Kóczy, B. Tusor, J. T. Tóth: A Fuzzy Data Structure for Variable Length Data and Missing Value Classification. In Recent Advances in Technology Research and Education (Advances in Intelligent Systems and Computing 660). Cham, CH : Springer, pp. 297-304, 2018. ISSN 2194-5357 link

CONFERENCE PAPER, BOOK CHAPTER: A Fuzzy Shape Extraction Method

A. R. Várkonyi-Kóczy, B. Tusor, J. T. Tóth: A Fuzzy Shape Extraction Method. In Recent Developments and the New Direction in Soft-Computing Foundations and Applications (Studies in Fuzziness and Soft Computing 361). Cham, CH : Springer, pp. 383-395, 2018. ISSN 1434-9922 link