Convolutional Neural Network Feature Extraction Using Covariance Tensor Decomposition

This work describes a new method to extract image features using tensor decomposition to model data. Given a set of sample images, we extract patches from images, compute the covariance tensor for all patches, decompose with the Tucker model, and obtain the most critical features from a tensor core. To extract features, we factorize the covariance tensor (CovTen) into its core and propose a new interpretation of the resultant tensor structure, which holds relevant features in a block-wise arrangement (also named filters, weights, or kernels). This tensorial representation allows preserving the spatial structure, learning multichannel filters, and establishing linear dependence between dimensions, reducing the dimensional complexity (the curse of dimensionality). Thus, the proposed method generates filters by a single feed-forward step using a few samples per class as low as 1. Besides, in kernel generation, labels are not needed. The obtained features were extensively tested using a convolutional neural network for classification. All tests were conducted under the VGG architecture conventions. The experiments helped us identify the proposed method’s advantages versus traditional convolutional neural networks in inference capacity and kernels initialization. We also performed experiments to select hyperparameters (nonLinearity, max pooling, samples, filter size) according to their performance. The inference capacity results showed an increased classification accuracy around 67% with CIFAR 10, 64% with CIFAR 100, and 98% with MNIST, using 10,100,1000 samples with a single feed-forward training. On the other hand, the initialization experiments showed the feature extraction capability versus available initializers (He random, He uniform, Glorot, random), confirming linear tensor constraints’ usefulness to generate features. Using the method as kernel initializer returns comparable findings with state of the art around 91% with CIFAR 10, 72% with CIFAR 100, and 99% with MNIST.