The Big Data revolution promises to transform how we live, work, and think by enabling process optimization, empowering insight discovery and improving decision making. New sciences that heavily relies on processing the data, allows us to achieve such high goals. Machine learning methods that are among the best data science techniques and generally aims to learn from data helps us to better understand the underlying and hidden information within the data. Although classic machine learning techniques that rely heavily on physical memory works just fine with casual data, the big data brings its own challenges. One way to deal with this issue is to develop new data structures. The usual way to store the data is through vectorization, which can cause some information loss and or phenomena known as dimensional misery. Recently, tensors which are among the best tools for storing and representing the data have been used to better handle data. Different tensor decomposition methods can be deployed in order to achieve better results. In this thesis, based on a newly-defined production operator, we have presented a tensor decomposition technique which transfers data into a new feature space, in which some useful information can be retrieved. Based on this decomposition, we have developed classification and clustering techniques that show superior performances compared to other state-of-the-art methods.