A topological data analysis based classifier

Abstract

Topological Data Analysis (TDA) is an emerging field that aims to discover a dataset’s underlying topological information. TDA tools have been commonly used to create filters and topological descriptors to improve Machine Learning (ML) methods. This paper proposes a different TDA pipeline to classify balanced and imbalanced multi-class datasets without additional ML methods. Our proposed method was designed to solve multi-class and imbalanced classification problems with no data resampling preprocessing stage. The proposed TDA-based classifier (TDABC) builds a filtered simplicial complex on the dataset representing high-order data relationships. Following the assumption that a meaningful sub-complex exists in the filtration that approximates the data topology, we apply Persistent Homology (PH) to guide the selection of that sub-complex by considering detected topological features. We use each unlabeled point’s link and star operators to provide different-sized and multi-dimensional neighborhoods to propagate labels from labeled to unlabeled points. The labeling function depends on the filtration’s entire history of the filtered simplicial complex and it is encoded within the persistence diagrams at various dimensions. We select eight datasets with different dimensions, degrees of class overlap, and imbalanced samples per class to validate our method. The TDABC outperforms all baseline methods classifying multi-class imbalanced data with high imbalanced ratios and data with overlapped classes. Also, on average, the proposed method was better than K Nearest Neighbors (KNN) and weighted KNN and behaved competitively with Support Vector Machine and Random Forest baseline classifiers in balanced datasets.

Appendix 1: Implementation details

This Section provides some details about implementing our TDA-based classifier (TDABC). The TDABC was implemented on top of the Gudhi Library (Maria et al. 2014), Giotto Library (Pérez et al. 2021), Ripser (Bauer 2021; Zhang et al. 2020) to solve the computational topology aspects such as Simplicial Complexes and Persistent Homology (PH). Sci-kit learn (Pedregosa et al. 2012) for the Machine Learning algorithms such as baseline classifiers and PCA and TSNE dimensionality reduction methods. Numpy (Harris et al. 2020) for multi-dimensional arrays manipulation. UMAP-learn for UMAP-based dimensionality reduction (McInnes et al. 2020). HDF5 (The HDF Group 1997-2022) to handle large data in primary and secondary memory. Matplotlib (Hunter 2007) for visualization purposes. The source code of our proposed TDABC is available on https://github.com/rolan2kn/TDABC-4-ADAC. The following sections cover different aspects of the TDABC implementation.

1.1 Build simplicial complexes

1.1.1 Maximal edge length

A p-cycle can be born at any time and live unaltered up to the maximum edge length, in which case the p-cycle will die or be divided into two topological features. By controlling the maximal edge length, we control the topological feature-length and the size of the simplicial complex, combinatorial on the number of points and the simplex dimension. Thus, we recommend using the mean distance of the distance matrix as the maximal edge length. Consequently, noise points will only affect those cycles with a diameter twice the mean distance and make the filtration robust.

1.1.2 Edge collapse

Edge collapsing in Gudhi must be performed on the 1-skeleton of the simplicial complex and then expand from 1-skeleton to build all high dimensional simplices up to a maximal dimension $q \ll |P|$. The Algorithm 4 computes a simplex tree using the edge collapsing method (Boissonnat and Pritam 2020). A collapsing coefficient is defined to be dependent on the maximal dimension q. However, it could be enhanced by repeatedly calling the $collapse\_edges$ method until the simplex tree no longer changes. We recommend applying a collapsing factor (obtained experimentally) computed as a function of the point cloud’s ambient dimension and the simplicial complex’s maximal dimension. Edge collapse in Gudhi and Giotto supported only flag complexes like the Vietoris Rips. Our method can be used in any simplicial complex with minimal variations, but each complex has its intricacies to optimize the consumption of time and space resources.

1.2 Computing link and filtration values

The association function, $\Psi _{i}$ from Definition 5 depends on the $Lk_{\mathcal {K}}$ operation. However, up to now, the Python interface of Gudhi Library (v.3.6.0) (Rouvreau 2022) does not have an implementation of the simplex link operation. Regardless, it can be derived from the star and co-face operators according to Definition 2.

In Gudhi, each q-simplex $\sigma \in \mathcal {K}$ is stored with its filtration value $f(\sigma )$. Thus, the $star(\mathcal {S}_\mathcal {K}, \sigma )$ is a function in $\mathcal {S}_\mathcal {K}$ which returns a 2-tuple set

$$\begin{aligned} \{(\mu , f(\mu )) \mid \mu \in St_\mathcal {K}(\sigma ) \}. \end{aligned}$$

This data structure makes it easy to recover the filtration values required to implement Eq. 10, Algorithm 2, and Algorithm 3.

1.3 Finding neighborhoods of external points

The Algorithm 2 has a lot of room for optimizations. The search for the closest points to x can be drastically enhanced by applying computational geometry algorithms and spatial/metric data structures. As a few examples: a pivot point could be selected from P and then built as a Vantage Point Tree (VP-tree), Ball tree, or M-tree, among others, to perform multidimensional indexing of all elements by their distance (or proximity) to the pivot (Samet 2006). Other metric space searching data structures like the Spatial Approximation Tree (a-tree) (Navarro 2002) could be applied by considering the same complex as support instead a Delaunay triangulation. It could be more efficient than other space partitioning approaches to solve neighboring queries in datasets with more than 20 dimensions. These data structures could help to find points likely to share a simplex with x reducing the computational cost in time complexity to $O(|P| \log { |P|})$ in the worst case.

1.4 Permutation invariance

Given a proximity function $h(\cdot , \cdot )$, labeled ($X_l$) and unlabeled ($X_u$) point sets both subsets of P. Our TDABC is invariant to permutations of $X_l$ and permutations of $X_u$ building every time the same type of filtered simplicial complex. Let $\mathcal {K}({M_P})$ be a filtered simplicial complex constructed using a distance matrix $M_P$ over a point set P. For any permutation $\pi (P)$ we obtain a distance matrix $M_{\pi (P)}$ and a filtered simplicial complex $\mathcal {K}({M_{\pi (P)}})$. We know that $M_p$ and $M_{\pi (P)}$ are equivalent matrices because both have the same number of rows and cols. We can turn one into another by elemental row and column transpositions since they were constructed with the same $h(\cdot , \cdot )$ function over the same point set P. Thence, $\mathcal {K}({M_{\pi (P)}})$ and $\mathcal {K}({M_{P}})$ are the same complexes up to labeling permutations because we can always use a simplicial map that applies the inverse permutation ${\pi ^{-1}(P)}$ to each element of $\pi (P)$ to obtain the corresponding element in P this is equivalent to perform the column and row transpositions to rename simplices in $\mathcal {K}({M_{\pi (P)}})$ to obtain $\mathcal {K}({M_{P}})$. The case of permutations in unlabeled points $X_u$ is straightforward since the unlabeled points do not contribute to labeling other unlabeled points. Therefore, no matter which permutation is applied, TDABC will label the same point (among permutations) with the same label.

Although the results of the TDABC remain consistent across different permutations, there is room for improvement in terms of time execution. Certain permutations result in faster computations compared to others, contributing to the observed difference in performance. One of the reasons for this discrepancy is the locality property, where elements with small distances between them are clustered together in the element collection. In such cases, finding similar elements requires less time as they are located in close proximity. On the other hand, when similar elements are scattered in arbitrary positions, their retrieval becomes more time-consuming, leading to the wastage of computational resources. To address this issue, we propose using space-filling curves and related data structures (refer to Sect. 3), such as Z-order and Hilbert-order. These techniques, similar to those employed in Gudhi (Maria et al. 2014), have been shown to accelerate the construction of complexes and streamline simplicial queries.

1.5 Topological information

We complement Sect. 4.3 by presenting more information regarding the selected filtration value on the Swissroll and Sphere Datasets. See Figs. 10, 11 and Table 4.

1.5.1 Swissroll topological information

See Fig. 10.

Fig. 10

Results of applying the selection function on the Swissroll Dataset: the confusion matrices (first column), barcodes with the chosen interval (second column), and the chosen sub-complex $\mathcal {K}_i$ (third column), each 0-simplex has a color representing its label, unlabeled points in black

1.5.2 Sphere topological information

See Fig. 11.

Fig. 11

Results of applying the selection function on the Sphere Dataset: the confusion matrices (first column), barcodes with the chosen interval (second column), and the chosen sub-complex $\mathcal {K}_i$ (third column), each 0-simplex has a color representing its label, unlabeled points in black

1.6 Extended results

We also conduct experiments increasing the number of unlabeled points. We perform TDABC with three different cross-validation configurations by taking the fold size to be NORMAL (10%), EXTREME (60%), and HYPER EXTREME (90%) as fold size. We present the results of the F1 metric in Swissroll and Sphere Datasets in Table 4.

Table 4 Results varying the number of unlabeled points

1.7 Complexity analysis

We utilize the simplex tree from Gudhi as the chosen data structure in our paper because of its capability to represent any type of simplicial complex. It is worth mentioning that Gudhi offers various other data structures, as explained in the paragraph, which are specifically designed for maximal simplices (simplices without cofaces) (Boissonnat et al. 2017; Boissonnat and Karthik 2018). However, for our purposes, we focus on the simplex tree due to its versatility and ability to handle a broad range of complex types.

Fig. 12

A simplicial complex on ten vertices and its simplex tree. The deepest node represents the tetrahedron of the complex. Every label position at a given depth is linked in a list, as illustrated in the case of label 5. Picture and caption were taken from (Boissonnat and Maria 2014)

The simplex tree data structure was presented by Clement Maria and Jean-Daniel Boisonnat (Boissonnat and Maria 2014). Let $\mathcal {S}_\mathcal {K}$ be a simplex tree representation of a simplicial complex $\mathcal {K}$. Let $\sigma$ be a q-simplex where $\sigma \in \mathcal {K}$. The operations $insert(\mathcal {S}_\mathcal {K}, \sigma )$, and $search(\mathcal {S}_\mathcal {K}, \sigma )$ have a time complexity $O(|\sigma | \cdot log |P|)$ by using red-black trees to represent sibling nodes (nodes sharing its father node, see Fig. 12). When using hashing functions to represent sibling nodes, the time complexity is reduced to $O(|\sigma |)$. Insertion of a q-simplex $\sigma$ with all its faces has a complexity of $O(2^{|\sigma |} \cdot |\sigma | \cdot log |P|)$. See (Boissonnat and Maria 2014) for a more detailed explanation.

Let $\tau = \{\tau _0, \tau _1, \cdots , \tau _j\}$ be a j-simplex, with $\tau \in \mathcal {K}$. Compute $St_\mathcal {K}(\tau )$ in a simplex tree $\mathcal {S}_\mathcal {K}$ is performed by the operation $Locate\_cofaces(\mathcal {S}_\mathcal {K}, \tau )$ (Boissonnat and Maria 2014). To locate all cofaces of $\tau$ in $\mathcal {S}_\mathcal {K}$, it is needed to find all occurrences of $\tau _j$ in nodes whose depth is greater than j, and navigate upwards on $\mathcal {S}_\mathcal {K}$ looking for remaining elements of $\tau$. Those paths where $\tau$ was completely found, will contain the cofaces of $\tau$. Traversing a path in a simplex tree has a worst-case time complexity of $O(q+1)$ with $q = dim(\mathcal {K})$. Lets $O(\mathcal {T}_{\tau _j}^{>j})$ be the time complexity to locate all nodes at a depth greater than j, which contains $\tau _j$. Accordingly, the worst case time complexity of $St_\mathcal {K}(\tau )$ is $O((q+1)\cdot \mathcal {T}_{\tau _j}^{>j})$. In Gudhi, every path from the root to any leaf defines a maximal simplex.

A few algorithms remain missing from this section like the Algorithm 1 to label a test point set. This algorithm could be implemented considering the explanations mentioned above. The computation of PH is done using the method provided by Gudhi. In Sect. 3.3, we define the selection of persistence interval after obtaining the PH. The Algorithm 4 builds a simplicial complex with distance matrix and edge collapses. Computing the distance matrix can be done in time $O(d\cdot |P|^2)$ with the brute force method, but it could be at most $O(|P|)$ in a massive parallelism platform (Ji and Wang 2022). The edge collapse method runs in time $O(n\cdot n_c \cdot k^2)$, with $n, n_c$ the number of edges on the input and output graphs, k is the maximal degree of a vertex.

Algorithm 1 includes computing PH, which has a worst-case time complexity of $O(|\mathcal {K}|^3)$, but in practice, it has a time complexity near linear. The selection function computation is linear on the number of persistence intervals $O(|D|)$. The inverse level-set function $f^{-1}(\cdot )$ that has a worst-case time complexity of $O((q + 1) \cdot \log |P|)$ the same time required to find a simplex in the simplex tree, since we need to locate the simplex to ask for its filtration value. Algorithm 2 has a time complexity of $O(|P| \cdot \log { |P|} \cdot |U| \cdot (q+1)\cdot \mathcal {T}_{\tau _j}^{>j})$; it is an output-sensitive algorithm which depends of the number of points inside the $(2\varepsilon )$-ball and for each point the complexity of the star. There is much room for optimizations by applying dynamic programming techniques since multiple star queries on the same dense regions have many non-disjoint solutions. Algorithm 3 finds label contributions to label an unlabeled point x by building an implicit minimal spanning tree on the connected component containing $Lk_{\mathcal {K}_i}(\{x\})$. By the time the tree is finished, we have visited O(M) nodes performing a link operation per node, where M can be, at most, the number of simplices on the connected component. If the entire complex is connected, $M = |K|$. Enqueue and dequeue operations on the priority queue Q have a time complexity of $O(\log {M})$ in Q. Therefore the time complexity is $O(q \cdot \mathcal {T}_{\tau _j}^{>j} \cdot M \cdot \log { M})$. Each node in this implicit tree has space complexity $O((q+2) \cdot w)$ bits, $(q+1)$ the maximal q-simplex cardinality plus one because of the priority. Since we have O(M) nodes, the total space complexity is $O((q+2) \cdot w \cdot M)$ bits.