Pattern or Artifact? Interactively Exploring Embedding Quality with TRACE

TRACE¹ supports you in analyzing global and local quality 🕵🏽‍♀️ of two-dimensional embeddings, based on Regl-scatterplot² .

Installation

Data Preparation

The easiest way to load your data into TRACE is using the Dataset class to add embeddings and compute quality measures. This will create the necessary Anndata structure under the hood. Examples can be found in the notebooks of each dataset folder.

python trace_data = Dataset( hd_data=data, name="Gaussian Line", verbose=True, hd_metric="euclidean", )

1. Adding 2-dimensional embeddings

After preprocessing your data and computing a range of 2-dimensional embeddings using your favorite DR method, add the data and the embeddings to the data object:

```python

Repeat for each embedding

tracedata.addembedding( name= "tSNE (perplexity 30)", embedding = tsne_emb, category="tSNE", ) ```

2. Computing High-Dimensional Neighbors and Quality Measures

To provide snappy interactions in TRACE, the HD neighbors and a range of quality measures need to be precomputed. We use ANNOY to obtain the approximate neighbors and provide implementations of the following quality measures to be visualized via point colors in TRACE:

• neighborhood preservation measures the fraction of k high-dimensional neighbors that are preserved in the low-dimensional embedding.
• landmark distance correlation: Sampling landmark points using a random or kmeanss++ (supports only Euclidean distance) from the high-dimensional data. We then compute the pairwise distances between all landmarks in high-dimensional space and each embedding and the rank correlation of their distance vectors. Points that are not landmark points are colored according to their nearest landmark point in the embedding.
• random triplet accuracy quantifies the ratio of random triplets (i,j,k), where relative order of j and k with respect to i in the high-dimensional space is preserved in the embedding.
• point stability measures how much the distances between each point and a random sample of other points vary across all embeddings. If a point has a very different global or local position in the embeddings, the stability will be low.

To compute all available quality measures: python trace_data.compute_quality(filename="./gauss_line.h5ad", hd_metric="euclidean") trace_data.print_quality()

3. Add Dataset Configuration

To include a dataset in the dashboard you need to create a file /backend/data_configs.yaml following the examples in data_configs.yaml.template. For the Gaussian Line dataset adding the following lines would be sufficient: json "GaussLine": { "filepath": "../data/gauss_line/gauss_line.h5ad", "description": "Gaussian clusters shifted along a line from Böhm et al. (2022)", }

Example Datasets

Gaussian Line 🟢 🟠 🟣

A small example dataset that is included in the repository.

Mammoth 🦣

This dataset from Wang et al. can be downloaded from their PaCMAP repository. It then needs to be processed using the mammoth.ipynb notebook.

Single-Cell Mouse Data 🐁

The processed dataset of gene expressions from Guilliams et al. is not available online, please reach out if you are interested. A raw version is available under GSE192742.

Citation

TRACE was presented as a demo paper at ECML-PKDD 2024. If you find the tool useful and are using it in your research, we'd appreciate if you could cite our paper:

bibtex @inproceedings{heiter2024pattern, title={Pattern or Artifact? Interactively Exploring Embedding Quality with TRACE}, author={Heiter, Edith and Martens, Liesbet and Seurinck, Ruth and Guilliams, Martin and De Bie, Tijl and Saeys, Yvan and Lijffijt, Jefrey}, booktitle={Joint European Conference on Machine Learning and Knowledge Discovery in Databases}, pages={379--382}, year={2024}, organization={Springer} }

[1] TRACE stands for Two-dimensional representation Analysis and Comparison Engine
[2] Lekschas, Fritz. "Regl-Scatterplot: A Scalable Interactive JavaScript-based Scatter Plot Library." Journal of Open Source Software (2023)

⬆️ Back to top