Interactive Tools for Reliable Omics Visualization
Encoding and Efficiency
Different ways of encoding information are perceived with different accuracies. This means that any visualization implicitly prioritizes some comparisons over others.
Figure from [1].
Visualizing High Dimensions
Effective visualization becomes even more difficult when we measure many features.
There’s only so much we can show in a single view.
Motivation
Stacked bar plots are used in microbiome research to compare composition across groups or over time.
- \(x\)-axis position encodes samples
- Color encodes taxonomy
Figure from the Qiime2 paper [2].
Challenges
The basic limitation is that they can only show so many colors at a time. This makes it difficult to study rarer taxa.
Figure from the microbiomeViz documentation [3].
Challenges
The basic limitation is that they can only show so many colors at a time. This makes it difficult to study rarer taxa.
Figure from the Qiime2 documentation [2].
Approach
The basic idea of phylobar is to use this tree structure (context) to interactively paint (focus) the stacked bar plot.
Figure from the phylobar documentation [4].
Pseudobulk Data
This idea works whenever you cared about hierarchically organized compositions, like this COVID-19 immunology data from [5].
Notebooks
We can embed these visualizations into quarto documents.
Distortions in \(t\)-SNE and UMAP
Both \(t\)-SNE and UMAP introduce distortions. For example, they may not preserve density within different regions of the plot.
DensMAP is designed to address density distortion [6].
Distortions in \(t\)-SNE and UMAP
They can also fail to preserve the topology of the underlying data…
Example from [7].
Consequences
These distortions are not mere technical curiosities – they significantly impact scientific interpretation [7–9]. For example, they create misleading differences between cell types that are actually similar.
scDEED tunes hyperparameters to improve embedding reliability [10].
Precedent
Our approach is similar to Tissot’s indicatrix [11] except the embedding map doesn’t have an analytical formula.
Local Distortion
The RMetric method [12,13] estimates an embedding’s effect on a unit circle in the original space.
Fragmented Neighborhoods
For a more global summary of distortion, we fit a running median to true vs. embedding neighbor distances, flag outliers above \(\sigma \times \text{IQR}\), and mark points with many outlier links as “fragmented.”
Variable Density Swiss Roll
\(t\)-SNE (perplexity = 100) breaks the roll in the low-density region and artificially spreads the high density area.
Fragmented Neighborhoods
Fragmented Neighborhoods
Poorly Preserved Distances
Hydra Cell Atlas
Hydra Cell Atlas
Hydra Cell Atlas
Hydra Cell Atlas
Hydra Cell Atlas
Hydra Cell Atlas
DensMap vs. UMAP
Both variation in ellipses and fragmented neighborhood statistics can be used to compare competing algorithms, similarly to [10,14].
DensMap vs. UMAP
Both variation in ellipses and fragmented neighborhood statistics can be used to compare competing algorithms, similarly to [10,14].
Online App
You can experiment with this method in the browser at this link:
Focus-plus-context
The thread in both these projects is the focus-plus-context principle from data visualization [15].
We allowed the viewer to query for details relevant to the overall interpretation while maintaining relevant context.
RMetric Motivation
The RMetric algorithm [12,13] quantifies distortion geometrically. To motivate the algorithm, consider the distortion induced by mapping the sphere into latitude/longitude coordinates.
Half-Sphere Parameterization
Parameterize points on \(\mathcal{M}\) using spherical coordinates:
\[\begin{align*} \mathbf{x}\left(p\right) = \left(\cos\varphi\cos\theta, \cos\varphi \sin\theta, \sin\varphi\right) \end{align*}\]
The associated (latitude, longitude) embedding is
\[\begin{align*} \mathbf{z}\left(p\right) = \left(\theta\left(p\right), \varphi\left(p\right)\right). \end{align*}\]
Pushforward Metric
What does a small step in the embedding space correspond to in \(\mathcal{M}\)? The pushforward metric answers this,
\[\begin{align*} g_{ij} = \left\langle \frac{\partial\mathbf{x}}{\partial z^i}, \frac{\partial\mathbf{x}}{\partial z^j}\right\rangle_{\mathbb{R}^3} \end{align*}\]
This varies across \(p \in \mathcal{M}\) but we suppress it from the notation.
Pushforward Metric
In the sphere example, these derivatives can be directly computed and to obtain,
\[\begin{align*} G = \begin{pmatrix} \cos^2\varphi & 0 \\ 0 & 1 \end{pmatrix} \end{align*}\] Near the equator (\(\varphi \approx 0\)), a small step in \(\theta\) covers more distance than near the north pole (\(\varphi \approx \frac{\pi}{2}\)).
Dual Pushforward Metric
Alternatively, the gradients \(\nabla z^i\) of the embedding dimensions also reflect distortion.
Since the level sets of \(z^\theta\) become more compressed near the poles, the gradients \(\nabla z^\theta\) become larger there.
Dual Pushforward Metric
Alternatively, the gradients \(\nabla z^i\) of the embedding dimensions also reflect distortion.
In contrast, the gradients \(\nabla z^{\varphi}\) don’t depend on \(\varphi\).
RMetric Implementation
Let \(z_{k} \in \mathbb{R}^{N}\) be the \(k^{th}\) embedding dimension. Let \(L\) be the doubly-normalized graph Laplacian [17]. Compute
\[\begin{align*} H_{kk'}^{(\cdot)} := \frac{1}{2}\left[L\left(z_{k} \circ z_{k'}\right) - z_{k} \circ \left(L z_{k'}\right) - z_{k'} \circ \left(L z_{k}\right) \right] \in \mathbb{R}^{N} \end{align*}\]
The embedding distortion for sample \(n\) is given by \(H^{(n)} \in \mathbb{R}^{K \times K}\)
Interpretation
For a \(K\)-dimensional embedding map, it returns a collection of matrices \(H_{n} \in \mathbb{R}^{K \times K}\) showing how each dimension is dilated or compressed.
Interpretation
For a \(K\)-dimensional embedding map, it returns a collection of matrices \(H_{n} \in \mathbb{R}^{K \times K}\) showing how each dimension is dilated or compressed.
In this neighborhood, \(H_{n} \approx \begin{pmatrix} 0.5 & 0 \\ 0 & 0.5 \end{pmatrix}\).
Interpretation
For a \(K\)-dimensional embedding map, it returns a collection of matrices \(H_{n} \in \mathbb{R}^{K \times K}\) showing how each dimension is dilated or compressed.
In this neighborhood \(H_{n} \approx \begin{pmatrix} 2 & 0 \\ 0 & 0.5 \end{pmatrix}\).
Interpretation
For a \(K\)-dimensional embedding map, it returns a collection of matrices \(H_{n} \in \mathbb{R}^{K \times K}\) showing how each dimension is dilated or compressed.
In this neighborhood, \(H_{n} \approx \begin{pmatrix} 0.7 & 0.2 \\ 0.2 & 0.6 \end{pmatrix}\).
Interaction Gulfs
There are two classic challenges in interactive interfaces [18],
- Gulf of Execution: The gap between what you want to do and how you specify it in the system.
These challenges also apply to interactive data analysis.
Interaction Gulfs
There are two classic challenges in interactive interfaces [18],
- Gulf of Evaluation: The gap between what the system shows you and an understanding of what it represents.
These challenges also apply to interactive data analysis.
Comparison with LOO-map
The stability-based algorithm [8] gives a similar interpretation. But the visual encoding is more subtle, and the leave-one-out approach is time consuming even with approximations.
