class: title <div id="title"> Interactivity, Interpretability, and Generative Models </div> <div id="subtitle"> Kris Sankaran <br/> ksankaran@wisc.edu <br/> Lab: <a href="https://go.wisc.edu/pgb8nl">go.wisc.edu/pgb8nl</a> <br/> </div> <div id="subtitle_right"> 16 | June | 2024 <br/> INFORMS ALIO-ASOCIO <br/> Slides: <a href="https://go.wisc.edu/3u4m16">go.wisc.edu/3u4m16</a> </div> --- ### Generalist Models .pull-left[ 1. **Modern machine learning models are being designed to solve many problems simultaneously.** 1. Multimodal datasets are becoming the norm, and new systems allow navigation across many sources. 1. We are also seeing increasingly rich ways to interact with them. ] .pull-right[ <img src="figures/generalist_models.png"/> ] --- ### Generalist Models .pull-left[ 1. Modern machine learning models are being designed to solve many problems simultaneously. 1. **Multimodal datasets are becoming the norm, and new systems allow navigation across many sources.** 1. We are also seeing increasingly rich ways to interact with them. ] .pull-right[ <img src="figures/open_vocabulary.gif"/> ] --- ### Generalist Models .pull-left[ 1. Modern machine learning models are being designed to solve many problems simultaneously. 1. Multimodal datasets are becoming the norm, and new systems allow navigation across many sources. 1. **We are also seeing increasingly rich ways to interact with them.** ] .pull-right[ <img src="figures/image_editing.gif"/> ] --- ### What can go wrong? .center[ <img src="figures/bard-hallucination.webp" width=900/> ] --- # What Makes a Model Interpretable? <br/> .center[ <img src="figures/computer.png" width=350 style="position: absolute; left: 500px"/> ] --- # What Makes a Model Interpretable? <br/> .center[ <img src="figures/computer.png" width=350 style="position: absolute; left: 500px"/> ] <p style="font-size: 30px; position: absolute; left: 20px; top: 200px; width: 450px"> This is a difficult questions.... let's start with an easier one. </p> --- # What Makes a Visualization Good? <br/> .center[ <img src="figures/visualization.png" width=350 style="position: absolute; left: 450px"/> ] --- ### Key Properties .pull-left[ A good visualization is: 1. **Legible**: It omits extraneous, distracting elements. 1. **Annotated**: It shows data within the problem context. 1. **Information Dense**: It shows relevant variation efficiently. ] .pull-right[ <img src="figures/tufte.png" width=330/> ] --- ### Key Properties A good visualization is: 1. **Legible**: It omits extraneous, distracting elements. 1. **Annotated**: It shows data within the problem context. 1. **Information Dense**: It shows relevant variation efficiently. .center[ <img src="figures/tufte-2.png"/> ] --- ### Below-the-Surface More subtly, it should pay attention to: 1. **Data Provenance**: If we don't know the data sources, we should be skeptical or anything that's shown, no matter how compelling. 1. **Audience**: The effectiveness of a visualization is dependent on the visual vocabulary of its audience. 1. **Prioritization**: Every design emphasizes some comparisons over others. Are the "important" patterns visible? 1. **Interactivity**: Does it engage the reader's problem solving capacity? We should think about model interpretability with the same nuance that we think about data visualization. --- ### Vocabulary 1. **Interpretable Model**: A model that, by virtue of its design, is easy for its stakeholders to accurately describe and alter. 1. **Explainability Technique**: A method that shapes our mental models about black box systems. .center[ <img src="figures/black_box_flashlight.png" width=720/> ] --- ### Vocabulary 1. **Local Explanation**: An artifact for reasoning about individual predictions. 1. **Global Explanation**: An artifact for reasoning about an entire model. .center[ <img src="figures/explanation_types.png" width=800/> ] --- .center[ ## Interpretability Examples ] --- ### Hypothetical Study Problem: Imagine sampling longitudinal microbiome profiles from 500 study participants, some of whom eventually developed a disease. Can we discovery any microbiome-related risk factors? This simulation is motivated by microbiome studies of HIV risk (Gosmann et al., 2017). .center[ <img src="figures/simulated-data.svg" width=830/> ] --- ### Transformers .pull-left[ 1. A principle of deep learning is that end-to-end optimization is more general than expert design. 1. We can apply the GPT2 architecture to our problem, viewing a sequence of microbiome profiles like a sequence of words. ] .pull-right[ <img src="figures/transformers-analogy-2.png"/> ] --- ### Transformers .pull-left[ 1. A principle of deep learning is that end-to-end optimization is more general than expert design. 1. We can apply the GPT2 architecture to our problem, viewing a sequence of microbiome profiles like a sequence of words. ] .pull-right[ <img src="figures/transformer_analogy.png"/> ] --- ### Transformers .pull-left[ Applying a transformer model to the raw series, we reach a hold-out performance of ~ 83.2%, which is nearly as good as a model with knowledge of the true underlying features. ] .pull-right[ <img src="figures/transformer_probs.png"/> ] --- ### Embeddings In text data, we can understand context-dependent meaning by looking for clusters in the PCA of embeddings (Coenen et al., 2019). These represent a type of interaction. .center[ <img src="figures/bert_context.png" width=670/> ] --- ### Embeddings We can build the analogous visualization for our microbiome problem. Samples that are nearby in the embedding space are similar w.r.t. predictive features. .center[ <img src="figures/pca_comparison.svg" width=1400/> ] --- ### Interpolations Another common technique is to analyze linear interpolations in this space (Liu et al., 2019). This figure traces out the microbiome profiles between two samples. .center[ <img src="figures/species_21_interpolation.svg" width=940/> ] --- ### Concept Bottlenecks Alternatively, we can explain a decision by reducing the arbitrary feature space to a set of human-interpretable concepts (Koh et al., 2020). This is part of a larger body of work that attempts to establish shared language/representations for interacting with models. .center[ <img src="figures/koh_concept.png" width=750 style="position: absolute; top: 340px; left: 300px"/> ] --- ### Concept Bottlenecks We reconfigure our transformer model to first predict the concept label before making a final classification. .center[ <img src="figures/transformer-v-concepts.png" width=700/> ] --- ### Concept Bottlenecks .pull-left[ Performance is in fact slightly better than before (84%), and we also obtain concept labels to help us explain each instance's prediction. ] .pull-right[ <img src="figures/concept_probs.png"/> ] --- .center[ ## Interactivity ] --- ### Scientific Generative Models .pull-left[ 1. Simulators have emerged as a general problem-solving device across various domains, many of which now have rich, open-source libraries. 2. Where is the interface with statistics? - Experimental design, model building, and decision-making. ] .pull-right[ .center[ <img src="figures/esm.png"/> ] The E3SM is used for long-term climate projections. ] --- ### Scientific Generative Models .pull-left[ 1. Simulators have emerged as a general problem-solving device across various domains, many of which now have rich, open-source libraries. 2. Where is the interface with statistics? - Experimental design, model building, and decision-making. ] .pull-right[ .center[ <img src="figures/splatter.png"/> ] Splatter generates synthetic single-cell genomics data. ] --- ### Grammar of Generative Models Transparent simulators can be built by interactively composing simple modules. Probabilistic programming has streamlined the process. .pull-three-left[ <img src="figures/modules.jpeg" width=700/> ] .pull-three-right[ a. Regression <br/> b: Hierarchy <br/> c: Latent Structure <br/> d: Temporal Variation ] --- ### Discrepancy and Iterability By learning a discriminator to contrast real vs. simulated data, we can systematically improve the assumed generative mechanism. .center[ <img src="figures/iterability.jpeg" width=730/> ] --- ### Covasim Following the outbreak of COVID-19, the research community came together to build simulators that could inform pandemic response. * E.g., "What would happen if we held classes remotely for two weeks?" .center[ <img src="figures/covasim.png" width=700/> ] --- ### Covasim Covasim is an example of an agent-based model. Starting from local interaction rules, it lets us draw global inferences. <img src="figures/emulation.jpeg"/> Statistical emulators mimic the relationship between input hyperparameters and output data, substantially reducing the computational burden. --- ### Learn More * [Generative Models: An Interdisciplinary Perspective](https://doi.org/10.1146/annurev-statistics-033121-110134) * [Data Science Principles for Interpretable and Explainable AI](arxiv.org/abs/2405.10552) .pull-left[ <img src="figures/generative-qr.png" width=400/> ] .pull-right[ <img src="figures/interpretability-qr.png" width=375/> ] --- ### Thank you! Contact: [ksankaran@wisc.edu]() *Acknowledgments* * Lab Members: Margaret Thairu, Hanying Jiang, Shuchen Yan, Yuliang Peng, Kaiyan Ma, Kai Cui, Sam Merten, and Kobe Uko * Funding: NIGMS R01GM152744. --- ### References Coenen, A. et al. (2019). "Visualizing and Measuring the Geometry of BERT". In: _ArXiv_ abs/1906.02715. Gosmann, C. et al. (2017). "Lactobacillus‐Deficient Cervicovaginal Bacterial Communities Are Associated with Increased HIV Acquisition in Young South African Women". In: _Immunity_ 46, p. 29–37. Koh, P. W. et al. (2020). "Concept Bottleneck Models". In: _ArXiv_ abs/2007.04612. Liu, Y. et al. (2019). "Latent Space Cartography: Visual Analysis of Vector Space Embeddings". In: _Computer Graphics Forum_ 38. --- ### References Liu, Y. et al. (2019). "Latent Space Cartography: Visual Analysis of Vector Space Embeddings". In: _Computer Graphics Forum_ 38.