Skip to content

marginal

EQTLPredictor

Bases: Module

Predictor for eQTL models.

The challenge with eQTL analysis is that the SNF predictors can vary from gene to gene and that we must support donor-level effects as well as cell-level features. This requires a different form of forward function to keep us from running separate regressions across genes or copying the same patient-level information across all cells. For modeling the mean, the basic structure is

eta = W @ coefs["mean"]                     # shared block
    + U[donor_idx]                           # per-individual random intercept
    + gather(einsum('kjm,jm->kj', D, beta))  # genotype main effect
    + gather(einsum('kjm,jml->kjl', D, zeta)) # genotype interaction

Args: n_outcomes: number of genes (J) feature_dims: mapping with keys "mean" and "dispersion" -> number of shared covariate columns for each dosage: (K, J, K_max) individual x gene x SNP position tensor tracking genotypes mask: (J, K_max) indicator for valid SNP entries per gene. The issue is that some genes might only be located close to a small number of SNPs. interaction_dim: number of columns (L) in the genotype-interaction design, not counting the intercept column. 0 means no interactions. accumulator_names: Names of additional statistical summary statistics to store while accumulating each epoch. These are necessary for our posterior moment updates described in donor_moments.py. Appropriate values are,

        score = +l'       observed = -l''
        third = -l'''     fourth   = -l''''
Source code in scdesigner/src/scdesigner/base/marginal.py
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
class EQTLPredictor(nn.Module):
    """Predictor for eQTL models.

    The challenge with eQTL analysis is that the SNF predictors can vary from
    gene to gene and that we must support donor-level effects as well as
    cell-level features. This requires a different form of forward function to
    keep us from running separate regressions across genes or copying the same
    patient-level information across all cells. For modeling the mean, the basic
    structure is

        eta = W @ coefs["mean"]                     # shared block
            + U[donor_idx]                           # per-individual random intercept
            + gather(einsum('kjm,jm->kj', D, beta))  # genotype main effect
            + gather(einsum('kjm,jml->kjl', D, zeta)) # genotype interaction

    Args:
        n_outcomes: number of genes (J)
        feature_dims: mapping with keys "mean" and "dispersion" -> number of
            shared covariate columns for each
        dosage: (K, J, K_max) individual x gene x SNP position tensor tracking
            genotypes
        mask: (J, K_max) indicator for valid SNP entries per gene. The issue is
            that some genes might only be located close to a small number of SNPs.
        interaction_dim: number of columns (L) in the genotype-interaction
            design, not counting the intercept column. 0 means no interactions.
        accumulator_names: Names of additional statistical summary statistics to
            store while accumulating each epoch. These are necessary for our
            posterior moment updates described in donor_moments.py. Appropriate
            values are,

                score = +l'       observed = -l''
                third = -l'''     fourth   = -l''''
    """

    def __init__(
        self,
        n_outcomes: int,
        feature_dims: Dict[str, int],
        dosage: torch.Tensor,
        mask: torch.Tensor,
        interaction_dim: int,
        accumulator_names: Tuple[str, ...] = (),
        loss_fn: Optional[callable] = None,
        optimizer_class: Optional[callable] = torch.optim.AdamW,
        optimizer_kwargs: Optional[Dict] = None,
        device: Optional[torch.device] = None,
    ):
        super().__init__()
        self.n_outcomes = int(n_outcomes)
        self.feature_dims = dict(feature_dims)
        n_individuals, _, k_max = dosage.shape

        # Initialize the learnable parameters.
        self.coefs = nn.ParameterDict()
        for key in ("mean", "dispersion"):
            self.coefs[key] = nn.Parameter(torch.zeros(self.feature_dims[key], self.n_outcomes))
        self.beta = nn.Parameter(torch.zeros(self.n_outcomes, k_max))
        self.zeta = nn.Parameter(torch.zeros(self.n_outcomes, k_max, interaction_dim))
        self.U = nn.Parameter(torch.zeros(n_individuals, self.n_outcomes))
        self.reset_parameters()

        self.register_buffer("D", dosage)
        self.register_buffer("mask", mask)
        self.register_buffer("lam", torch.ones(self.n_outcomes))

        # Summary statistics used when defining the EM updates.
        valid_accumulators = ("score", "observed", "third", "fourth")
        requested = set(accumulator_names)
        self.accumulator_names = tuple(
            name for name in valid_accumulators if name in requested
        )
        for name in self.accumulator_names:
            self.register_buffer(
                f"{name}_accum", torch.zeros(n_individuals, self.n_outcomes)
            )

        # Summary statistics used to assess the convergence of the EN step
        self.register_buffer("mode_gap", torch.zeros(self.n_outcomes))
        self.register_buffer("invalid_variances", torch.zeros((), dtype=torch.long))
        self.interaction_dim = int(interaction_dim)

        # Only used when simulating new donors. Off by default.
        self.register_buffer("U_sim", torch.zeros(n_individuals, self.n_outcomes))
        self.use_simulated_donor_effects = False

        self.loss_fn = loss_fn
        self.to(get_device(device))

        optimizer_kwargs = dict(optimizer_kwargs or {})
        optimizer_kwargs.setdefault("lr", 0.01)
        filtered_kwargs = _filter_kwargs(
            optimizer_kwargs, DEFAULT_ALLOWED_KWARGS['optimizer']
        )
        self.optimizer = optimizer_class(self.parameters(), **filtered_kwargs)

    @property
    def accumulators(self) -> Dict[str, torch.Tensor]:
        """Return the epoch accumulators as a dictionary keyed by derivative."""
        return {
            name: getattr(self, f"{name}_accum")
            for name in self.accumulator_names
        }

    def zero_accumulators(self):
        """Reset all per-epoch accumulators."""
        with torch.no_grad():
            for accumulator in self.accumulators.values():
                accumulator.zero_()

    def reset_parameters(self):
        for p in self.coefs.values():
            nn.init.normal_(p, mean=0.0, std=1e-4)

    def forward(self, obs_dict: Dict[str, torch.Tensor]) -> Dict[str, torch.Tensor]:
        """Forward pass to assemble mean prediction

        obs_dict must contain:
            "mean": (B, p) shared mean design
            "dispersion": (B, p_disp) shared dispersion design
            "genotype": (B, 1) integer-valued donor code column
            "interaction": (B, L + 1) genotype-interaction design (only read
                when interaction_dim > 0). Its first column is the intercept,
                which we drop so that zeta isn't collinear with beta.
        """
        donor_idx = obs_dict["genotype"][:, 0].long()

        # Introduce the donor-level random intercepts.
        eta = obs_dict["mean"] @ self.coefs["mean"]
        donor_effects = self.U_sim if self.use_simulated_donor_effects else self.U
        eta = eta + donor_effects[donor_idx]

        # Construct the genotype effects.
        donors, inverse = torch.unique(donor_idx, return_inverse=True)
        D_masked = self.D[donors] * self.mask.unsqueeze(0)
        M = torch.einsum("kjm,jm->kj", D_masked, self.beta)
        eta = eta + M[inverse]

        if self.interaction_dim > 0:
            V = obs_dict["interaction"][:, 1:] # drop intercept
            A = torch.einsum("kjm,jml->kjl", D_masked, self.zeta)
            eta = eta + torch.einsum("bjl,bl->bj", A[inverse], V)

        disp = obs_dict["dispersion"] @ self.coefs["dispersion"]
        return {"mean": torch.exp(eta), "dispersion": torch.exp(disp)}

accumulators property

Return the epoch accumulators as a dictionary keyed by derivative.

forward(obs_dict)

Forward pass to assemble mean prediction

obs_dict must contain: "mean": (B, p) shared mean design "dispersion": (B, p_disp) shared dispersion design "genotype": (B, 1) integer-valued donor code column "interaction": (B, L + 1) genotype-interaction design (only read when interaction_dim > 0). Its first column is the intercept, which we drop so that zeta isn't collinear with beta.

Source code in scdesigner/src/scdesigner/base/marginal.py
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
def forward(self, obs_dict: Dict[str, torch.Tensor]) -> Dict[str, torch.Tensor]:
    """Forward pass to assemble mean prediction

    obs_dict must contain:
        "mean": (B, p) shared mean design
        "dispersion": (B, p_disp) shared dispersion design
        "genotype": (B, 1) integer-valued donor code column
        "interaction": (B, L + 1) genotype-interaction design (only read
            when interaction_dim > 0). Its first column is the intercept,
            which we drop so that zeta isn't collinear with beta.
    """
    donor_idx = obs_dict["genotype"][:, 0].long()

    # Introduce the donor-level random intercepts.
    eta = obs_dict["mean"] @ self.coefs["mean"]
    donor_effects = self.U_sim if self.use_simulated_donor_effects else self.U
    eta = eta + donor_effects[donor_idx]

    # Construct the genotype effects.
    donors, inverse = torch.unique(donor_idx, return_inverse=True)
    D_masked = self.D[donors] * self.mask.unsqueeze(0)
    M = torch.einsum("kjm,jm->kj", D_masked, self.beta)
    eta = eta + M[inverse]

    if self.interaction_dim > 0:
        V = obs_dict["interaction"][:, 1:] # drop intercept
        A = torch.einsum("kjm,jml->kjl", D_masked, self.zeta)
        eta = eta + torch.einsum("bjl,bl->bj", A[inverse], V)

    disp = obs_dict["dispersion"] @ self.coefs["dispersion"]
    return {"mean": torch.exp(eta), "dispersion": torch.exp(disp)}

zero_accumulators()

Reset all per-epoch accumulators.

Source code in scdesigner/src/scdesigner/base/marginal.py
890
891
892
893
894
def zero_accumulators(self):
    """Reset all per-epoch accumulators."""
    with torch.no_grad():
        for accumulator in self.accumulators.values():
            accumulator.zero_()

GLMPredictor

Bases: Module

GLM-style predictor with arbitrary named parameters.

Args: n_outcomes: number of model outputs (e.g. genes) feature_dims: mapping from param name -> number of covariate features link_fns: optional mapping from param name -> callable(link) applied to linear predictor

The module will create one coefficient matrix per named parameter with shape (n_features_for_param, n_outcomes) and expose them as Parameters under self.coefs[param_name].

Source code in scdesigner/src/scdesigner/base/marginal.py
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
class GLMPredictor(nn.Module):
    """GLM-style predictor with arbitrary named parameters.

    Args:
        n_outcomes: number of model outputs (e.g. genes)
        feature_dims: mapping from param name -> number of covariate features
        link_fns: optional mapping from param name -> callable(link) applied to linear predictor

    The module will create one coefficient matrix per named parameter with shape
    (n_features_for_param, n_outcomes) and expose them as Parameters under
    `self.coefs[param_name]`.
    """
    def __init__(
        self,
        n_outcomes: int,
        feature_dims: Dict[str, int],
        link_fns: Dict[str, callable] = None,
        loss_fn: Optional[callable] = None,
        optimizer_class: Optional[callable] = torch.optim.AdamW,
        optimizer_kwargs: Optional[Dict] = None,
        device: Optional[torch.device] = None,
    ):
        super().__init__()
        self.n_outcomes = int(n_outcomes)
        self.feature_dims = dict(feature_dims)
        self.param_names = list(self.feature_dims.keys())

        self.link_fns = link_fns or {k: torch.exp for k in self.param_names}
        self.coefs = nn.ParameterDict()
        for key, dim in self.feature_dims.items():
            self.coefs[key] = nn.Parameter(torch.zeros(dim, self.n_outcomes))
        self.reset_parameters()

        self.loss_fn = loss_fn
        self.to(get_device(device))

        optimizer_kwargs = dict(optimizer_kwargs or {})
        optimizer_kwargs.setdefault("lr", 0.01)
        filtered_kwargs = _filter_kwargs(
            optimizer_kwargs, DEFAULT_ALLOWED_KWARGS['optimizer']
        )
        self.optimizer = optimizer_class(self.parameters(), **filtered_kwargs)

    def reset_parameters(self):
        for p in self.coefs.values():
            nn.init.normal_(p, mean=0.0, std=1e-4)

    def forward(self, obs_dict: Dict[str, torch.Tensor]) -> Dict[str, torch.Tensor]:
        """
        Forward Pass for Given Covariates

        obs_dict : Dict of (str -> torch.Tensor)
            A dictionary of tensors, with one key/value pair per parameter.
            These tensors are the conditioning information to pass to the
            .predict() function of this distribution class. They are the
            numerical design matrices implied by the initializing formulas.
        """
        out = {}
        for name in self.param_names:
            x_beta = obs_dict[name] @ self.coefs[name]
            link = self.link_fns.get(name, torch.exp)
            out[name] = link(x_beta)
        return out

forward(obs_dict)

Forward Pass for Given Covariates

obs_dict : Dict of (str -> torch.Tensor) A dictionary of tensors, with one key/value pair per parameter. These tensors are the conditioning information to pass to the .predict() function of this distribution class. They are the numerical design matrices implied by the initializing formulas.

Source code in scdesigner/src/scdesigner/base/marginal.py
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
def forward(self, obs_dict: Dict[str, torch.Tensor]) -> Dict[str, torch.Tensor]:
    """
    Forward Pass for Given Covariates

    obs_dict : Dict of (str -> torch.Tensor)
        A dictionary of tensors, with one key/value pair per parameter.
        These tensors are the conditioning information to pass to the
        .predict() function of this distribution class. They are the
        numerical design matrices implied by the initializing formulas.
    """
    out = {}
    for name in self.param_names:
        x_beta = obs_dict[name] @ self.coefs[name]
        link = self.link_fns.get(name, torch.exp)
        out[name] = link(x_beta)
    return out

Marginal

Bases: ABC

A Feature-wise Marginal Model

This is a class for handling feature-wise (e.g., gene-level) modeling that ignores any correlation between features. For example, it can be used to model the relationship between experimental design features, such as cell type or treatment, and the parameters of a collection of negative binomial models (one for each gene).

These marginals can be plugged into a copula to build the complete scDesign3 simulator. Methods are expected for estimating model parameters, evaluating likelihoods on new samples, and generating new samples given conditioning information. Since these marginal models are intended to be used within a copula, they should also provide utilities for evaluating quantiles and computing cumulative distribution functions.

Parameters:

Name Type Description Default
formula dict or str

A dictionary or string specifying the relationship between the columns of an input data frame (adata.obs, adata.var, or similar attributes) and the parameters of the marginal model. If only a string is provided, then the means are allowed to depend on the design parameters, while all other parameters are treated as fixed. If a dictionary is provided, each key should correspond to a parameter. The string values should be in a format that can be parsed by the formulaic package. For example, '~ x' will ensure that the parameter varies linearly with X.

required
penalties sequence of CoefficientPenalty

Penalties P (:mod:scdesigner.distributions.penalties) added to the training minibatch losses.

()

Attributes:

Name Type Description
formula dict or str

A dictionary or string specifying the relationship between the columns of an input data frame (adata.obs, adata.var, or similar attributes) and the parameters of the marginal model. If only a string is provided, then the means are allowed to depend on the design parameters, while all other parameters are treated as fixed. If a dictionary is provided, each key should correspond to a parameter. The string values should be in a format that can be parsed by the formulaic package. For example, '~ x' will ensure that the parameter varies linearly with X.

penalties tuple of CoefficientPenalty
chunk_size int or None

For backed anndata objects, we read continguous blocks of rows at a time. This parameter governs the size of those chunks and is used in :meth:setup_data and for feature summaries (e.g., for regularization).

design Design

Design matrix factory initially adapted to the full obs data in :meth:setup_data and then applied to prediction and sampling.

feature_dims dict

A dictionary containing the number of predictors associated with each distributional parameter. Note that this number is repeated for every feature (e.g., gene) in the marginal model. This information is often useful for computing the complexity of the estimated model.

loader DataLoader

A torch DataLoader object that returns batches of data for use during training. This loader is constructed internally within the setup_data method. Enumerating this loader returns a tuple: the first element contains a tensor of feature measurements (y), and the second element is a dictionary of tensors containing predictors to use for each parameter (x, for each parameter theta(x)). This design is useful because the design matrices may differ between parameters of the marginal model, y | x ~ F_(theta(x))(y)

train_loader DataLoader

Loader used for marginal optimization. With early stopping enabled, this is a subset of :attr:loader; otherwise it points to :attr:loader.

validation_loader DataLoader or None

Held-out loader used to monitor unpenalized validation likelihood.

fit_history list of dict

Per-epoch training and validation losses, with best/stop indicators.

_validation_split_config tuple or None

Cached (val_frac, validation_seed), where val_frac is the fraction of observations held out for validation, and validation_seed controls the deterministic random split.

n_outcomes int

The number of features modeled by this marginal model. For example, this corresponds to the number of genes being simulated.

predict Module

A torch.nn.Module storing the relationship between predictors for each parameter and the predicted feature-wise outcomes. This module is expected to take the second element of the tuple defined by each batch and then predict a tensor with the same shape as the first element of the batch tuple.

predictor_names dict of list of str

A dictionary whose keys are the parameter names associated with this marginal model. The values for each key are the names of predictors in the design matrix implied by the associated formula. Note that these names may have been expanded from the original formula specification. For example, if cell_type is included in the formula, then the predictor names will include the unique levels of cell_type as separate columns in the design matrix and therefore as separate elements in this list.

parameters dict of pandas.DataFrame

A dictionary whose keys are the parameter names associated with this marginal model. The values for each key are pandas DataFrames storing the fitted parameter values. The rows of each DataFrame are the experimental features specified by the associated formula object (the rownames are the same as those in predictor_names). The columns are features that are being predicted.

device device

The device (e.g., cpu, mps, cuda) used by the model and data loaders. You can set this when initializing the model and change it with :meth:to.

Examples:

>>> class DummyModel(Marginal):
...     def fit(self):
...         pass
...
...     def likelihood(self, batch: Tuple[torch.Tensor, Dict[str, torch.Tensor]]):
...         pass
...
...     def uniformize(self, y: torch.Tensor, x: Dict[str, torch.Tensor]):
...         pass
...
...     def invert(self, u: torch.Tensor, x: Dict[str, torch.Tensor]):
...         pass
...
...     def setup_optimizer(self):
...         pass
...
>>> model = DummyModel("~ cell_type")
>>> model.fit()
Source code in scdesigner/src/scdesigner/base/marginal.py
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
class Marginal(ABC):
    """
    A Feature-wise Marginal Model

    This is a class for handling feature-wise (e.g., gene-level) modeling that
    ignores any correlation between features. For example, it can be used to
    model the relationship between experimental design features, such as cell
    type or treatment, and the parameters of a collection of negative binomial
    models (one for each gene).

    These marginals can be plugged into a copula to build the complete scDesign3
    simulator. Methods are expected for estimating model parameters, evaluating
    likelihoods on new samples, and generating new samples given conditioning
    information. Since these marginal models are intended to be used within a
    copula, they should also provide utilities for evaluating quantiles and
    computing cumulative distribution functions.

    Parameters
    ----------
    formula : dict or str
        A dictionary or string specifying the relationship between the columns
        of an input data frame (adata.obs, adata.var, or similar attributes) and
        the parameters of the marginal model. If only a string is provided,
        then the means are allowed to depend on the design parameters, while all
        other parameters are treated as fixed. If a dictionary is provided,
        each key should correspond to a parameter. The string values should be
        in a format that can be parsed by the formulaic package.  For example,
        '~ x' will ensure that the parameter varies linearly with X.
    penalties : sequence of CoefficientPenalty
        Penalties ``P`` (:mod:`scdesigner.distributions.penalties`) added to
        the training minibatch losses.

    Attributes
    ----------
    formula : dict or str
        A dictionary or string specifying the relationship between the columns
        of an input data frame (adata.obs, adata.var, or similar attributes) and
        the parameters of the marginal model. If only a string is provided,
        then the means are allowed to depend on the design parameters, while all
        other parameters are treated as fixed. If a dictionary is provided,
        each key should correspond to a parameter. The string values should be
        in a format that can be parsed by the formulaic package.  For example,
        '~ x' will ensure that the parameter varies linearly with X.

    penalties : tuple of CoefficientPenalty

    chunk_size : int or None
        For backed anndata objects, we read continguous blocks of rows at a
        time. This parameter governs the size of those chunks and is used in
        :meth:`setup_data` and for feature summaries (e.g., for regularization).

    design : Design
        Design matrix factory initially adapted to the full `obs` data in
        :meth:`setup_data` and then applied to prediction and sampling.

    feature_dims : dict
        A dictionary containing the number of predictors associated with each
        distributional parameter.  Note that this number is repeated for every
        feature (e.g., gene) in the marginal model.  This information is often
        useful for computing the complexity of the estimated model.

    loader : torch.utils.data.DataLoader
        A torch DataLoader object that returns batches of data for use during
        training. This loader is constructed internally within the setup_data
        method. Enumerating this loader returns a tuple: the
        first element contains a tensor of feature measurements (y), and the
        second element is a dictionary of tensors containing predictors to use
        for each parameter (x, for each parameter theta(x)). This design is
        useful because the design matrices may differ between parameters of the
        marginal model, y | x ~ F_(theta(x))(y)

    train_loader : torch.utils.data.DataLoader
        Loader used for marginal optimization. With early stopping enabled,
        this is a subset of :attr:`loader`; otherwise it points to
        :attr:`loader`.

    validation_loader : torch.utils.data.DataLoader or None
        Held-out loader used to monitor unpenalized validation likelihood.

    fit_history : list of dict
        Per-epoch training and validation losses, with best/stop indicators.

    _validation_split_config : tuple or None
        Cached ``(val_frac, validation_seed)``, where ``val_frac`` is
        the fraction of observations held out for validation, and
        ``validation_seed`` controls the deterministic random split.

    n_outcomes : int
        The number of features modeled by this marginal model. For example,
        this corresponds to the number of genes being simulated.

    predict : nn.Module
        A torch.nn.Module storing the relationship between predictors for each
        parameter and the predicted feature-wise outcomes. This module is expected
        to take the second element of the tuple defined by each batch and then
        predict a tensor with the same shape as the first element of the batch
        tuple.

    predictor_names : dict of list of str
        A dictionary whose keys are the parameter names associated with this
        marginal model. The values for each key are the names of predictors in
        the design matrix implied by the associated formula. Note that these
        names may have been expanded from the original formula specification.
        For example, if cell_type is included in the formula, then the predictor
        names will include the unique levels of cell_type as separate columns in
        the design matrix and therefore as separate elements in this list.

    parameters : dict of pandas.DataFrame
        A dictionary whose keys are the parameter names associated with this
        marginal model. The values for each key are pandas DataFrames storing
        the fitted parameter values. The rows of each DataFrame are the
        experimental features specified by the associated formula object (the
        rownames are the same as those in predictor_names). The columns are
        features that are being predicted.

    device : torch.device
        The device (e.g., cpu, mps, cuda) used by the model and data loaders.
        You can set this when initializing the model and change it with
        :meth:`to`.

    Examples
    --------
    >>> class DummyModel(Marginal):
    ...     def fit(self):
    ...         pass
    ...
    ...     def likelihood(self, batch: Tuple[torch.Tensor, Dict[str, torch.Tensor]]):
    ...         pass
    ...
    ...     def uniformize(self, y: torch.Tensor, x: Dict[str, torch.Tensor]):
    ...         pass
    ...
    ...     def invert(self, u: torch.Tensor, x: Dict[str, torch.Tensor]):
    ...         pass
    ...
    ...     def setup_optimizer(self):
    ...         pass
    ...
    >>> model = DummyModel("~ cell_type")
    >>> model.fit()
    """
    def __init__(
        self,
        formula: Union[Dict, str],
        device: Optional[torch.device] = None,
        *,
        penalties: Sequence = (),
    ):
        self.formula = formula
        self.penalties = tuple(penalties)
        self._bound_penalties = ()
        self.chunk_size = None
        self.design = None
        self.feature_dims = None
        self.loader = None
        self.train_loader = None
        self.validation_loader = None
        self.fit_history = []
        self._validation_split_config = None
        self.best_epoch = None
        self.best_validation_loss = None
        self.stopped_epoch = None
        self.n_outcomes = None
        self.predict = None
        self.predictor_names = None
        self.parameters = None
        self.device = get_device(device)

    ##########################################################################
    ## data loading helpers
    ##########################################################################

    def setup_data(
        self,
        adata: AnnData,
        batch_size: int = 1024,
        *,
        chunk_size: Optional[int] = None,
        observation_transform=None,
        **kwargs,
    ):
        """Set up the dataloader for the AnnData object.

        The simulator class definition doesn’t actually require any particular
        template dataset. This is helpful for reasoning about simulators
        abstractly, but when we actually want to estimate parameters, we need a
        template. This method takes an input template dataset and adds
        attributes to the simulator object for later estimation and sampling
        steps.

        Parameters
        ----------
        adata : AnnData
            This is the object on which we want to estimate the simulator. This
            serves as the template for all downstream fitting.
        batch_size : int
            The number of sample to return on each call of the data loader.
            Defaults to 1024.
        chunk_size : int or None
            For backed anndata objects, we read continguous blocks of rows at a
            time. This parameter governs the size of those chunks and is used in
            :meth:`setup_data` and for feature summaries (e.g., for regularization).
        observation_transform : SpatialDesign, optional
            Transform that appends spatial basis columns to ``adata.obs``.
        **kwargs : Any
            Other keyword arguments passed to data loader construction. Any
            argument recognized by the PyTorch DataLoader can be passed in here.

        Returns
        -------
        None
            This method doesn't return anything but modifies the self.adata,
            self.design, self.loader, self.n_outcomes, self.feature_dims, and
            self.predictor_names attributes.
        """
        # keep a reference to the AnnData for later use (e.g., var_names)
        self.adata = adata
        self.chunk_size = chunk_size

        # learn the design matrix from the obs object, and initialize an
        # associated loader
        self.design = fit_design(
            adata.obs, self.formula, transform=observation_transform,
            chunked=resolve_chunk_size(adata, chunk_size) is not None,
        )
        kwargs["device"] = self.device
        self.loader = adata_loader(
            adata, self.design, chunk_size=chunk_size, batch_size=batch_size, **kwargs
        )

        # other attributes updated during training
        X_batch, obs_batch = next(iter(self.loader))
        self.n_outcomes = X_batch.shape[1]
        self.feature_dims = {k: v.shape[1] for k, v in obs_batch.items()}
        self.predictor_names = self.design.predictor_names
        self.train_loader = self.loader
        self.validation_loader = None
        self.fit_history = []
        self._validation_split_config = None
        self.best_epoch = None
        self.best_validation_loss = None
        self.stopped_epoch = None
        self._bound_penalties = ()

    def setup_validation_split(
        self,
        val_frac: float = 0.1,
        validation_seed: int = 0,
    ) -> None:
        """Create train and validation loaders that split the full dataset.

        The full-data loader remains available as :attr:`loader` for downstream
        workflows such as copula fitting. When validation is disabled or the
        dataset is too small to split, :attr:`train_loader` points to
        :attr:`loader` and :attr:`validation_loader` is ``None``.
        """
        if self.loader is None:
            raise RuntimeError("self.loader is not set (call setup_data first)")
        if not 0 <= val_frac < 1:
            raise ValueError("val_frac must satisfy 0 <= val_frac < 1")

        split_config = (float(val_frac), int(validation_seed))
        if (
            self._validation_split_config == split_config
            and self.train_loader is not None
        ):
            return

        n_obs = len(self.loader.dataset)
        if val_frac == 0 or n_obs < 2:
            self.train_loader = self.loader
            self.validation_loader = None
            self._validation_split_config = split_config
            return

        rng = np.random.default_rng(validation_seed)
        indices = rng.permutation(n_obs)
        n_validation = int(np.ceil(n_obs * val_frac))
        n_validation = min(max(n_validation, 1), n_obs - 1)
        validation_indices = indices[:n_validation]
        train_indices = indices[n_validation:]

        self.train_loader = self._subset_loader(train_indices, training=True)
        self.validation_loader = self._subset_loader(validation_indices, training=False)
        self._validation_split_config = split_config

    def _subset_loader(self, indices: np.ndarray, training: bool) -> DataLoader:
        # Resort rows after setup_validation_split randomizes row order. This
        # ensures backed chunk reads don't have to revisit previous chunks
        if isinstance(self.loader.dataset, AnnDataDataset):
            indices = np.sort(indices)

        sampler_is_random = isinstance(self.loader.sampler, RandomSampler)
        return DataLoader(
            Subset(self.loader.dataset, indices.tolist()),
            batch_size=self.loader.batch_size,
            shuffle=training and sampler_is_random,
            num_workers=self.loader.num_workers,
            collate_fn=self.loader.collate_fn,
            drop_last=self.loader.drop_last,
            pin_memory=self.loader.pin_memory,
            timeout=self.loader.timeout,
            worker_init_fn=self.loader.worker_init_fn,
            multiprocessing_context=self.loader.multiprocessing_context,
            generator=self.loader.generator,
            prefetch_factor=self.loader.prefetch_factor,
            persistent_workers=self.loader.persistent_workers,
        )

    ##########################################################################
    ## device managmeent
    ##########################################################################
    @staticmethod
    def _move_tensors(value, device):
        if torch.is_tensor(value):
            return value.to(device)
        if isinstance(value, dict):
            return {key: Marginal._move_tensors(item, device) for key, item in value.items()}
        if isinstance(value, list):
            return [Marginal._move_tensors(item, device) for item in value]
        if isinstance(value, tuple):
            return tuple(Marginal._move_tensors(item, device) for item in value)
        return value

    def to(self, device):
        """Move the predictor, optimizer state, and loaded data to ``device``."""
        self.device = get_device(device)
        if self.predict is not None:
            self.predict.to(self.device)
            optimizer = getattr(self.predict, "optimizer", None)
            if optimizer is not None:
                for parameter, state in optimizer.state.items():
                    optimizer.state[parameter] = self._move_tensors(state, self.device)
        if self.loader is not None:
            self.loader.dataset.to(self.device)
        return self

    def _move_batch_to_device(self, batch):
        y, x = batch
        if y.device != self.device:
            y = y.to(self.device)
            x = {k: v.to(self.device) for k, v in x.items()}
        return y, x

    ##########################################################################
    ## early stopping checks
    ##########################################################################
    def _validation_loss(self) -> Optional[float]:
        if self.validation_loader is None:
            return None

        # save current mode of the model
        was_training = self.predict.training
        self.predict.eval()
        total_loss = 0.0
        total_entries = 0
        with torch.no_grad():
            for batch in self.validation_loader:
                y, x = self._move_batch_to_device(batch)
                total_loss += -self.likelihood((y, x)).sum().item()
                total_entries += y.numel()
        # restore the original mode of the model
        self.predict.train(was_training)

        # could happen when the drop_last option of the validation loader is True
        # and the validation set is smaller than the batch size
        if total_entries == 0:
            return None
        return total_loss / total_entries

    @staticmethod
    def _validation_improved(
        val_loss: float,
        best_loss: float,
        loss_tol: float,
    ) -> bool:
        """Check whether validation loss improved by a relative tolerance."""
        if not np.isfinite(val_loss):
            return False
        if not np.isfinite(best_loss):
            return True
        required_decrease = loss_tol * max(abs(best_loss), 1e-12)
        return best_loss - val_loss >= required_decrease

    def _validate_early_stopping_args(
        self,
        min_epochs: int,
        loss_tol: float,
        patience: int,
    ) -> None:
        if min_epochs < 0:
            raise ValueError("min_epochs must be nonnegative")
        if loss_tol < 0:
            raise ValueError("loss_tol must be nonnegative")
        if patience < 0:
            raise ValueError("patience must be nonnegative")

    ##########################################################################
    ## estimate model coefficients
    ##########################################################################
    def fit(
        self,
        max_epochs: int = 2000,
        verbose: bool = True,
        val_frac: float = 0.1,
        min_epochs: int = 10,
        loss_tol: float = 1e-4,
        patience: int = 40,
        validation_seed: int = 0,
        **kwargs,
    ):
        """Fit the marginal predictor using vanilla PyTorch training loop.

        This method runs stochastic gradient optimization using the template
        dataset defined by the setup_data method. The specific optimizer used
        can be modified with the setup_optimizer method and defaults to Adam.

        Note that, unlike `fit` in class `Simulator`, this method does not allow
        the template dataset as input. This requires `.setup_data()` to be
        called first. We want to give finer-grained control over the data
        loading and optimization in this class relative to the specific
        `Simulator` implementations, which are designed to be easy to run with
        as few steps as possible.

        Parameters
        ----------
        max_epochs : int
            The maximum number of epochs. This is the number of times we feed
            through our cells in the dataset.
        verbose : bool
            Should we print intermediate training outputs?
        val_frac : float
            Fraction of observations held out for validation. Set to 0 to
            disable early stopping and train on the full dataset.
        min_epochs : int
            Minimum number of completed epochs before early stopping is allowed.
        loss_tol : float
            Required relative decrease in validation loss to reset patience.
            For example, ``1e-4`` requires a 0.01% decrease from the current
            best validation loss.
        patience : int
            Number of non-improving validation epochs allowed after the model is
            trained for ``min_epochs``.
        validation_seed : int
            Seed controlling the deterministic validation split.

        Returns
        -------
        None
            This method doesn't return anything but modifies the self.parameters
            attribute with the trained model parameters.
        """
        # Set up validation split and model optimization state.
        self._bound_penalties = ()
        self._validate_early_stopping_args(min_epochs, loss_tol, patience)
        self.setup_validation_split(val_frac=val_frac, validation_seed=validation_seed)
        self.setup_optimizer(**kwargs)
        self._initialize_parameters(**kwargs)
        self._bind_penalties()

        # Initialize validation tracking used for early stopping.
        self.fit_history = []
        self.best_epoch = None
        self.best_validation_loss = None
        self.stopped_epoch = None
        best_loss = float("inf")
        best_state = None
        wait = 0

        train_loader = self.train_loader
        for epoch in range(1, max_epochs + 1):
            epoch_loss = 0.0
            n_entries = 0

            # Main training pass.
            self.predict.train()
            for batch in train_loader:
                y, x = self._move_batch_to_device(batch)

                self.predict.optimizer.zero_grad()
                loss = self._training_loss((y, x))
                loss.backward()
                self.predict.optimizer.step()

                epoch_loss += loss.item()
                n_entries += y.numel()
            self._on_epoch_end(epoch)

            avg_loss = epoch_loss / n_entries if n_entries > 0 else float("nan")
            val_loss = self._validation_loss()
            is_best = False
            stopped = False

            # Validation pass and early-stopping decision.
            if val_loss is not None:
                if self._validation_improved(val_loss, best_loss, loss_tol):
                    best_loss = val_loss
                    wait = 0
                    best_state = copy.deepcopy(self.predict.state_dict())
                    self.best_epoch = epoch
                    self.best_validation_loss = best_loss
                    is_best = True
                else:
                    wait += 1

                if epoch >= min_epochs and wait >= patience:
                    stopped = True
                    self.stopped_epoch = epoch

            self.fit_history.append(
                {
                    "epoch": epoch,
                    "train_loss": avg_loss,
                    "val_loss": val_loss,
                    "best": is_best,
                    "stopped": stopped,
                }
            )
            if verbose:
                msg = f"Epoch {epoch}/{max_epochs}, Loss: {avg_loss:.4f}"
                if val_loss is not None:
                    msg += f", Val Loss: {val_loss:.4f}"
                print(msg, end='\r')
            if stopped:
                break
        if verbose:
            print() # Maintain the loss output

        # Restore best validation checkpoint before formatting parameters.
        if best_state is not None:
            self.predict.load_state_dict(best_state)
        self.parameters = self.format_parameters()

    def _bind_penalties(self) -> None:
        """Bind :attr:`penalties` to the current predictor's coefficients.

        Some of the penalties need to look over the gene-level means (these are
        the penalities with .uses_feature_means). We check that this is actually
        needed before computing it.
        """
        feature_means = None
        if any(spec.uses_feature_means for spec in self.penalties):
            feature_means = summarize_features(self.adata, chunk_size=self.chunk_size).mean
        self._bound_penalties = tuple(
            spec.bind(self.predictor_names, self.predict.coefs, feature_means)
            for spec in self.penalties
        )

    def _training_loss(
        self, batch: Tuple[torch.Tensor, Dict[str, torch.Tensor]]
    ) -> torch.Tensor:
        """``predict.loss_fn(batch)`` plus each bound penalty ``P``."""
        loss = self.predict.loss_fn(batch)
        for penalty in self._bound_penalties:
            loss = loss + penalty.value()
        return loss

    @property
    def fit_history_df(self) -> pd.DataFrame:
        """Return the fit history as a pandas DataFrame."""
        if len(self.fit_history) == 0:
            raise ValueError("fit_history is empty. Call fit() before accessing fit_history_df.")
        return pd.DataFrame(self.fit_history)

    ##########################################################################
    ## printing and summarizing model fits
    ##########################################################################

    def format_parameters(self):
        """Convert fitted coefficient tensors into pandas DataFrames.

        Returns:
            dict: mapping from parameter name -> pandas.DataFrame with rows
                corresponding to predictor column names (from
                `self.predictor_names[param]`) and columns corresponding to
                `self.adata.var_names` (gene names). The values are moved to
                CPU and converted to numpy floats.
        """
        var_names = list(self.adata.var_names)

        dfs = {}
        for param, tensor in self.predict.coefs.items():
            coef_np = tensor.detach().cpu().numpy()
            row_names = list(self.predictor_names[param])
            dfs[param] = pd.DataFrame(coef_np, index=row_names, columns=var_names)
        return dfs

    def num_params(self):
        """Return the number of parameters.

        Count the number of parameters in the marginal simulator. Usually this
        is just the number of predictors times the number of genes, because we
        use a linear model. However, in specific implementations, it’s possible
        to use more flexible models, in which case the number of parameters
        would increase.
        """
        if self.predict is None:
            return 0
        return sum(p.numel() for p in self.predict.parameters() if p.requires_grad)

    ##########################################################################
    ## optimizer and sampling helpers
    ##########################################################################

    @abstractmethod
    def setup_optimizer(self, **kwargs):
        raise NotImplementedError

    def _initialize_parameters(self, **kwargs):
        """Initialize fitted coefficients after train/validation loaders exist.

        Distribution subclasses can override this hook when initialization
        depends on the training data. The default GLM parameter initialization
        from :class:`GLMPredictor` is sufficient for distributions that do not
        need a data-dependent initializer.
        """
        return None

    def _on_epoch_end(self, epoch: int):
        """Run after training but validation for each epoch.

        By default, nothing happens, but for some models, we might want to have
        an estimation step after intermediate parameter updates. For example,
        this can be used to define an E-step in an EM algorithm.
        """
        return None

    @abstractmethod
    def likelihood(self, batch: Tuple[torch.Tensor, Dict[str, torch.Tensor]]) -> torch.Tensor:
        """Compute the log-likelihood for a batch.

        The likelihood is used for maximum likelihood estimation. It is also
        used when computing AIC and BIC scores, which are important when
        choosing an appropriate model complexity.

        Parameters
        ----------
        batch : tuple of (torch.Tensor, dict of str -> torch.Tensor)
            A tuple of gene expression (y) and experimental factors (x) used to
            evaluate the model likelihood. The first element of the tuple is a
            cells x genes tensor. The second is a a dictionary of tensors, with
            one key/value pair per parameter.  These tensors are the
            conditioning information to pass to the .predict() function of this
            distribution class. They are the numerical design matrices implied
            by the initializing formulas.

        Returns
        -------
        torch.Tensor
            A scalar containing the log-likelihood of the batch under the
            current model parameters.
        """
        raise NotImplementedError

    @abstractmethod
    def invert(self, u: torch.Tensor, x: Dict[str, torch.Tensor]):
        """Invert pseudoobservations.

        Return a quantile from the distribution. This handles the link between
        the marginal and the copula model. The idea is that the copula will
        generate pseudo-observations on the unit cube. All the values will be
        between zero and one. By calling invert, we transform these zero-to-one
        valued pseudo-observations into observations in the original data space;
        values between zero and one can be thought of like quantiles.

        Parameters
        ----------
        u : torch.Tensor
            Scalars between [0, 1] that specify the quantile level we want.
        x : Dict of (str -> torch.Tensor)
            A dictionary of tensors, with one key/value pair per parameter.
            These tensors are the conditioning information to pass to the
            .predict() function of this distribution class. They are the
            numerical design matrices implied by the initializing formulas.

        Returns
        -------
        z : torch.Tensor
            A tensor with dimension dim(u) x num_genes. Each row gives the
            requested quantile for the marginal distributions across all genes.
        """
        raise NotImplementedError

    @abstractmethod
    def uniformize(self, y: torch.Tensor, x: Dict[str, torch.Tensor]):
        """Uniformize using learned CDF.

        Apply a quantile/CDF transformation to observations y, accounting for
        conditioning variables x. This step is used in training the copula
        model. Since the copula needs to operate on the unit cube, we need to
        transform the original data onto the unit cube. This can be done by
        applying a CDF transformation.

        Parameters
        ----------
        y: torch.Tensor
            A cells x genes tensor with gene expression levels across all cells.
        x : Dict of (str -> torch.Tensor)
            A dictionary of tensors, with one key/value pair per parameter.
            These tensors are the conditioning information to pass to the
            .predict() function of this distribution class. They are the
            numerical design matrices implied by the initializing formulas.
        """
        raise NotImplementedError

fit_history_df property

Return the fit history as a pandas DataFrame.

fit(max_epochs=2000, verbose=True, val_frac=0.1, min_epochs=10, loss_tol=0.0001, patience=40, validation_seed=0, **kwargs)

Fit the marginal predictor using vanilla PyTorch training loop.

This method runs stochastic gradient optimization using the template dataset defined by the setup_data method. The specific optimizer used can be modified with the setup_optimizer method and defaults to Adam.

Note that, unlike fit in class Simulator, this method does not allow the template dataset as input. This requires .setup_data() to be called first. We want to give finer-grained control over the data loading and optimization in this class relative to the specific Simulator implementations, which are designed to be easy to run with as few steps as possible.

Parameters:

Name Type Description Default
max_epochs int

The maximum number of epochs. This is the number of times we feed through our cells in the dataset.

2000
verbose bool

Should we print intermediate training outputs?

True
val_frac float

Fraction of observations held out for validation. Set to 0 to disable early stopping and train on the full dataset.

0.1
min_epochs int

Minimum number of completed epochs before early stopping is allowed.

10
loss_tol float

Required relative decrease in validation loss to reset patience. For example, 1e-4 requires a 0.01% decrease from the current best validation loss.

0.0001
patience int

Number of non-improving validation epochs allowed after the model is trained for min_epochs.

40
validation_seed int

Seed controlling the deterministic validation split.

0

Returns:

Type Description
None

This method doesn't return anything but modifies the self.parameters attribute with the trained model parameters.

Source code in scdesigner/src/scdesigner/base/marginal.py
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
def fit(
    self,
    max_epochs: int = 2000,
    verbose: bool = True,
    val_frac: float = 0.1,
    min_epochs: int = 10,
    loss_tol: float = 1e-4,
    patience: int = 40,
    validation_seed: int = 0,
    **kwargs,
):
    """Fit the marginal predictor using vanilla PyTorch training loop.

    This method runs stochastic gradient optimization using the template
    dataset defined by the setup_data method. The specific optimizer used
    can be modified with the setup_optimizer method and defaults to Adam.

    Note that, unlike `fit` in class `Simulator`, this method does not allow
    the template dataset as input. This requires `.setup_data()` to be
    called first. We want to give finer-grained control over the data
    loading and optimization in this class relative to the specific
    `Simulator` implementations, which are designed to be easy to run with
    as few steps as possible.

    Parameters
    ----------
    max_epochs : int
        The maximum number of epochs. This is the number of times we feed
        through our cells in the dataset.
    verbose : bool
        Should we print intermediate training outputs?
    val_frac : float
        Fraction of observations held out for validation. Set to 0 to
        disable early stopping and train on the full dataset.
    min_epochs : int
        Minimum number of completed epochs before early stopping is allowed.
    loss_tol : float
        Required relative decrease in validation loss to reset patience.
        For example, ``1e-4`` requires a 0.01% decrease from the current
        best validation loss.
    patience : int
        Number of non-improving validation epochs allowed after the model is
        trained for ``min_epochs``.
    validation_seed : int
        Seed controlling the deterministic validation split.

    Returns
    -------
    None
        This method doesn't return anything but modifies the self.parameters
        attribute with the trained model parameters.
    """
    # Set up validation split and model optimization state.
    self._bound_penalties = ()
    self._validate_early_stopping_args(min_epochs, loss_tol, patience)
    self.setup_validation_split(val_frac=val_frac, validation_seed=validation_seed)
    self.setup_optimizer(**kwargs)
    self._initialize_parameters(**kwargs)
    self._bind_penalties()

    # Initialize validation tracking used for early stopping.
    self.fit_history = []
    self.best_epoch = None
    self.best_validation_loss = None
    self.stopped_epoch = None
    best_loss = float("inf")
    best_state = None
    wait = 0

    train_loader = self.train_loader
    for epoch in range(1, max_epochs + 1):
        epoch_loss = 0.0
        n_entries = 0

        # Main training pass.
        self.predict.train()
        for batch in train_loader:
            y, x = self._move_batch_to_device(batch)

            self.predict.optimizer.zero_grad()
            loss = self._training_loss((y, x))
            loss.backward()
            self.predict.optimizer.step()

            epoch_loss += loss.item()
            n_entries += y.numel()
        self._on_epoch_end(epoch)

        avg_loss = epoch_loss / n_entries if n_entries > 0 else float("nan")
        val_loss = self._validation_loss()
        is_best = False
        stopped = False

        # Validation pass and early-stopping decision.
        if val_loss is not None:
            if self._validation_improved(val_loss, best_loss, loss_tol):
                best_loss = val_loss
                wait = 0
                best_state = copy.deepcopy(self.predict.state_dict())
                self.best_epoch = epoch
                self.best_validation_loss = best_loss
                is_best = True
            else:
                wait += 1

            if epoch >= min_epochs and wait >= patience:
                stopped = True
                self.stopped_epoch = epoch

        self.fit_history.append(
            {
                "epoch": epoch,
                "train_loss": avg_loss,
                "val_loss": val_loss,
                "best": is_best,
                "stopped": stopped,
            }
        )
        if verbose:
            msg = f"Epoch {epoch}/{max_epochs}, Loss: {avg_loss:.4f}"
            if val_loss is not None:
                msg += f", Val Loss: {val_loss:.4f}"
            print(msg, end='\r')
        if stopped:
            break
    if verbose:
        print() # Maintain the loss output

    # Restore best validation checkpoint before formatting parameters.
    if best_state is not None:
        self.predict.load_state_dict(best_state)
    self.parameters = self.format_parameters()

format_parameters()

Convert fitted coefficient tensors into pandas DataFrames.

Returns: dict: mapping from parameter name -> pandas.DataFrame with rows corresponding to predictor column names (from self.predictor_names[param]) and columns corresponding to self.adata.var_names (gene names). The values are moved to CPU and converted to numpy floats.

Source code in scdesigner/src/scdesigner/base/marginal.py
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
def format_parameters(self):
    """Convert fitted coefficient tensors into pandas DataFrames.

    Returns:
        dict: mapping from parameter name -> pandas.DataFrame with rows
            corresponding to predictor column names (from
            `self.predictor_names[param]`) and columns corresponding to
            `self.adata.var_names` (gene names). The values are moved to
            CPU and converted to numpy floats.
    """
    var_names = list(self.adata.var_names)

    dfs = {}
    for param, tensor in self.predict.coefs.items():
        coef_np = tensor.detach().cpu().numpy()
        row_names = list(self.predictor_names[param])
        dfs[param] = pd.DataFrame(coef_np, index=row_names, columns=var_names)
    return dfs

invert(u, x) abstractmethod

Invert pseudoobservations.

Return a quantile from the distribution. This handles the link between the marginal and the copula model. The idea is that the copula will generate pseudo-observations on the unit cube. All the values will be between zero and one. By calling invert, we transform these zero-to-one valued pseudo-observations into observations in the original data space; values between zero and one can be thought of like quantiles.

Parameters:

Name Type Description Default
u Tensor

Scalars between [0, 1] that specify the quantile level we want.

required
x Dict of (str -> torch.Tensor)

A dictionary of tensors, with one key/value pair per parameter. These tensors are the conditioning information to pass to the .predict() function of this distribution class. They are the numerical design matrices implied by the initializing formulas.

required

Returns:

Name Type Description
z Tensor

A tensor with dimension dim(u) x num_genes. Each row gives the requested quantile for the marginal distributions across all genes.

Source code in scdesigner/src/scdesigner/base/marginal.py
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
@abstractmethod
def invert(self, u: torch.Tensor, x: Dict[str, torch.Tensor]):
    """Invert pseudoobservations.

    Return a quantile from the distribution. This handles the link between
    the marginal and the copula model. The idea is that the copula will
    generate pseudo-observations on the unit cube. All the values will be
    between zero and one. By calling invert, we transform these zero-to-one
    valued pseudo-observations into observations in the original data space;
    values between zero and one can be thought of like quantiles.

    Parameters
    ----------
    u : torch.Tensor
        Scalars between [0, 1] that specify the quantile level we want.
    x : Dict of (str -> torch.Tensor)
        A dictionary of tensors, with one key/value pair per parameter.
        These tensors are the conditioning information to pass to the
        .predict() function of this distribution class. They are the
        numerical design matrices implied by the initializing formulas.

    Returns
    -------
    z : torch.Tensor
        A tensor with dimension dim(u) x num_genes. Each row gives the
        requested quantile for the marginal distributions across all genes.
    """
    raise NotImplementedError

likelihood(batch) abstractmethod

Compute the log-likelihood for a batch.

The likelihood is used for maximum likelihood estimation. It is also used when computing AIC and BIC scores, which are important when choosing an appropriate model complexity.

Parameters:

Name Type Description Default
batch tuple of (torch.Tensor, dict of str -> torch.Tensor)

A tuple of gene expression (y) and experimental factors (x) used to evaluate the model likelihood. The first element of the tuple is a cells x genes tensor. The second is a a dictionary of tensors, with one key/value pair per parameter. These tensors are the conditioning information to pass to the .predict() function of this distribution class. They are the numerical design matrices implied by the initializing formulas.

required

Returns:

Type Description
Tensor

A scalar containing the log-likelihood of the batch under the current model parameters.

Source code in scdesigner/src/scdesigner/base/marginal.py
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
@abstractmethod
def likelihood(self, batch: Tuple[torch.Tensor, Dict[str, torch.Tensor]]) -> torch.Tensor:
    """Compute the log-likelihood for a batch.

    The likelihood is used for maximum likelihood estimation. It is also
    used when computing AIC and BIC scores, which are important when
    choosing an appropriate model complexity.

    Parameters
    ----------
    batch : tuple of (torch.Tensor, dict of str -> torch.Tensor)
        A tuple of gene expression (y) and experimental factors (x) used to
        evaluate the model likelihood. The first element of the tuple is a
        cells x genes tensor. The second is a a dictionary of tensors, with
        one key/value pair per parameter.  These tensors are the
        conditioning information to pass to the .predict() function of this
        distribution class. They are the numerical design matrices implied
        by the initializing formulas.

    Returns
    -------
    torch.Tensor
        A scalar containing the log-likelihood of the batch under the
        current model parameters.
    """
    raise NotImplementedError

num_params()

Return the number of parameters.

Count the number of parameters in the marginal simulator. Usually this is just the number of predictors times the number of genes, because we use a linear model. However, in specific implementations, it’s possible to use more flexible models, in which case the number of parameters would increase.

Source code in scdesigner/src/scdesigner/base/marginal.py
603
604
605
606
607
608
609
610
611
612
613
614
def num_params(self):
    """Return the number of parameters.

    Count the number of parameters in the marginal simulator. Usually this
    is just the number of predictors times the number of genes, because we
    use a linear model. However, in specific implementations, it’s possible
    to use more flexible models, in which case the number of parameters
    would increase.
    """
    if self.predict is None:
        return 0
    return sum(p.numel() for p in self.predict.parameters() if p.requires_grad)

setup_data(adata, batch_size=1024, *, chunk_size=None, observation_transform=None, **kwargs)

Set up the dataloader for the AnnData object.

The simulator class definition doesn’t actually require any particular template dataset. This is helpful for reasoning about simulators abstractly, but when we actually want to estimate parameters, we need a template. This method takes an input template dataset and adds attributes to the simulator object for later estimation and sampling steps.

Parameters:

Name Type Description Default
adata AnnData

This is the object on which we want to estimate the simulator. This serves as the template for all downstream fitting.

required
batch_size int

The number of sample to return on each call of the data loader. Defaults to 1024.

1024
chunk_size int or None

For backed anndata objects, we read continguous blocks of rows at a time. This parameter governs the size of those chunks and is used in :meth:setup_data and for feature summaries (e.g., for regularization).

None
observation_transform SpatialDesign

Transform that appends spatial basis columns to adata.obs.

None
**kwargs Any

Other keyword arguments passed to data loader construction. Any argument recognized by the PyTorch DataLoader can be passed in here.

{}

Returns:

Type Description
None

This method doesn't return anything but modifies the self.adata, self.design, self.loader, self.n_outcomes, self.feature_dims, and self.predictor_names attributes.

Source code in scdesigner/src/scdesigner/base/marginal.py
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
def setup_data(
    self,
    adata: AnnData,
    batch_size: int = 1024,
    *,
    chunk_size: Optional[int] = None,
    observation_transform=None,
    **kwargs,
):
    """Set up the dataloader for the AnnData object.

    The simulator class definition doesn’t actually require any particular
    template dataset. This is helpful for reasoning about simulators
    abstractly, but when we actually want to estimate parameters, we need a
    template. This method takes an input template dataset and adds
    attributes to the simulator object for later estimation and sampling
    steps.

    Parameters
    ----------
    adata : AnnData
        This is the object on which we want to estimate the simulator. This
        serves as the template for all downstream fitting.
    batch_size : int
        The number of sample to return on each call of the data loader.
        Defaults to 1024.
    chunk_size : int or None
        For backed anndata objects, we read continguous blocks of rows at a
        time. This parameter governs the size of those chunks and is used in
        :meth:`setup_data` and for feature summaries (e.g., for regularization).
    observation_transform : SpatialDesign, optional
        Transform that appends spatial basis columns to ``adata.obs``.
    **kwargs : Any
        Other keyword arguments passed to data loader construction. Any
        argument recognized by the PyTorch DataLoader can be passed in here.

    Returns
    -------
    None
        This method doesn't return anything but modifies the self.adata,
        self.design, self.loader, self.n_outcomes, self.feature_dims, and
        self.predictor_names attributes.
    """
    # keep a reference to the AnnData for later use (e.g., var_names)
    self.adata = adata
    self.chunk_size = chunk_size

    # learn the design matrix from the obs object, and initialize an
    # associated loader
    self.design = fit_design(
        adata.obs, self.formula, transform=observation_transform,
        chunked=resolve_chunk_size(adata, chunk_size) is not None,
    )
    kwargs["device"] = self.device
    self.loader = adata_loader(
        adata, self.design, chunk_size=chunk_size, batch_size=batch_size, **kwargs
    )

    # other attributes updated during training
    X_batch, obs_batch = next(iter(self.loader))
    self.n_outcomes = X_batch.shape[1]
    self.feature_dims = {k: v.shape[1] for k, v in obs_batch.items()}
    self.predictor_names = self.design.predictor_names
    self.train_loader = self.loader
    self.validation_loader = None
    self.fit_history = []
    self._validation_split_config = None
    self.best_epoch = None
    self.best_validation_loss = None
    self.stopped_epoch = None
    self._bound_penalties = ()

setup_validation_split(val_frac=0.1, validation_seed=0)

Create train and validation loaders that split the full dataset.

The full-data loader remains available as :attr:loader for downstream workflows such as copula fitting. When validation is disabled or the dataset is too small to split, :attr:train_loader points to :attr:loader and :attr:validation_loader is None.

Source code in scdesigner/src/scdesigner/base/marginal.py
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
def setup_validation_split(
    self,
    val_frac: float = 0.1,
    validation_seed: int = 0,
) -> None:
    """Create train and validation loaders that split the full dataset.

    The full-data loader remains available as :attr:`loader` for downstream
    workflows such as copula fitting. When validation is disabled or the
    dataset is too small to split, :attr:`train_loader` points to
    :attr:`loader` and :attr:`validation_loader` is ``None``.
    """
    if self.loader is None:
        raise RuntimeError("self.loader is not set (call setup_data first)")
    if not 0 <= val_frac < 1:
        raise ValueError("val_frac must satisfy 0 <= val_frac < 1")

    split_config = (float(val_frac), int(validation_seed))
    if (
        self._validation_split_config == split_config
        and self.train_loader is not None
    ):
        return

    n_obs = len(self.loader.dataset)
    if val_frac == 0 or n_obs < 2:
        self.train_loader = self.loader
        self.validation_loader = None
        self._validation_split_config = split_config
        return

    rng = np.random.default_rng(validation_seed)
    indices = rng.permutation(n_obs)
    n_validation = int(np.ceil(n_obs * val_frac))
    n_validation = min(max(n_validation, 1), n_obs - 1)
    validation_indices = indices[:n_validation]
    train_indices = indices[n_validation:]

    self.train_loader = self._subset_loader(train_indices, training=True)
    self.validation_loader = self._subset_loader(validation_indices, training=False)
    self._validation_split_config = split_config

to(device)

Move the predictor, optimizer state, and loaded data to device.

Source code in scdesigner/src/scdesigner/base/marginal.py
340
341
342
343
344
345
346
347
348
349
350
351
def to(self, device):
    """Move the predictor, optimizer state, and loaded data to ``device``."""
    self.device = get_device(device)
    if self.predict is not None:
        self.predict.to(self.device)
        optimizer = getattr(self.predict, "optimizer", None)
        if optimizer is not None:
            for parameter, state in optimizer.state.items():
                optimizer.state[parameter] = self._move_tensors(state, self.device)
    if self.loader is not None:
        self.loader.dataset.to(self.device)
    return self

uniformize(y, x) abstractmethod

Uniformize using learned CDF.

Apply a quantile/CDF transformation to observations y, accounting for conditioning variables x. This step is used in training the copula model. Since the copula needs to operate on the unit cube, we need to transform the original data onto the unit cube. This can be done by applying a CDF transformation.

Parameters:

Name Type Description Default
y Tensor

A cells x genes tensor with gene expression levels across all cells.

required
x Dict of (str -> torch.Tensor)

A dictionary of tensors, with one key/value pair per parameter. These tensors are the conditioning information to pass to the .predict() function of this distribution class. They are the numerical design matrices implied by the initializing formulas.

required
Source code in scdesigner/src/scdesigner/base/marginal.py
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
@abstractmethod
def uniformize(self, y: torch.Tensor, x: Dict[str, torch.Tensor]):
    """Uniformize using learned CDF.

    Apply a quantile/CDF transformation to observations y, accounting for
    conditioning variables x. This step is used in training the copula
    model. Since the copula needs to operate on the unit cube, we need to
    transform the original data onto the unit cube. This can be done by
    applying a CDF transformation.

    Parameters
    ----------
    y: torch.Tensor
        A cells x genes tensor with gene expression levels across all cells.
    x : Dict of (str -> torch.Tensor)
        A dictionary of tensors, with one key/value pair per parameter.
        These tensors are the conditioning information to pass to the
        .predict() function of this distribution class. They are the
        numerical design matrices implied by the initializing formulas.
    """
    raise NotImplementedError