Skip to content

composite

Composite simulator that combines multiple marginals with a Gaussian copula.

This module provides :class:CompositeCopula, a simulator that fits several marginal models and then couples their dependence structure with a :class:~scdesigner.copulas.StandardCopula.

CompositeCopula

Bases: SCD3Simulator

Composite simulator: multiple marginals + a shared Gaussian copula.

The composite simulator fits each marginal model independently on a (potentially different) subset of variables, and then fits a Gaussian copula on the merged uniformized outputs from all marginals to capture cross-feature dependence.

Each marginal is provided as a pair (sel, marginal) where:

  • sel selects which variables in adata the marginal is responsible for (e.g. a list of gene names, a single gene name).
  • marginal is an object implementing the marginal simulator interface

Parameters:

Name Type Description Default
marginals list

List of (sel, marginal) pairs.

required
copula_formula str

Formula passed to :class:~scdesigner.copulas.StandardCopula to determine copula grouping structure (e.g. "group ~ 1"). If None, uses the copula's default.

None

Attributes:

Name Type Description
marginals list

The provided marginal specifications.

copula StandardCopula

The fitted copula component.

template AnnData or None

Training dataset (set during :meth:fit).

parameters dict or None

Fitted parameters, with keys "marginal" and "copula".

merged_formula dict or None

Merged (prefixed) formula dictionary used to construct the copula data loader.

Examples:

Fit two marginal models on disjoint gene sets and then fit a copula:

>>> import numpy as np
>>> import pandas as pd
>>> from anndata import AnnData
>>> from scdesigner.simulators import CompositeCopula
>>> from scdesigner.distributions import NegBin, Poisson
>>>
>>> X = np.random.poisson(1.0, size=(100, 10)).astype(float)
>>> obs = pd.DataFrame({"cell_type": np.random.choice(["A", "B"], size=100)})
>>> adata = AnnData(X=X, obs=obs)
>>> adata.var_names = [f"g{i}" for i in range(adata.n_vars)]
>>>
>>> # Example selectors: first 5 genes vs last 5 genes
>>> sel1 = adata.var_names[:5].tolist()
>>> sel2 = adata.var_names[5:].tolist()
>>> m1 = NegBin(formula={"mean": "~ cell_type", "dispersion": "~ 1"})
>>> m2 = Poisson(formula={"mean": "~ cell_type"})
>>>
>>> composite = CompositeCopula([(sel1, m1), (sel2, m2)])
>>> composite.fit(adata, batch_size=256, verbose=False)
>>> params = composite.predict(adata.obs.iloc[:3], batch_size=3)
Source code in scdesigner/src/scdesigner/simulators/composite.py
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
class CompositeCopula(SCD3Simulator):
    """
    Composite simulator: multiple marginals + a shared Gaussian copula.

    The composite simulator fits each marginal model independently on a
    (potentially different) subset of variables, and then fits a Gaussian
    copula on the *merged* uniformized outputs from all marginals to capture
    cross-feature dependence.

    Each marginal is provided as a pair ``(sel, marginal)`` where:

    - ``sel`` selects which variables in ``adata`` the marginal is responsible
      for (e.g. a list of gene names, a single gene name).
    - ``marginal`` is an object implementing the marginal simulator interface

    Parameters
    ----------
    marginals : list
        List of ``(sel, marginal)`` pairs.
    copula_formula : str, optional
        Formula passed to :class:`~scdesigner.copulas.StandardCopula`
        to determine copula grouping structure (e.g. ``"group ~ 1"``). If
        ``None``, uses the copula's default.

    Attributes
    ----------
    marginals : list
        The provided marginal specifications.
    copula : StandardCopula
        The fitted copula component.
    template : AnnData or None
        Training dataset (set during :meth:`fit`).
    parameters : dict or None
        Fitted parameters, with keys ``"marginal"`` and ``"copula"``.
    merged_formula : dict or None
        Merged (prefixed) formula dictionary used to construct the copula data loader.

    Examples
    --------
    Fit two marginal models on disjoint gene sets and then fit a copula:

    >>> import numpy as np
    >>> import pandas as pd
    >>> from anndata import AnnData
    >>> from scdesigner.simulators import CompositeCopula
    >>> from scdesigner.distributions import NegBin, Poisson
    >>>
    >>> X = np.random.poisson(1.0, size=(100, 10)).astype(float)
    >>> obs = pd.DataFrame({"cell_type": np.random.choice(["A", "B"], size=100)})
    >>> adata = AnnData(X=X, obs=obs)
    >>> adata.var_names = [f"g{i}" for i in range(adata.n_vars)]
    >>>
    >>> # Example selectors: first 5 genes vs last 5 genes
    >>> sel1 = adata.var_names[:5].tolist()
    >>> sel2 = adata.var_names[5:].tolist()
    >>> m1 = NegBin(formula={"mean": "~ cell_type", "dispersion": "~ 1"})
    >>> m2 = Poisson(formula={"mean": "~ cell_type"})
    >>>
    >>> composite = CompositeCopula([(sel1, m1), (sel2, m2)])
    >>> composite.fit(adata, batch_size=256, verbose=False)
    >>> params = composite.predict(adata.obs.iloc[:3], batch_size=3)
    """
    def __init__(self, marginals: List,
                 copula_formula: Optional[str] = None,
                 estimator=None,
                 top_k: Optional[int] = None) -> None:
        """Create a composite simulator.

        Parameters
        ----------
        marginals : list
            List of ``(sel, marginal)`` pairs. See class docstring for details.
        copula_formula : str, optional
            Copula grouping formula passed to :class:`StandardCopula`.
        estimator : CovarianceEstimator, str, or None, optional
            Covariance estimator for the copula. See :class:`StandardCopula`.
        top_k : int, optional
            Model only the ``top_k`` most expressed genes jointly.
        """
        self.marginals = marginals
        self.copula = StandardCopula(
            copula_formula if copula_formula is not None else "~ 1",
            estimator=estimator,
            top_k=top_k,
        )
        self.template = None
        self.parameters = None
        self.merged_formula = None

    def _loader_kwargs(self, kwargs: dict) -> dict:
        """Build loader options from the shared component device."""
        return {**kwargs, "device": self.marginals[0][1].device}

    def to(self, device):
        """Move each component model to the ``device``."""
        for _, marginal in self.marginals:
            marginal.to(device)
        self.copula.to(device)
        return self

    def fit(
        self,
        adata: AnnData,
        verbose: bool = True,
        val_frac: float = 0.1,
        min_epochs: int = 10,
        loss_tol: float = 1e-4,
        patience: int = 40,
        validation_seed: int = 0,
        **kwargs,):
        """Fit all marginals and then fit the copula on merged uniforms.

        Parameters
        ----------
        adata : AnnData
            Training dataset.
        verbose : bool, optional
            Whether to print verbose output.
        val_frac : float
            Fraction of observations held out for each marginal validation
            split. Set to 0 to disable early stopping.
        min_epochs : int
            Minimum number of marginal epochs before early stopping is allowed.
        loss_tol : float
            Required relative validation-loss decrease to reset patience.
        patience : int
            Number of non-improving validation epochs allowed after warmup.
        validation_seed : int
            Seed controlling deterministic marginal validation splits.
        **kwargs
            Additional keyword arguments forwarded to marginal setup/fit methods
            and to the copula's ``setup_data`` call (e.g. ``batch_size``).
            Copula settings like ``estimator`` and ``top_k`` are given when the
            simulator is constructed, not here.
        """
        kwargs = self._loader_kwargs(kwargs)
        self.template = adata
        merged_formula = {}

        # fit each marginal model
        for m in range(len(self.marginals)):
            marginal = self.marginals[m][1]
            marginal.setup_data(adata[:, self.marginals[m][0]], **kwargs)
            marginal.fit(
                **kwargs,
                verbose=verbose,
                val_frac=val_frac,
                min_epochs=min_epochs,
                loss_tol=loss_tol,
                patience=patience,
                validation_seed=validation_seed,
            )

            # prepare formula for copula loader
            f = marginal.formula
            prefixed_f = {f"group{m}_{k}": v for k, v in f.items()}
            merged_formula = merged_formula | prefixed_f

        # copula simulator
        self.merged_formula = merged_formula
        self.copula.setup_data(adata, merged_formula, **kwargs)
        self.copula.fit(self.merged_uniformize)
        self.parameters = {
            "marginal": [m[1].parameters for m in self.marginals],
            "copula": self.copula.parameters
        }

    def merged_uniformize(self, y: torch.Tensor, x: Dict[str, torch.Tensor]) -> torch.Tensor:
        """Produce a merged uniformized matrix for all marginals.

        Delegates to each marginal's `uniformize` method and places the
        result into the columns of a full matrix according to the variable
        selection given in `self.marginals[m][0]`.
        """
        y_np = y.detach().cpu().numpy()
        u = np.empty_like(y_np, dtype=float)

        for m in range(len(self.marginals)):
            sel = self.marginals[m][0]
            ix = _var_indices(sel, self.template)

            # remove the `group{m}_` prefix we used to distinguish the marginals
            prefix = f"group{m}_"
            cur_x = {k.removeprefix(prefix): v if k.startswith(prefix) else v for k, v in x.items()}

            # slice the subset of y for this marginal and call its uniformize
            y_sub = torch.from_numpy(y_np[:, ix])
            u[:, ix] = self.marginals[m][1].uniformize(y_sub, cur_x)
        return torch.from_numpy(u)

    def predict(self, obs=None, batch_size: int = 1000, **kwargs):
        """Predict marginal parameters for observations (batched).

        This method constructs an internal loader for ``obs`` from the copula's
        fitted design over the merged (prefixed) formulas, then dispatches
        per-marginal ``predict`` calls on each batch after stripping the
        prefixes.

        Parameters
        ----------
        obs : pandas.DataFrame, optional
            Observation metadata. Defaults to ``self.template.obs``.
        batch_size : int, optional
            Batch size for the internal observation loader.
        **kwargs
            Forwarded to :func:`~scdesigner.data.loader.obs_loader`.

        Returns
        -------
        list[dict[str, np.ndarray]]
            List with one element per marginal. Each element is a dict mapping
            parameter names to numpy arrays, concatenated across batches.
        """
        # prepare an internal data loader for this obs
        kwargs = self._loader_kwargs(kwargs)
        if obs is None:
            obs = self.template.obs
        loader = obs_loader(obs, self.copula.design, batch_size=batch_size, **kwargs)

        # prepare per-marginal collectors
        n_marginals = len(self.marginals)
        local_pred = [[] for _ in range(n_marginals)]

        # for each batch, call each marginal's predict on its subset of x
        for _, x_dict in loader:
            for m in range(n_marginals):
                prefix = f"group{m}_"
                # build cur_x where prefixed keys are unprefixed for the marginal
                cur_x = {k.removeprefix(prefix): v for k, v in x_dict.items()}
                params = self.marginals[m][1].predict(cur_x)
                local_pred[m].append(params)

        # merge batch-wise parameter dicts for each marginal and return
        results = []
        for m in range(n_marginals):
            parts = local_pred[m]
            keys = list(parts[0].keys())
            results.append({k: torch.cat([d[k] for d in parts]).detach().cpu().numpy() for k in keys})

        return results

__init__(marginals, copula_formula=None, estimator=None, top_k=None)

Create a composite simulator.

Parameters:

Name Type Description Default
marginals list

List of (sel, marginal) pairs. See class docstring for details.

required
copula_formula str

Copula grouping formula passed to :class:StandardCopula.

None
estimator CovarianceEstimator, str, or None

Covariance estimator for the copula. See :class:StandardCopula.

None
top_k int

Model only the top_k most expressed genes jointly.

None
Source code in scdesigner/src/scdesigner/simulators/composite.py
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def __init__(self, marginals: List,
             copula_formula: Optional[str] = None,
             estimator=None,
             top_k: Optional[int] = None) -> None:
    """Create a composite simulator.

    Parameters
    ----------
    marginals : list
        List of ``(sel, marginal)`` pairs. See class docstring for details.
    copula_formula : str, optional
        Copula grouping formula passed to :class:`StandardCopula`.
    estimator : CovarianceEstimator, str, or None, optional
        Covariance estimator for the copula. See :class:`StandardCopula`.
    top_k : int, optional
        Model only the ``top_k`` most expressed genes jointly.
    """
    self.marginals = marginals
    self.copula = StandardCopula(
        copula_formula if copula_formula is not None else "~ 1",
        estimator=estimator,
        top_k=top_k,
    )
    self.template = None
    self.parameters = None
    self.merged_formula = None

fit(adata, verbose=True, val_frac=0.1, min_epochs=10, loss_tol=0.0001, patience=40, validation_seed=0, **kwargs)

Fit all marginals and then fit the copula on merged uniforms.

Parameters:

Name Type Description Default
adata AnnData

Training dataset.

required
verbose bool

Whether to print verbose output.

True
val_frac float

Fraction of observations held out for each marginal validation split. Set to 0 to disable early stopping.

0.1
min_epochs int

Minimum number of marginal epochs before early stopping is allowed.

10
loss_tol float

Required relative validation-loss decrease to reset patience.

0.0001
patience int

Number of non-improving validation epochs allowed after warmup.

40
validation_seed int

Seed controlling deterministic marginal validation splits.

0
**kwargs

Additional keyword arguments forwarded to marginal setup/fit methods and to the copula's setup_data call (e.g. batch_size). Copula settings like estimator and top_k are given when the simulator is constructed, not here.

{}
Source code in scdesigner/src/scdesigner/simulators/composite.py
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
def fit(
    self,
    adata: AnnData,
    verbose: bool = True,
    val_frac: float = 0.1,
    min_epochs: int = 10,
    loss_tol: float = 1e-4,
    patience: int = 40,
    validation_seed: int = 0,
    **kwargs,):
    """Fit all marginals and then fit the copula on merged uniforms.

    Parameters
    ----------
    adata : AnnData
        Training dataset.
    verbose : bool, optional
        Whether to print verbose output.
    val_frac : float
        Fraction of observations held out for each marginal validation
        split. Set to 0 to disable early stopping.
    min_epochs : int
        Minimum number of marginal epochs before early stopping is allowed.
    loss_tol : float
        Required relative validation-loss decrease to reset patience.
    patience : int
        Number of non-improving validation epochs allowed after warmup.
    validation_seed : int
        Seed controlling deterministic marginal validation splits.
    **kwargs
        Additional keyword arguments forwarded to marginal setup/fit methods
        and to the copula's ``setup_data`` call (e.g. ``batch_size``).
        Copula settings like ``estimator`` and ``top_k`` are given when the
        simulator is constructed, not here.
    """
    kwargs = self._loader_kwargs(kwargs)
    self.template = adata
    merged_formula = {}

    # fit each marginal model
    for m in range(len(self.marginals)):
        marginal = self.marginals[m][1]
        marginal.setup_data(adata[:, self.marginals[m][0]], **kwargs)
        marginal.fit(
            **kwargs,
            verbose=verbose,
            val_frac=val_frac,
            min_epochs=min_epochs,
            loss_tol=loss_tol,
            patience=patience,
            validation_seed=validation_seed,
        )

        # prepare formula for copula loader
        f = marginal.formula
        prefixed_f = {f"group{m}_{k}": v for k, v in f.items()}
        merged_formula = merged_formula | prefixed_f

    # copula simulator
    self.merged_formula = merged_formula
    self.copula.setup_data(adata, merged_formula, **kwargs)
    self.copula.fit(self.merged_uniformize)
    self.parameters = {
        "marginal": [m[1].parameters for m in self.marginals],
        "copula": self.copula.parameters
    }

merged_uniformize(y, x)

Produce a merged uniformized matrix for all marginals.

Delegates to each marginal's uniformize method and places the result into the columns of a full matrix according to the variable selection given in self.marginals[m][0].

Source code in scdesigner/src/scdesigner/simulators/composite.py
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
def merged_uniformize(self, y: torch.Tensor, x: Dict[str, torch.Tensor]) -> torch.Tensor:
    """Produce a merged uniformized matrix for all marginals.

    Delegates to each marginal's `uniformize` method and places the
    result into the columns of a full matrix according to the variable
    selection given in `self.marginals[m][0]`.
    """
    y_np = y.detach().cpu().numpy()
    u = np.empty_like(y_np, dtype=float)

    for m in range(len(self.marginals)):
        sel = self.marginals[m][0]
        ix = _var_indices(sel, self.template)

        # remove the `group{m}_` prefix we used to distinguish the marginals
        prefix = f"group{m}_"
        cur_x = {k.removeprefix(prefix): v if k.startswith(prefix) else v for k, v in x.items()}

        # slice the subset of y for this marginal and call its uniformize
        y_sub = torch.from_numpy(y_np[:, ix])
        u[:, ix] = self.marginals[m][1].uniformize(y_sub, cur_x)
    return torch.from_numpy(u)

predict(obs=None, batch_size=1000, **kwargs)

Predict marginal parameters for observations (batched).

This method constructs an internal loader for obs from the copula's fitted design over the merged (prefixed) formulas, then dispatches per-marginal predict calls on each batch after stripping the prefixes.

Parameters:

Name Type Description Default
obs DataFrame

Observation metadata. Defaults to self.template.obs.

None
batch_size int

Batch size for the internal observation loader.

1000
**kwargs

Forwarded to :func:~scdesigner.data.loader.obs_loader.

{}

Returns:

Type Description
list[dict[str, ndarray]]

List with one element per marginal. Each element is a dict mapping parameter names to numpy arrays, concatenated across batches.

Source code in scdesigner/src/scdesigner/simulators/composite.py
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
def predict(self, obs=None, batch_size: int = 1000, **kwargs):
    """Predict marginal parameters for observations (batched).

    This method constructs an internal loader for ``obs`` from the copula's
    fitted design over the merged (prefixed) formulas, then dispatches
    per-marginal ``predict`` calls on each batch after stripping the
    prefixes.

    Parameters
    ----------
    obs : pandas.DataFrame, optional
        Observation metadata. Defaults to ``self.template.obs``.
    batch_size : int, optional
        Batch size for the internal observation loader.
    **kwargs
        Forwarded to :func:`~scdesigner.data.loader.obs_loader`.

    Returns
    -------
    list[dict[str, np.ndarray]]
        List with one element per marginal. Each element is a dict mapping
        parameter names to numpy arrays, concatenated across batches.
    """
    # prepare an internal data loader for this obs
    kwargs = self._loader_kwargs(kwargs)
    if obs is None:
        obs = self.template.obs
    loader = obs_loader(obs, self.copula.design, batch_size=batch_size, **kwargs)

    # prepare per-marginal collectors
    n_marginals = len(self.marginals)
    local_pred = [[] for _ in range(n_marginals)]

    # for each batch, call each marginal's predict on its subset of x
    for _, x_dict in loader:
        for m in range(n_marginals):
            prefix = f"group{m}_"
            # build cur_x where prefixed keys are unprefixed for the marginal
            cur_x = {k.removeprefix(prefix): v for k, v in x_dict.items()}
            params = self.marginals[m][1].predict(cur_x)
            local_pred[m].append(params)

    # merge batch-wise parameter dicts for each marginal and return
    results = []
    for m in range(n_marginals):
        parts = local_pred[m]
        keys = list(parts[0].keys())
        results.append({k: torch.cat([d[k] for d in parts]).detach().cpu().numpy() for k in keys})

    return results

to(device)

Move each component model to the device.

Source code in scdesigner/src/scdesigner/simulators/composite.py
109
110
111
112
113
114
def to(self, device):
    """Move each component model to the ``device``."""
    for _, marginal in self.marginals:
        marginal.to(device)
    self.copula.to(device)
    return self