Skip to content

donor_moments

Approximating E[sigma^2 | y] for between donor variance

This notebook includes helpers for estimating moments used in the updates for between-donor variance in the negative binomial mixed effects models implemented by NegBinEQTL. The outputs here are plugged into a larger EM loop. Mathematical details in the docstrings.

edgeworth_moments(u, info, third=None, fourth=None)

Return Edgeworth-corrected donor contributions to the variance update

This function has the same goal as laplace_moments, returning the donor-level quantity m2_k used in the variance update

sigma^2 <- (1 / K) * sum_k m2_k.

In laplace_moments, m2_k is a contribution coming from differentiating the first-order Laplace approximation to the marginal log-likelihood,

sum_k [l_k(u_k; sigma^2) - 0.5 log H_k],

This function uses a higher-order edgeworth expansion to approximate the donor posterior moment. I'll skip the derivation, but you can show that instead of the Laplace update,

    m2_k = u_k^2 +
        sigma_p,k * c3_k * u_k +
        sigma_p,k^2.

an edgeworth correction updates that last term.

    m2_k = u_k^2
       + sigma_p,k * c3_k * u_k
       + sigma_p,k^2 * (1 + 5 * c3_k^2 / 4 + c4_k / 2).

Parameters:

Name Type Description Default
u Tensor

Posterior mode u* for each donor.

required
info Tensor

Positive curvature H = -l''(u*) at the mode. The Laplace posterior standard deviation is sigma_p = H^{-1/2}.

required
third Optional[Tensor]

Negative third derivative -l'''(u*).

None
fourth Optional[Tensor]

Negative fourth derivative -l'''(u*).

None
Source code in scdesigner/src/scdesigner/distributions/donor_moments.py
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
def edgeworth_moments(
    u: torch.Tensor,
    info: torch.Tensor,
    third: Optional[torch.Tensor] = None,
    fourth: Optional[torch.Tensor] = None,
) -> Moments:
    """Return Edgeworth-corrected donor contributions to the variance update

    This function has the same goal as `laplace_moments`, returning the
    donor-level quantity m2_k used in the variance update

        sigma^2 <- (1 / K) * sum_k m2_k.

    In `laplace_moments`, m2_k is a contribution coming from differentiating the
    first-order Laplace approximation to the marginal log-likelihood,

        sum_k [l_k(u_k; sigma^2) - 0.5 log H_k],

    This function uses a higher-order edgeworth expansion to approximate the
    donor posterior moment. I'll skip the derivation, but you can show that
    instead of the Laplace update,

            m2_k = u_k^2 +
                sigma_p,k * c3_k * u_k +
                sigma_p,k^2.

    an edgeworth correction updates that last term.

            m2_k = u_k^2
               + sigma_p,k * c3_k * u_k
               + sigma_p,k^2 * (1 + 5 * c3_k^2 / 4 + c4_k / 2).

    Parameters
    ----------
    u
        Posterior mode u* for each donor.
    info
        Positive curvature H = -l''(u*) at the mode. The Laplace posterior
        standard deviation is sigma_p = H^{-1/2}.
    third
        Negative third derivative -l'''(u*).
    fourth
        Negative fourth derivative -l'''(u*).
    """
    sigma_p = info.rsqrt()
    c3 = -third * sigma_p.pow(3)
    c4 = -fourth * sigma_p.pow(4)
    mean = u + 0.5 * sigma_p * c3
    m2 = (
        u.square()
        + sigma_p * c3 * u
        + sigma_p.square() * (1.0 + 1.25 * c3.square() + 0.5 * c4)
    )
    return mean, m2

laplace_moments(u, info, third=None, fourth=None)

Second moment implied by a Laplace approximation to the marginal likelihood

For donor k, the contribution of the marginal likelihood is

    p(y_k; sigma^2) = ∫ exp{l_k(a; sigma^2)} da,

where l_k is that donor's loglikelihood + their random effect drawn from the gaussian N(0, sigma^2_{donor} prior). The usual first-order Laplace approximation expands this log likelihood around the mode, which this code assumes has already been calculated

    H_k = -l_k''(u_k),      sigma_p,k = H_k^{-1/2},

so

log p(y_k; sigma^2) ≈ l_k(u_k; sigma^2) - 0.5 log H_k + const.

Now, the donor variance can be found using an EM update,

sigma^2 <- (1 / K) * sum_k E[u_k^2 | y],

For the exact marginal likelihood, this can be found by differentiating log p(y; sigma^2) w.r.t. sigma^2 and setting it to zero. Here we do the analogous calculation after replacing the donor-level integrals by their Laplace approximation. That is, we choose sigma^2 by setting the derivative

sum_k [l_k(u_k; sigma^2) - 0.5 log H_k]

equal to zero. This derivative can be rearranged to look like,

sigma^2 <- (1 / K) * sum_k m2_k,

where m2_k is the donor contribution from donor k (a kind of approximation to E[u_k^2 | y]). Differentiating the Hessian log H_k involves the third derivatives of the loglikelihood (which is why we need it as an argument to this function). Working it out and using the notation,

sigma_p,k = H_k^{-1/2},
c3_k = l_k'''(u_k) * sigma_p,k^3,

the it's possible to express,

m2_k = u_k^2 + sigma_p,k * c3_k * u_k + sigma_p,k^2.

Parameters:

Name Type Description Default
u Tensor

Posterior mode u* for each donor.

required
info Tensor

Positive curvature H = -l''(u*) at the mode. The Laplace posterior standard deviation is sigma_p = H^{-1/2}.

required
third Optional[Tensor]

Negative third derivative -l'''(u*).

None
fourth Optional[Tensor]

Ignored. Only kept so that the API looks the same as the edgeworth approach.

None
Source code in scdesigner/src/scdesigner/distributions/donor_moments.py
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
def laplace_moments(
    u: torch.Tensor,
    info: torch.Tensor,
    third: Optional[torch.Tensor] = None,
    fourth: Optional[torch.Tensor] = None,
) -> Moments:
    """Second moment implied by a Laplace approximation to the marginal
    likelihood

    For donor k, the contribution of the marginal likelihood is

            p(y_k; sigma^2) = ∫ exp{l_k(a; sigma^2)} da,

    where l_k is that donor's loglikelihood + their random effect drawn from the
    gaussian N(0, sigma^2_{donor} prior). The usual first-order Laplace
    approximation expands this log likelihood around the mode, which this code
    assumes has already been calculated

            H_k = -l_k''(u_k),      sigma_p,k = H_k^{-1/2},

    so

        log p(y_k; sigma^2) ≈ l_k(u_k; sigma^2) - 0.5 log H_k + const.

    Now, the donor variance can be found using an EM update,

        sigma^2 <- (1 / K) * sum_k E[u_k^2 | y],

    For the exact marginal likelihood, this can be found by differentiating
    log p(y; sigma^2) w.r.t. sigma^2 and setting it to zero. Here we do the
    analogous calculation after replacing the donor-level integrals by their
    Laplace approximation. That is, we choose sigma^2 by setting the derivative

        sum_k [l_k(u_k; sigma^2) - 0.5 log H_k]

    equal to zero. This derivative can be rearranged to look like,

        sigma^2 <- (1 / K) * sum_k m2_k,

    where m2_k is the donor contribution from donor k (a kind of approximation
    to E[u_k^2 | y]). Differentiating the Hessian log H_k involves the third
    derivatives of the loglikelihood (which is why we need it as an argument to
    this function). Working it out and using the notation,

        sigma_p,k = H_k^{-1/2},
        c3_k = l_k'''(u_k) * sigma_p,k^3,

    the it's possible to express,

        m2_k = u_k^2 + sigma_p,k * c3_k * u_k + sigma_p,k^2.

    Parameters
    ----------
    u
        Posterior mode u* for each donor.
    info
        Positive curvature H = -l''(u*) at the mode. The Laplace posterior
        standard deviation is sigma_p = H^{-1/2}.
    third
        Negative third derivative -l'''(u*).
    fourth
        Ignored. Only kept so that the API looks the same as the edgeworth
        approach.
    """
    sigma_p = info.rsqrt()
    c3 = -third * sigma_p.pow(3)
    mean = u + 0.5 * sigma_p * c3
    m2 = u.square() + sigma_p * c3 * u + sigma_p.square()
    return mean, m2