Compare Effects from Experimental vs. Null Mediation Data

One way to calibrate our conclusions from complex workflows is to see how they would look on data where we know that there is no effect. This function compares estimators f between real and synthetic null data, where the null removes a set of edges according to the nullfication argument.

Usage

null_contrast(model, exper, nullification = "T->Y", f = direct_effect)

Arguments

model

An object of class multimedia with specified mediation and outcome models that we want to re-estimate across B bootstrap samples.

exper

An object of class multimedia_data containing the mediation and outcome data from which the direct effects are to be estimated.

nullification

A string specifying the types of edges whose effects we want to remove in the null samples. Valid options are "T->Y" (the default), "T->M", "M->Y", which remove direct effects, treatment to mediator effects, and mediator to treatment effects, respectively.

f

The estimator that we want to compare between real and null data. This is assumed to be a function taking counterfactual samples, for example direct_effects or indirect_effects.

Value

A data.frame containing estimates on the real and synthetic data for every coordinate in the estimator f. The column source specifies whether the estimate was calculated using real or synthetic null data.

See also

null_contrast fdr_summary

Examples

# example with null data - notice synthetic data has larger effect.
exper <- demo_joy() |>
    mediation_data("PHQ", "treatment", starts_with("ASV"))
multimedia(exper) |>
    estimate(exper) |>
    null_contrast(exper)
#> Fitting the nullified model...
#> Generating synthetic data...
#> Fitting the full model on synthetic data...
#> Estimating effects on real and synthetic data...
#>      source outcome indirect_setting            contrast direct_effect
#> 1      real     PHQ          Control Control - Treatment     0.3049548
#> 2      real     PHQ        Treatment Control - Treatment     0.3049548
#> 3 synthetic     PHQ          Control Control - Treatment    -0.1012618
#> 4 synthetic     PHQ        Treatment Control - Treatment    -0.1012618

# example with another dataset - synthetic effect is smaller.
exper <- demo_spline(tau = c(2, 1)) |>
    mediation_data(starts_with("outcome"), "treatment", "mediator")
multimedia(exper) |>
    estimate(exper) |>
    null_contrast(exper)
#> Fitting the nullified model...
#> Generating synthetic data...
#> Fitting the full model on synthetic data...
#> Estimating effects on real and synthetic data...
#>      source   outcome indirect_setting contrast direct_effect
#> 1      real outcome_1                0    0 - 1   -2.13901067
#> 2      real outcome_2                0    0 - 1   -0.82760387
#> 3      real outcome_1                1    0 - 1   -2.13901067
#> 4      real outcome_2                1    0 - 1   -0.82760387
#> 5 synthetic outcome_1                0    0 - 1   -0.02611773
#> 6 synthetic outcome_2                0    0 - 1   -0.02276149
#> 7 synthetic outcome_1                1    0 - 1   -0.02611773
#> 8 synthetic outcome_2                1    0 - 1   -0.02276149