Silly difference: Backends have inversed contrasts

[DominiqueMakowski]

dat <- iris
levels(dat$Species) <- c("Setosa", "Versicolor", "Virginica")
m <- lm(Sepal.Length ~ Species, data=iris)
modelbased::estimate_contrasts(m, contrast = "Species", backend = "emmeans")
#> Marginal Contrasts Analysis
#> 
#> Level1     | Level2     | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |      -0.93 | [-1.13, -0.73] | 0.10 |  -9.03 | < .001
#> setosa     | virginica  |      -1.58 | [-1.79, -1.38] | 0.10 | -15.37 | < .001
#> versicolor | virginica  |      -0.65 | [-0.86, -0.45] | 0.10 |  -6.33 | < .001
#> 
#> Variable predicted: Sepal.Length
#> Predictors contrasted: Species
#> p-values are uncorrected.
modelbased::estimate_contrasts(m, contrast = "Species", backend = "marginaleffects")
#> Marginal Contrasts Analysis
#> 
#> Level1     | Level2     | Difference |   SE |       95% CI | t(147) |      p
#> ----------------------------------------------------------------------------
#> versicolor | setosa     |       0.93 | 0.10 | [0.73, 1.13] |   9.03 | < .001
#> virginica  | setosa     |       1.58 | 0.10 | [1.38, 1.79] |  15.37 | < .001
#> virginica  | versicolor |       0.65 | 0.10 | [0.45, 0.86] |   6.33 | < .001
#> 
#> Variable predicted: Sepal.Length
#> Predictors contrasted: Species
#> p-values are uncorrected.

^{Created on 2026-07-11 with reprex v2.1.1}

Level1 and Level2 are swapped.

Now, which one to change, my guess is that emmeans' treatment is currently more convenient: If you specify A as your reference factor, it makes sense to have the contrasts be expressed as "A minus X", so that positive differences means A > X?

@mattansb please correct me if I'm wrong

[mattansb]

Yup lol - but you can use (rev)pairwise to make them match:

dat <- iris
levels(dat$Species) <- c("Setosa", "Versicolor", "Virginica")
m <- lm(Sepal.Length ~ Species, data = iris)

emmeans pairwise = mfx revpairwise

emmeans::emmeans(m, pairwise ~ Species)[[2]]
#>  contrast               estimate    SE  df t.ratio p.value
#>  setosa - versicolor      -0.930 0.103 147  -9.033 <0.0001
#>  setosa - virginica       -1.582 0.103 147 -15.366 <0.0001
#>  versicolor - virginica   -0.652 0.103 147  -6.333 <0.0001
#> 
#> P value adjustment: tukey method for comparing a family of 3 estimates

marginaleffects::avg_predictions(
  m,
  variables = "Species",
  hypothesis = ~revpairwise
)
#> 
#>                  Hypothesis Estimate Std. Error      z Pr(>|z|)     S  2.5 %
#>  (setosa) - (versicolor)      -0.930      0.103  -9.03   <0.001  62.4 -1.132
#>  (setosa) - (virginica)       -1.582      0.103 -15.37   <0.001 174.6 -1.784
#>  (versicolor) - (virginica)   -0.652      0.103  -6.33   <0.001  32.0 -0.854
#>  97.5 %
#>  -0.728
#>  -1.380
#>  -0.450
#> 
#> Type: response

emmeans revpairwise = mfx pairwise

emmeans::emmeans(m, revpairwise ~ Species)[[2]]
#>  contrast               estimate    SE  df t.ratio p.value
#>  versicolor - setosa       0.930 0.103 147   9.033 <0.0001
#>  virginica - setosa        1.582 0.103 147  15.366 <0.0001
#>  virginica - versicolor    0.652 0.103 147   6.333 <0.0001
#> 
#> P value adjustment: tukey method for comparing a family of 3 estimates

marginaleffects::avg_predictions(
  m,
  variables = "Species",
  hypothesis = ~pairwise
)
#> 
#>                  Hypothesis Estimate Std. Error     z Pr(>|z|)     S 2.5 %
#>  (versicolor) - (setosa)       0.930      0.103  9.03   <0.001  62.4 0.728
#>  (virginica) - (setosa)        1.582      0.103 15.37   <0.001 174.6 1.380
#>  (virginica) - (versicolor)    0.652      0.103  6.33   <0.001  32.0 0.450
#>  97.5 %
#>   1.132
#>   1.784
#>   0.854
#> 
#> Type: response

^{Created on 2026-07-11 with reprex v2.1.1}

[DominiqueMakowski]

but do you think the current emmeans "order" is best as our default one? (i.e., so that we make the swap for marginaleffects so that the two backends match)

[mattansb]

Seems pretty arbitrary to me? But I would keep them consistent at least.

[DominiqueMakowski]

Then let's go with emmeans' order, somehow it seems more natural to have the first levels as Level1

[strengejacke]

I think the most natural is probably what you would get from the coefficient, ie higher level compared to lower

[bwiernik]

I think the most natural is probably what you would get from the coefficient, ie higher level compared to lower

Particularly considering the case where the default level is "control"

[strengejacke]

m <- lm(Sepal.Length ~ Species, data = iris)
parameters::model_parameters(m)
#> Parameter            | Coefficient |   SE |       95% CI | t(147) |      p
#> --------------------------------------------------------------------------
#> (Intercept)          |        5.01 | 0.07 | [4.86, 5.15] |  68.76 | < .001
#> Species [versicolor] |        0.93 | 0.10 | [0.73, 1.13] |   9.03 | < .001
#> Species [virginica]  |        1.58 | 0.10 | [1.38, 1.79] |  15.37 | < .001
#> 
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#>   using a Wald t-distribution approximation.

^{Created on 2026-07-17 with reprex v2.1.1}

So we would use the "marginaleffects" backend as default and change emmeans behaviour?