| Title: | Set Alpha Based on Sample Size Using Bayes Factors |
|---|---|
| Description: | Sets the alpha level for coefficients in a regression model as a decreasing function of the sample size through the use of Jeffreys' Approximate Bayes factor. You tell alphaN() your sample size, and it tells you to which value you must lower alpha to avoid Lindley's Paradox. For details, see Wulff and Taylor (2024) <doi:10.1177/14761270231214429>. Alpha can also be calibrated to the effect-size and moment Bayes factors of Klauer, Meyer-Grant, and Kellen (2025) <doi:10.3758/s13423-024-02612-2>, which center the alternative hypothesis on an effect size of your choosing. |
| Authors: | Jesper Wulff [aut, cre] (ORCID: <https://orcid.org/0000-0002-7976-0939>), Luke Taylor [aut] |
| Maintainer: | Jesper Wulff <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.2.0.9000 |
| Built: | 2026-07-23 11:05:56 UTC |
| Source: | https://github.com/jespernwulff/alphan |
Computes the alpha level required to achieve a desired level of evidence,
expressed as a Bayes factor, when testing a coefficient in a regression
model. The alpha level is a decreasing function of the sample size.
Vectorized over n and BF.
alphaN( n, BF = 1, method = "JAB", upper = 1, de = 0.5, nu = NULL, r = NULL, q = 1, p = 0 )alphaN( n, BF = 1, method = "JAB", upper = 1, de = 0.5, nu = NULL, r = NULL, q = 1, p = 0 )
n |
Sample size. A positive numeric vector. |
BF |
Bayes factor you would like to match. 1 to avoid Lindley's Paradox, 3 to achieve moderate evidence and 10 to achieve strong evidence. |
method |
Which Bayes factor to calibrate alpha to. The first four
options invert Jeffreys' approximate Bayes factor and differ in the choice
of the prior fraction 'b'; the last two invert the exact test-statistic
Bayes factors of Klauer et al. (2025), whose priors center the alternative
hypothesis on a prespecified effect size
|
upper |
The upper limit for the range of realistic effect sizes. Only
relevant when method="balanced". Defaults to 1 such that the range of
realistic effect sizes is uniformly distributed between 0 and 1, U(0,1).
Conceptually, |
de |
The prespecified (targeted) effect size in standardized units:
Cohen's d for |
nu |
Degrees of freedom of the prior t distribution for methods "ES"
and "moment". The default, NULL, uses the values recommended by Klauer
et al. (2025): 3 for "ES" and |
r |
Scale of the two prior mixture components for method "ES". The
default, NULL, uses the recommendation of Klauer et al. (2025),
|
q |
Number of coefficients tested jointly. Only used by methods "ES"
and "moment". The default, 1, is the test of a single coefficient; for
|
p |
Number of parameters retained in the model under the null,
including any intercept. Only used by methods "ES" and "moment". The
effective sample size of Klauer et al. (2025) is |
For methods "ES" and "moment" with q = 1, the alpha level is found by
solving for the critical t value at which the effect-size or moment Bayes
factor equals BF, and converting that critical value to a two-sided
p-value on the t distribution with n - p - 1 degrees of freedom. For
q > 1, the same inversion runs through the noncentral-F form of the
Bayes factors (Klauer et al., 2025, Table 4), and alpha is a p-value on
the F distribution with (q, n - p - q) degrees of freedom. The
implementation is validated against the Bayes factors reported in Tables
7 and 8 of that paper. Because the moment prior assigns effects near zero
a priori density zero, the alpha level it implies decreases much faster
with n than under JAB.
Klauer et al. (2025) derive these Bayes factors under the normal linear
model (t tests, linear regression, ANOVA). For other generalized linear
models, the "ES" and "moment" calibrations hold in the same asymptotic
sense as the prior-fraction methods, whose Wald-based derivation covers
any generalized linear model directly; for non-Gaussian models with
modest n, prefer the prior-fraction methods.
As a special case, setting method = "ES", nu = 1, de = 0 with an
explicit scale (e.g. r = 1) calibrates alpha to the default
(Jeffreys-Zellner-Siow type) Bayes factor of Rouder et al. (2009).
For effective sample sizes greater than 50,000, methods "ES" and "moment" evaluate the noncentral density ratio in its normal (or chi-square) limit, which is accurate to a fraction of a percent there.
Numeric alpha level required to achieve the desired level of evidence.
Gu et al. (2016). Error probabilities in default Bayesian hypothesis testing. Journal of Mathematical Psychology, 72, 130–143.
Gu et al. (2018). Approximated adjusted fractional Bayes factors: A general method for testing informative hypotheses. The British Journal of Mathematical and Statistical Psychology, 71(2).
Klauer, K. C., Meyer-Grant, C. G., & Kellen, D. (2025). On Bayes factors for hypothesis tests. Psychonomic Bulletin & Review, 32, 1070–1094. doi:10.3758/s13423-024-02612-2
O’Hagan, A. (1995). Fractional Bayes Factors for Model Comparison. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 99–138.
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225–237.
Wagenmakers, E.-J. (2022). Approximate objective Bayes factors from p-values and sample size: The 3p(sqrt(n)) rule. PsyArXiv.
Wulff, J. N., & Taylor, L. (2024). How and why alpha should depend on sample size: A Bayesian-frequentist compromise for significance testing. Strategic Organization, 22(3), 550–581. doi:10.1177/14761270231214429
# Plot of alpha level as a function of n seqN <- seq(50, 1000, 1) plot(seqN, alphaN(seqN), type = "l") # Alpha calibrated to the effect-size Bayes factor (Klauer et al., 2025), # targeting moderate evidence for a medium-sized effect alphaN(1000, BF = 3, method = "ES", de = 0.5) # The same calibration under the moment Bayes factor alphaN(1000, BF = 3, method = "moment", de = 0.5) # Alpha for the joint F test of q = 2 coefficients in a model with an # intercept and one further retained coefficient (p = 2), targeting a # medium effect on Cohen's f scale alphaN(200, BF = 3, method = "ES", q = 2, p = 2, de = sqrt(0.15))# Plot of alpha level as a function of n seqN <- seq(50, 1000, 1) plot(seqN, alphaN(seqN), type = "l") # Alpha calibrated to the effect-size Bayes factor (Klauer et al., 2025), # targeting moderate evidence for a medium-sized effect alphaN(1000, BF = 3, method = "ES", de = 0.5) # The same calibration under the moment Bayes factor alphaN(1000, BF = 3, method = "moment", de = 0.5) # Alpha for the joint F test of q = 2 coefficients in a model with an # intercept and one further retained coefficient (p = 2), targeting a # medium effect on Cohen's f scale alphaN(200, BF = 3, method = "ES", q = 2, p = 2, de = sqrt(0.15))
Draws alpha as a decreasing function of the sample size for any selection
of the calibration methods offered by alphaN(). The prior-fraction
curves ("JAB", "min", "robust", "balanced") are evaluated exactly at every
sample size; the "ES" and "moment" curves are evaluated at twelve
log-spaced sample sizes and interpolated by a spline on the log-log
scale, which keeps the plot fast (expect roughly a second of computation
per Klauer-type curve). Colors follow the colorblind-safe Okabe-Ito
palette.
alphaN_plot( BF = 1, max = 10000, ylim = NULL, methods = c("JAB", "min", "robust", "balanced"), de = 0.5, log = "" )alphaN_plot( BF = 1, max = 10000, ylim = NULL, methods = c("JAB", "min", "robust", "balanced"), de = 0.5, log = "" )
BF |
Bayes factor you would like to match. 1 to avoid Lindley's Paradox, 3 to achieve moderate evidence and 10 to achieve strong evidence. |
max |
The maximum number of sample size. Defaults to 10,000. |
ylim |
Limits for the y-axis. The default, NULL, covers all requested curves. Set to e.g. c(0, 0.05) to zoom in on small alpha levels. |
methods |
Character vector with the methods to draw, any subset of c("JAB", "min", "robust", "balanced", "ES", "moment"). Defaults to the four prior-fraction methods, matching the behavior of earlier package versions. |
de |
The prespecified (targeted) effect size in standardized units:
Cohen's d for |
log |
Passed to |
Prints a plot.
# Plot of alpha level as a function of n for a Bayes factor of 3 alphaN_plot(BF = 3) # Compare JAB with the effect-size and moment calibrations alphaN_plot(BF = 3, methods = c("JAB", "ES", "moment"), log = "xy")# Plot of alpha level as a function of n for a Bayes factor of 3 alphaN_plot(BF = 3) # Compare JAB with the effect-size and moment calibrations alphaN_plot(BF = 3, methods = c("JAB", "ES", "moment"), log = "xy")
Computes the power of the two-sided coefficient test at the alpha level
that alphaN() calibrates to a target Bayes factor, for a standardized
effect of size d. Together with the calibrated alpha itself, this is
the quantity worth preregistering: it shows what the chosen evidence
target costs against the effects the researcher cares about. Vectorized
over n and d (recycled).
alphaN_power( n, d, BF = 3, method = "JAB", upper = 1, de = 0.5, nu = NULL, r = NULL, q = 1, p = 0 )alphaN_power( n, d, BF = 3, method = "JAB", upper = 1, de = 0.5, nu = NULL, r = NULL, q = 1, p = 0 )
n |
Sample size. A positive numeric vector. |
d |
The standardized effect size at which power is evaluated, on the
same scale as |
BF |
Target Bayes factor for the calibration. Defaults to 3. |
method |
Which Bayes factor to calibrate alpha to. The first four
options invert Jeffreys' approximate Bayes factor and differ in the choice
of the prior fraction 'b'; the last two invert the exact test-statistic
Bayes factors of Klauer et al. (2025), whose priors center the alternative
hypothesis on a prespecified effect size
|
upper |
The upper limit for the range of realistic effect sizes. Only
relevant when method="balanced". Defaults to 1 such that the range of
realistic effect sizes is uniformly distributed between 0 and 1, U(0,1).
Conceptually, |
de |
The prespecified (targeted) effect size in standardized units:
Cohen's d for |
nu |
Degrees of freedom of the prior t distribution for methods "ES"
and "moment". The default, NULL, uses the values recommended by Klauer
et al. (2025): 3 for "ES" and |
r |
Scale of the two prior mixture components for method "ES". The
default, NULL, uses the recommendation of Klauer et al. (2025),
|
q |
Number of coefficients tested jointly. Only used by methods "ES"
and "moment". The default, 1, is the test of a single coefficient; for
|
p |
Number of parameters retained in the model under the null,
including any intercept. Only used by methods "ES" and "moment". The
effective sample size of Klauer et al. (2025) is |
The power computation is exact under the normal linear model and carries the usual Wald-asymptotic interpretation for other generalized linear models, mirroring the scope of the calibration itself. When the calibrated alpha is 1 (the evidence target is met vacuously), the power is 1 for every effect size.
Power against an effect stated on a model-specific scale (an odds ratio,
a rate ratio, an R-squared increment) additionally depends on the design:
the covariate's distribution, its correlation with the other covariates,
and, in a logistic model, the baseline probability all enter the implied
standardized effect. alphaN_power() takes d as given rather than
deriving it from such design inputs. For those cases, combine the
calibrated alpha with a model-specific power calculator instead: the
functions of the pwrss package accept the significance level as
an argument, so alpha = alphaN(n, BF = 3) plugs the calibration
directly into, for example, pwrss::power.z.logistic().
A numeric vector with the power of the two-sided test (noncentral
t for q = 1, noncentral F for q > 1, both at the residual degrees
of freedom implied by n, p, and q) at the calibrated alpha.
alphaN(), alphaN_power_plot(), alphaN_report()
# Power against a small effect at the JAB-calibrated alpha, n = 1,000 alphaN_power(n = 1000, d = 0.1, BF = 3) # The same design under the balanced calibration keeps more power alphaN_power(n = 1000, d = 0.1, BF = 3, method = "balanced") # A power curve across sample sizes alphaN_power(n = c(100, 500, 1000, 5000), d = 0.2, BF = 3) # Model-specific power at the calibrated alpha via the pwrss package: # a logistic-regression coefficient with odds ratio 1.5 pwrss::power.z.logistic(odds.ratio = 1.5, base.prob = 0.2, n = 1000, alpha = alphaN(1000, BF = 3), verbose = FALSE)$power# Power against a small effect at the JAB-calibrated alpha, n = 1,000 alphaN_power(n = 1000, d = 0.1, BF = 3) # The same design under the balanced calibration keeps more power alphaN_power(n = 1000, d = 0.1, BF = 3, method = "balanced") # A power curve across sample sizes alphaN_power(n = c(100, 500, 1000, 5000), d = 0.2, BF = 3) # Model-specific power at the calibrated alpha via the pwrss package: # a logistic-regression coefficient with odds ratio 1.5 pwrss::power.z.logistic(odds.ratio = 1.5, base.prob = 0.2, n = 1000, alpha = alphaN(1000, BF = 3), verbose = FALSE)$power
Draws, for each requested effect size, the power of the two-sided
single-coefficient test as a function of the sample size, where every
calibration method is evaluated at its own alpha level for the given
Bayes factor target. A fixed reference level (default 0.05) is drawn as
a dashed curve. This is the design-time companion of alphaN_plot():
one figure shows what each calibration costs in power. Colors follow the
colorblind-safe Okabe-Ito palette; the effect-size and moment curves use
the same log-spaced spline interpolation as alphaN_plot().
alphaN_power_plot( d = c(0.1, 0.5), BF = 3, max = 10000, methods = c("JAB", "min", "robust", "balanced"), de = 0.5, ref = 0.05 )alphaN_power_plot( d = c(0.1, 0.5), BF = 3, max = 10000, methods = c("JAB", "min", "robust", "balanced"), de = 0.5, ref = 0.05 )
d |
Standardized effect sizes to draw, one panel per element, on
the scale of |
BF |
Target Bayes factor for the calibration. Defaults to 3. |
max |
The maximum number of sample size. Defaults to 10,000. |
methods |
Character vector with the methods to draw, any subset of c("JAB", "min", "robust", "balanced", "ES", "moment"). Defaults to the four prior-fraction methods, matching the behavior of earlier package versions. |
de |
The prespecified (targeted) effect size in standardized units:
Cohen's d for |
ref |
A fixed significance level drawn as a dashed reference curve, or NULL to omit it. Defaults to 0.05. |
Prints a plot.
# The power cost of evidence calibration for a small and a medium effect alphaN_power_plot(d = c(0.1, 0.5), BF = 3, methods = c("JAB", "balanced", "moment"), max = 2000)# The power cost of evidence calibration for a small and a medium effect alphaN_power_plot(d = c(0.1, 0.5), BF = 3, methods = c("JAB", "balanced", "moment"), max = 2000)
Generates a short, human-readable Markdown report that records every input behind a calibrated alpha level together with the result: the sample size, the evidence target, the calibration method and its prior settings, the resulting alpha, and the decision rule. The report is designed to be attached to a preregistration protocol or a supplementary appendix, so that an alpha level chosen before data collection leaves a citable trace. It mirrors the downloadable report of the package's companion Shiny application.
alphaN_report( n, BF = 1, method = "JAB", upper = 1, de = 0.5, nu = NULL, r = NULL, q = 1, p = 0, file = NULL, width = 72, power_at = NULL )alphaN_report( n, BF = 1, method = "JAB", upper = 1, de = 0.5, nu = NULL, r = NULL, q = 1, p = 0, file = NULL, width = 72, power_at = NULL )
n |
Sample size. A single positive number (one report describes one design). |
BF |
Target Bayes factor. A single positive number. |
method |
Which Bayes factor to calibrate alpha to. The first four
options invert Jeffreys' approximate Bayes factor and differ in the choice
of the prior fraction 'b'; the last two invert the exact test-statistic
Bayes factors of Klauer et al. (2025), whose priors center the alternative
hypothesis on a prespecified effect size
|
upper |
The upper limit for the range of realistic effect sizes. Only
relevant when method="balanced". Defaults to 1 such that the range of
realistic effect sizes is uniformly distributed between 0 and 1, U(0,1).
Conceptually, |
de |
The prespecified (targeted) effect size in standardized units:
Cohen's d for |
nu |
Degrees of freedom of the prior t distribution for methods "ES"
and "moment". The default, NULL, uses the values recommended by Klauer
et al. (2025): 3 for "ES" and |
r |
Scale of the two prior mixture components for method "ES". The
default, NULL, uses the recommendation of Klauer et al. (2025),
|
q |
Number of coefficients tested jointly. Only used by methods "ES"
and "moment". The default, 1, is the test of a single coefficient; for
|
p |
Number of parameters retained in the model under the null,
including any intercept. Only used by methods "ES" and "moment". The
effective sample size of Klauer et al. (2025) is |
file |
Optional path. If supplied, the report is also written to this file. |
width |
Maximum line width of the report; longer lines are wrapped with a hanging indent. Defaults to 72 characters. |
power_at |
Optional numeric vector of standardized effect sizes. If
supplied, the report includes the power of the calibrated test against
each of them (computed with |
The report as a character vector of lines, invisibly. The report is printed to the console.
alphaN_report(n = 1000, BF = 3, method = "JAB") # Effect-size calibration with a power section, written to a file f <- tempfile(fileext = ".md") alphaN_report(n = 1000, BF = 3, method = "ES", de = 0.5, power_at = c(0.1, 0.2, 0.5), file = f)alphaN_report(n = 1000, BF = 3, method = "JAB") # Effect-size calibration with a power section, written to a file f <- tempfile(fileext = ".md") alphaN_report(n = 1000, BF = 3, method = "ES", de = 0.5, power_at = c(0.1, 0.2, 0.5), file = f)
Extracts the test statistic of one coefficient, or of every coefficient, from a fitted model object and converts it into Jeffreys' approximate Bayes factor, given the sample size used in the fit.
JAB(glm_obj, covariate = NULL, method = "JAB", upper = 1)JAB(glm_obj, covariate = NULL, method = "JAB", upper = 1)
glm_obj |
a glm or lm object. |
covariate |
the name of the covariate that you want a BF for, as a
string. The default, NULL, returns a named vector with the Bayes factor
of every coefficient except the intercept (request the intercept
explicitly with |
method |
Used for the choice of 'b'. Currently one of:
|
upper |
The upper limit for the range of realistic effect sizes. Only relevant when method="balanced". Defaults to 1 such that the range of realistic effect sizes is uniformly distributed between 0 and 1, U(0,1). |
A numeric value with the BF in favour of H1, or a named vector of
BFs when covariate = NULL.
# Simulate data ## Sample size n <- 200 ## Regressors Z1 <- runif(n, -1, 1) Z2 <- runif(n, -1, 1) Z3 <- runif(n, -1, 1) Z4 <- runif(n, -1, 1) X <- runif(n, -1, 1) ## Error term U <- rnorm(n, 0, 0.5) ## Outcome Y <- X/sqrt(n) + U # Run a GLM LM <- glm(Y ~ X + Z1 + Z2 + Z3 + Z4) # Compute JAB for "X" based on the regression results JAB(LM, "X") # Compute JAB for every coefficient at once JAB(LM) # Compute JAB using the minimum prior JAB(LM, "X", method = "min")# Simulate data ## Sample size n <- 200 ## Regressors Z1 <- runif(n, -1, 1) Z2 <- runif(n, -1, 1) Z3 <- runif(n, -1, 1) Z4 <- runif(n, -1, 1) X <- runif(n, -1, 1) ## Error term U <- rnorm(n, 0, 0.5) ## Outcome Y <- X/sqrt(n) + U # Run a GLM LM <- glm(Y ~ X + Z1 + Z2 + Z3 + Z4) # Compute JAB for "X" based on the regression results JAB(LM, "X") # Compute JAB for every coefficient at once JAB(LM) # Compute JAB using the minimum prior JAB(LM, "X", method = "min")
Plots JAB as a function of the p-value
JAB_plot(n, BF = 1, method = "JAB", upper = 1)JAB_plot(n, BF = 1, method = "JAB", upper = 1)
n |
Sample size. A positive numeric vector. |
BF |
Bayes factor you would like to match. 1 to avoid the Lindley Paradox, 3 to achieve moderate evidence and 10 to achieve strong evidence. |
method |
Used for the choice of 'b'. Currently one of:
|
upper |
The upper limit for the range of realistic effect sizes. Only relevant when method="balanced". Defaults to 1 such that the range of realistic effect sizes is uniformly distributed between 0 and 1, U(0,1). |
Prints a plot.
# Plot JAB as function of the p-value for a sample size of 2000 JAB_plot(2000)# Plot JAB as function of the p-value for a sample size of 2000 JAB_plot(2000)
Converts a two-sided p-value from a z- or t-test into Jeffreys' approximate Bayes factor, given the sample size.
JABp(n, p, z = TRUE, df = NULL, method = "JAB", upper = 1)JABp(n, p, z = TRUE, df = NULL, method = "JAB", upper = 1)
n |
Sample size. A positive numeric vector. |
p |
The two-sided p-value. |
z |
Is the p-value based on a z- or t-statistic? TRUE if z. |
df |
If z=FALSE, provide the degrees of freedom for the t-statistic. |
method |
Used for the choice of 'b'. Currently one of:
|
upper |
The upper limit for the range of realistic effect sizes. Only relevant when method="balanced". Defaults to 1 such that the range of realistic effect sizes is uniformly distributed between 0 and 1, U(0,1). |
A numeric value for the BF in favour of H1.
# Transform a p-value of 0.007038863 from a z-test into JAB # using a sample size of 200. JABp(200, 0.007038863) # Transform a p-value of 0.007038863 from a t-test with 190 # degrees of freedom into JAB using a sample size of 200. JABp(200, 0.007038863, z=FALSE, df=190)# Transform a p-value of 0.007038863 from a z-test into JAB # using a sample size of 200. JABp(200, 0.007038863) # Transform a p-value of 0.007038863 from a t-test with 190 # degrees of freedom into JAB using a sample size of 200. JABp(200, 0.007038863, z=FALSE, df=190)
Converts a t-statistic (or z-statistic) into Jeffreys' approximate Bayes
factor, given the sample size. Vectorized over n and t.
JABt(n, t, method = "JAB", upper = 1)JABt(n, t, method = "JAB", upper = 1)
n |
Sample size. A positive numeric vector. |
t |
The t-statistic. |
method |
Used for the choice of 'b'. Currently one of:
|
upper |
The upper limit for the range of realistic effect sizes. Only relevant when method="balanced". Defaults to 1 such that the range of realistic effect sizes is uniformly distributed between 0 and 1, U(0,1). |
A numeric value for the BF in favour of H1.
# Transform a t-statistic of 2.695 computed based on a sample size of 200 into JAB JABt(200, 2.695)# Transform a t-statistic of 2.695 computed based on a sample size of 200 into JAB JABt(200, 2.695)
Computes the effect-size or moment Bayes factor of Klauer, Meyer-Grant,
and Kellen (2025) from a t statistic (one-sample test or single
regression coefficient) or from an F statistic (joint test of q
coefficients). These are the Bayes factors that alphaN() inverts for
method = "ES" and method = "moment", so a reported test statistic can
be converted into evidence under the same prior used to set the alpha
level. Vectorized over t (or Fstat) and n.
klauerBF( n, t = NULL, Fstat = NULL, q = 1, p = 0, method = "ES", de = 0.5, nu = NULL, r = NULL )klauerBF( n, t = NULL, Fstat = NULL, q = 1, p = 0, method = "ES", de = 0.5, nu = NULL, r = NULL )
n |
Sample size. A positive numeric vector. |
t |
The t-statistic. Used when |
Fstat |
The F-statistic of the model comparison. Required when
|
q |
Number of coefficients tested jointly. The default, 1, covers the one-sample test and the test of a single regression coefficient. |
p |
Number of parameters retained in the reduced model, including
any intercept. The effective sample size of Klauer et al. (2025) is
|
method |
|
de |
The prespecified (targeted) effect size: Cohen's d for |
nu |
Degrees of freedom of the prior t distribution. The default,
NULL, uses the recommendations of Klauer et al. (2025): 3 for |
r |
Scale of the prior mixture components for method |
For q = 1 the Bayes factor is evaluated in its noncentral-t form with
n - p - 1 degrees of freedom, and for q > 1 in its noncentral-F form
with (q, n - p - q) degrees of freedom (Table 4 of Klauer et al.,
2025). The implementation is validated against all printed Bayes factors
in Tables 7 and 8 of that paper.
As a special case, q = 1, nu = 1, de = 0 with an explicit scale (e.g.
r = 1) gives the default (Jeffreys-Zellner-Siow type) Bayes factor of
Rouder et al. (2009).
A numeric vector with the Bayes factor in favour of H1.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences
(second edition). Lawrence Erlbaum.
Klauer, K. C., Meyer-Grant, C. G., & Kellen, D. (2025). On Bayes factors
for hypothesis tests. Psychonomic Bulletin & Review, 32, 1070-1094.
doi:10.3758/s13423-024-02612-2
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G.
(2009). Bayesian t tests for accepting and rejecting the null hypothesis.
Psychonomic Bulletin & Review, 16, 225-237.
alphaN() for the inverse mapping from a target Bayes factor to
an alpha level, and JABt() for Jeffreys' approximate Bayes factor.
# Effect-size Bayes factor for t(79) = 2.24 targeting a medium effect # (Table 7 of Klauer et al., 2025) klauerBF(n = 80, t = 2.24, de = 0.5) # The moment Bayes factor for the same statistic klauerBF(n = 80, t = 2.24, method = "moment", de = 0.5) # Joint test of q = 2 coefficients in a regression with 3 retained # parameters (Table 8 of Klauer et al., 2025, model M9) klauerBF(n = 175, Fstat = 1.17, q = 2, p = 3, de = sqrt(0.15))# Effect-size Bayes factor for t(79) = 2.24 targeting a medium effect # (Table 7 of Klauer et al., 2025) klauerBF(n = 80, t = 2.24, de = 0.5) # The moment Bayes factor for the same statistic klauerBF(n = 80, t = 2.24, method = "moment", de = 0.5) # Joint test of q = 2 coefficients in a regression with 3 retained # parameters (Table 8 of Klauer et al., 2025, model M9) klauerBF(n = 175, Fstat = 1.17, q = 2, p = 3, de = sqrt(0.15))
Computes the effective sample size recommended by Wulff and Taylor (2024)
for calibrating alpha with clustered (panel) data, where observations are
not independent and the nominal sample size overstates the information in
the data: n_e = n * (se / se_robust)^2, the total number of
observations deflated by the squared ratio of the classical to the
cluster-robust standard error. Vectorized over its arguments (recycled).
n_effective(n, se, se_robust)n_effective(n, se, se_robust)
n |
Total number of observations. A positive numeric vector. |
se |
The classical (non-robust) standard error of the coefficient. |
se_robust |
The cluster-robust standard error of the same coefficient. |
Wulff and Taylor (2024) recommend calibrating alpha with the total number
of observations, which is the conservative choice, and then checking
whether conclusions survive when alpha and the Bayes factor are
recomputed with the effective sample size. Because cluster-robust
standard errors typically exceed classical ones, n_effective() is
typically smaller than n, which yields a larger calibrated alpha.
A numeric vector with the effective sample size.
Wulff, J. N., & Taylor, L. (2024). How and why alpha should depend on sample size: A Bayesian-frequentist compromise for significance testing. Strategic Organization, 22(3), 550-581. doi:10.1177/14761270231214429
# A regression on 237 clustered observations where the cluster-robust # standard error is twice the classical one implies an effective sample # size four times smaller (Wulff & Taylor, 2024, Example 3) n_effective(n = 237, se = 0.1, se_robust = 0.2) # Alpha for moderate evidence at the nominal and the effective sample size alphaN(n = 237, BF = 3, method = "robust") alphaN(n = n_effective(237, 0.1, 0.2), BF = 3, method = "robust")# A regression on 237 clustered observations where the cluster-robust # standard error is twice the classical one implies an effective sample # size four times smaller (Wulff & Taylor, 2024, Example 3) n_effective(n = 237, se = 0.1, se_robust = 0.2) # Alpha for moderate evidence at the nominal and the effective sample size alphaN(n = 237, BF = 3, method = "robust") alphaN(n = n_effective(237, 0.1, 0.2), BF = 3, method = "robust")