Package 'alphaN'

Title: Set Alpha Based on Sample Size Using Bayes Factors
Description: Sets the alpha level for coefficients in a regression model as a decreasing function of the sample size through the use of Jeffreys' Approximate Bayes factor. You tell alphaN() your sample size, and it tells you to which value you must lower alpha to avoid Lindley's Paradox. For details, see Wulff and Taylor (2023) <doi:10.31234/osf.io/3cbh7>.
Authors: Jesper Wulff [aut, cre] , Luke Taylor [aut]
Maintainer: Jesper Wulff <[email protected]>
License: MIT + file LICENSE
Version: 0.1.0
Built: 2025-02-17 03:14:04 UTC
Source: https://github.com/jespernwulff/alphan

Help Index

Set the alpha level based on sample size for coefficients in a regression models.

Description

Set the alpha level based on sample size for coefficients in a regression models.

Usage

alphaN(n, BF = 1, method = "JAB", upper = 1)

Arguments

n

Sample size
BF

Bayes factor you would like to match. 1 to avoid Lindley's Paradox, 3 to achieve moderate evidence and 10 to achieve strong evidence.
method

Used for the choice of 'b'. Currently one of:

  • "JAB": this choice of b produces Jeffery's approximate BF (Wagenmakers, 2022)

  • "min": uses the minimal training sample for the prior (Gu et al., 2018)

  • "robust": a robust version of "min" that prevents too small b (O'Hagan, 1995)

  • "balanced": this choice of b balances the type I and type II errors (Gu et al, 2016)

upper

The upper limit for the range of realistic effect sizes. Only relevant when method="balanced". Defaults to 1 such that the range of realistic effect sizes is uniformly distributed between 0 and 1, U(0,1).

Value

Numeric alpha level required to achieve the desired level of evidence.

References

Gu et al. (2016). Error probabilities in default Bayesian hypothesis testing. Journal of Mathematical Psychology, 72, 130–143.

Gu et al. (2018). Approximated adjusted fractional Bayes factors: A general method for testing informative hypotheses. The British Journal of Mathematical and Statistical Psychology, 71(2).

O’Hagan, A. (1995). Fractional Bayes Factors for Model Comparison. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 99–138.

Wagenmakers (2002). Approximate Objective Bayes Factors From PValues and Sample Size: The 3pn Rule. psyarxiv.

Wulff & Taylor (2023). How and why alpha should depend on sample size: A Bayesian-frequentist compromise for significance testing. PsyArXiv.

Examples

# Plot of alpha level as a function of n
seqN <- seq(50, 1000, 1)
plot(seqN, alphaN(seqN), type = "l")

Creates a plot of alpha as function of sample size for each of the four prior options

Description

Creates a plot of alpha as function of sample size for each of the four prior options

Usage

alphaN_plot(BF = 1, max = 10000)

Arguments

BF

Bayes factor you would like to match. 1 to avoid Lindley's Paradox, 3 to achieve moderate evidence and 10 to achieve strong evidence.
max

The maximum number of sample size. Defaults to 10,000.

Value

Prints a plot.

Examples

# Plot of alpha level as a function of n for a Bayes factor of 3
alphaN_plot(BF = 3)

Transforms a t-statistic from a glm or lm object into Jeffreys' approximate Bayes factor

Description

Transforms a t-statistic from a glm or lm object into Jeffreys' approximate Bayes factor

Usage

JAB(glm_obj, covariate, method = "JAB", upper = 1)

Arguments

glm_obj

a glm or lm object.
covariate

the name of the covariate that you want a BF for as a string.
method

Used for the choice of 'b'. Currently one of:

  • "JAB": this choice of b produces Jeffery's approximate BF (Wagenmakers, 2022)

  • "min": uses the minimal training sample for the prior (Gu et al., 2018)

  • "robust": a robust version of "min" that prevents too small b (O'Hagan, 1995)

  • "balanced": this choice of b balances the type I and type II errors (Gu et al, 2016)

upper

The upper limit for the range of realistic effect sizes. Only relevant when method="balanced". Defaults to 1 such that the range of realistic effect sizes is uniformly distributed between 0 and 1, U(0,1).

Value

A numeric value for the BF in favour of H1.

Examples

# Simulate data

## Sample size
n <- 200

## Regressors
Z1 <- runif(n, -1, 1)
Z2 <- runif(n, -1, 1)
Z3 <- runif(n, -1, 1)
Z4 <- runif(n, -1, 1)
X <- runif(n, -1, 1)

## Error term
U <- rnorm(n, 0, 0.5)

## Outcome
Y <- X/sqrt(n) + U

# Run a GLM
LM <- glm(Y ~ X + Z1 + Z2 + Z3 + Z4)

# Compute JAB for "X" based on the regression results
JAB(LM, "X")

# Compute JAB using the minimum prior
JAB(LM, "X", method = "min")

Plots JAB as a function of the p-value

Description

Plots JAB as a function of the p-value

Usage

JAB_plot(n, BF = 1, method = "JAB")

Arguments

n

Sample size
BF

Bayes factor you would like to match. 1 to avoid the Lindley Paradox, 3 to achieve moderate evidence and 10 to achieve strong evidence.
method

Used for the choice of 'b'. Currently one of:

  • "JAB": this choice of b produces Jeffery's approximate BF

  • "min": uses the minimal training sample for the prior (Gu et al., '17)

  • "robust": a robust version of "min" that prevents too small b (O'Hagan, '95)

  • "balanced": this choice of b balances the type I and type II errors

Value

Prints a plot.

Examples

# Plot JAB as function of the p-value for a sample size of 2000
JAB_plot(2000)

Title

Description

Title

Usage

JABp(n, p, z = TRUE, df = NULL, method = "JAB", upper = 1)

Arguments

n

Sample size.
p

The p-value.
z

Is the p-value based on a z- or t-statistic? TRUE if z.
df

If z=FALSE, provide the degrees of freedom for the t-statistic.
method

Used for the choice of 'b'. Currently one of:

  • "JAB": this choice of b produces Jeffery's approximate BF (Wagenmakers, 2022)

  • "min": uses the minimal training sample for the prior (Gu et al., 2018)

  • "robust": a robust version of "min" that prevents too small b (O'Hagan, 1995)

  • "balanced": this choice of b balances the type I and type II errors (Gu et al, 2016)

upper

The upper limit for the range of realistic effect sizes. Only relevant when method="balanced". Defaults to 1 such that the range of realistic effect sizes is uniformly distributed between 0 and 1, U(0,1).

Value

A numeric value for the BF in favour of H1.

Examples

# Transform a p-value of 0.007038863 from a z-test into JAB
# using a sample size of 200.
JABp(200, 0.007038863)

# Transform a p-value of 0.007038863 from a t-test with 190
# degrees of freedom into JAB using a sample size of 200.
JABp(200, 0.007038863, z=FALSE, df=190)

Transforms a t-statistic into Jeffreys' approximate Bayes factor

Description

Transforms a t-statistic into Jeffreys' approximate Bayes factor

Usage

JABt(n, t, method = "JAB", upper = 1)

Arguments

n

Sample size.
t

The t-statistic.
method

Used for the choice of 'b'. Currently one of:

  • "JAB": this choice of b produces Jeffery's approximate BF (Wagenmakers, 2022)

  • "min": uses the minimal training sample for the prior (Gu et al., 2018)

  • "robust": a robust version of "min" that prevents too small b (O'Hagan, 1995)

  • "balanced": this choice of b balances the type I and type II errors (Gu et al, 2016)

upper

The upper limit for the range of realistic effect sizes. Only relevant when method="balanced". Defaults to 1 such that the range of realistic effect sizes is uniformly distributed between 0 and 1, U(0,1).

Value

A numeric value for the BF in favour of H1.

Examples

# Transform a t-statistic of 2.695 computed based on a sample size of 200 into JAB
JABt(200, 2.695)