This is a simple calculator for determining cost ratios to use in the optimal alpha tool. The costs considered here are those related to making a Type I or Type II error.While some error costs are easy to quantiy in monetary terms (direct costs), others are difficult to put a dolar amount to (indirect costs). Examples of indirect costs may include time lost due to appeals and litigation, loss of credibility, damage to partnerships and working relationships. Despite being challenging to quantify, you should make an effort to put a number to them. Alternatively, costs could be specified in relative terms.

In the context of natural resource monitoring, failing to detect a change (Type II error) can be more harmful and costly than falsely claiming that a change occurred (Type I error). Rather than using arbitrary error rates (e.g., alpha = 0.05) for statitical analyses and tests which may lead to increased likelihood of making Type II errors, both error rates should be set at levels that minimize the chance of making any type of error. Because Type I and II error rates are related, the optimal error rate is the alpha level (Type I error rate) that minimizes the probability of Type I and Type II errors for given sample and effect sizes (see Mudge et al. 2012a, 2012b). The optimal error rate can be adjusted to account for differential costs of the different types of errors and for expectations (i.e., prior probabilities) of the likelihood of change occurring (or not occurring).

This tool uses R code provided by Mudge et al. (2012a) to calculate optimal Type I error rates. Specifically, this Shiny tool executes the following function:

optab(n1,n2,d,type,tails,T1T2cratio,HaHopratio

where n1 and n2 are sample sizes, d is Cohen's effect size, type is the type of t-Test (one sample, two-sample, paired), tails refers to whether the test is for a one-tailed or two-tailed alternative, T1T2cratio = the cost ration of Type I to Type II errors, and HaHopratio is the ratio of prior probabilities. A companion function was written to produce the plot showing how error rates change as Type I error rate goes from zero to one:

optab.plot(n1,n2,d,type,tails,T1T2cratio,HaHopratio)

- Mudge, Joseph F., Leanne F. Baker, Christopher B. Edge, and Jeff E. Houlahan. “Setting an Optimal Α That Minimizes Errors in Null Hypothesis Significance Tests.” Edited by Zheng Su. PLoS ONE 7, no. 2 (February 28, 2012): e32734. doi:10.1371/journal.pone.0032734.
- Mudge, Joseph F., Timothy J. Barrett, Kelly R. Munkittrick, and Jeff E. Houlahan. “Negative Consequences of Using Α = 0.05 for Environmental Monitoring Decisions: A Case Study from a Decade of Canada’s Environmental Effects Monitoring Program.” Environmental Science & Technology 46, no. 17 (September 4, 2012): 9249–55. doi:10.1021/es301320n.