Calculates intraclass correlations (ICC) for simulated samples of raters and evaluations.
simulateIRR(nRaters = c(2), nRatersPerEvent = nRaters, nLevels = 4, nEvents = 100, nSamples = 100, agreements = seq(0.1, 0.9, by = 0.1), response.probs = rep(1/nLevels, nLevels), showShinyProgress = FALSE, showTextProgress = !showShinyProgress, numCores = (parallel::detectCores() - 1), parallel = (numCores > 1), ...)
the number of available raters
the number of ratings for each per scoring event.
the number of possible outcomes there are for each rating.
the number of rating events within each matrix.
the number of sample matrices to estimate at each agreement level.
vector of percent agreements to simulate.
probability weights for the distribution of scores.
number of cores to use if the simulation is run in parallel.
whether to simulated the data using multiple cores.
a list of length
nSamples * length(nRaters) * length(agreements).
Each element of the list represents one simulation with the following
the number of raters used in the simulation.
the calculated percent agreement from the sample.
the specified percent agreement used for drawing the random sample.
skewness of all responses.
Kurtosis for all responses.
the difference between the most and least freqeuent responses.
ICC1 as described in Shrout and Fleiss (1979)
ICC2 as described in Shrout and Fleiss (1979)
ICC3 as described in Shrout and Fleiss (1979)
ICC1k as described in Shrout and Fleiss (1979)
ICC2k as described in Shrout and Fleiss (1979)
ICC3k as described in Shrout and Fleiss (1979)
Fleiss' Kappa for m raters as described in Fleiss (1971).
Cohen's Kappa as calculated in psych::cohen.kappa. Note that this calculated for all datasets even though it is only appropriate for two raters.
The simulated matrix
icctest <- simulateIRR(nLevels = 3, nRaters = 2, nSamples = 10, parallel = FALSE, showTextProgress = FALSE) summary(icctest)#> Error in simpleLoess(y, x, w, span, degree = degree, parametric = parametric, drop.square = drop.square, normalize = normalize, statistics = control$statistics, surface = control$surface, cell = control$cell, iterations = iterations, iterTrace = control$iterTrace, trace.hat = control$trace.hat): invalid 'x'