Calculates intraclass correlations (ICC) for simulated samples of raters and evaluations.

simulateIRR(nRaters = c(2), nRatersPerEvent = nRaters, nLevels = 4,
nEvents = 100, nSamples = 100, agreements = seq(0.1, 0.9, by =
0.1), response.probs = rep(1/nLevels, nLevels),
showShinyProgress = FALSE, showTextProgress = !showShinyProgress,
numCores = (parallel::detectCores() - 1), parallel = (numCores > 1),
...)

Arguments

nRaters the number of available raters the number of ratings for each per scoring event. the number of possible outcomes there are for each rating. the number of rating events within each matrix. the number of sample matrices to estimate at each agreement level. vector of percent agreements to simulate. probability weights for the distribution of scores. See simulateRatingMatrix for more information. number of cores to use if the simulation is run in parallel. whether to simulated the data using multiple cores. other parameters.

Value

a list of length nSamples * length(nRaters) * length(agreements). Each element of the list represents one simulation with the following values:

k

the number of raters used in the simulation.

simAgreement

the calculated percent agreement from the sample.

agreement

the specified percent agreement used for drawing the random sample.

skewness

skewness of all responses.

kurtosis

Kurtosis for all responses.

MaxResponseDiff

the difference between the most and least freqeuent responses.

ICC1

ICC1 as described in Shrout and Fleiss (1979)

ICC2

ICC2 as described in Shrout and Fleiss (1979)

ICC3

ICC3 as described in Shrout and Fleiss (1979)

ICC1k

ICC1k as described in Shrout and Fleiss (1979)

ICC2k

ICC2k as described in Shrout and Fleiss (1979)

ICC3k

ICC3k as described in Shrout and Fleiss (1979)

Fleiss_Kappa

Fleiss' Kappa for m raters as described in Fleiss (1971).

Cohen_Kappa

Cohen's Kappa as calculated in psych::cohen.kappa. Note that this calculated for all datasets even though it is only appropriate for two raters.

data

The simulated matrix

icctest <- simulateIRR(nLevels = 3, nRaters = 2, nSamples = 10, parallel = FALSE, showTextProgress = FALSE)
summary(icctest)#> Error in simpleLoess(y, x, w, span, degree = degree, parametric = parametric,     drop.square = drop.square, normalize = normalize, statistics = control$statistics, surface = control$surface, cell = control$cell, iterations = iterations, iterTrace = control$iterTrace, trace.hat = control\$trace.hat): invalid 'x'