Calculates intraclass correlations (ICC) for simulated samples of raters and evaluations.

simulateIRR(nRaters = c(2), nRatersPerEvent = nRaters, nLevels = 4,
  nEvents = 100, nSamples = 100, agreements = seq(0.1, 0.9, by =
  0.1), response.probs = rep(1/nLevels, nLevels),
  showShinyProgress = FALSE, showTextProgress = !showShinyProgress,
  numCores = (parallel::detectCores() - 1), parallel = (numCores > 1),
  ...)

Arguments

nRaters

the number of available raters

nRatersPerEvent

the number of ratings for each per scoring event.

nLevels

the number of possible outcomes there are for each rating.

nEvents

the number of rating events within each matrix.

nSamples

the number of sample matrices to estimate at each agreement level.

agreements

vector of percent agreements to simulate.

response.probs

probability weights for the distribution of scores. See simulateRatingMatrix for more information.

numCores

number of cores to use if the simulation is run in parallel.

parallel

whether to simulated the data using multiple cores.

...

other parameters.

Value

a list of length nSamples * length(nRaters) * length(agreements). Each element of the list represents one simulation with the following values:

k

the number of raters used in the simulation.

simAgreement

the calculated percent agreement from the sample.

agreement

the specified percent agreement used for drawing the random sample.

skewness

skewness of all responses.

kurtosis

Kurtosis for all responses.

MaxResponseDiff

the difference between the most and least freqeuent responses.

ICC1

ICC1 as described in Shrout and Fleiss (1979)

ICC2

ICC2 as described in Shrout and Fleiss (1979)

ICC3

ICC3 as described in Shrout and Fleiss (1979)

ICC1k

ICC1k as described in Shrout and Fleiss (1979)

ICC2k

ICC2k as described in Shrout and Fleiss (1979)

ICC3k

ICC3k as described in Shrout and Fleiss (1979)

Fleiss_Kappa

Fleiss' Kappa for m raters as described in Fleiss (1971).

Cohen_Kappa

Cohen's Kappa as calculated in psych::cohen.kappa. Note that this calculated for all datasets even though it is only appropriate for two raters.

data

The simulated matrix

See also

as.data.frame.IRRsim

Examples

icctest <- simulateIRR(nLevels = 3, nRaters = 2, nSamples = 10, parallel = FALSE, showTextProgress = FALSE) summary(icctest)
#> Error in simpleLoess(y, x, w, span, degree = degree, parametric = parametric, drop.square = drop.square, normalize = normalize, statistics = control$statistics, surface = control$surface, cell = control$cell, iterations = iterations, iterTrace = control$iterTrace, trace.hat = control$trace.hat): invalid 'x'