This function will generate a nEvents x k scoring matrix.

simulateRatingMatrix(nLevels, k, k_per_event = 2, agree, nEvents = 100,
  response.probs = rep(1/nLevels, nLevels))

Arguments

nLevels

the number of possible outcomes there are for each rating.

k

the total number of available raters.

k_per_event

number of raters per scoring event.

agree

the percent of time the raters agree. Note that the actual agreement of the simulated matrix will vary from this value (see sample).

nEvents

the number of rating events within each matrix.

response.probs

probability weights for the distribution of scores. By default, each of the levels has equal probability of being selected. This allows situations where some responses are more common than others (e.g. 50% of students get a 3, 30% get a 2, and 20% get a 1). This is independent of the percent agreement parameter.

Examples

test <- simulateRatingMatrix(nLevels = 3, k = 2, agree = 0.6, nEvents = 100) psych::ICC(test)
#> singular fit
#> Call: psych::ICC(x = test) #> #> Intraclass correlation coefficients #> type ICC F df1 df2 p lower bound upper bound #> Single_raters_absolute ICC1 0.67 5.1 99 100 4.3e-15 0.55 0.77 #> Single_random_raters ICC2 0.67 5.1 99 99 5.4e-15 0.55 0.77 #> Single_fixed_raters ICC3 0.67 5.1 99 99 5.4e-15 0.55 0.77 #> Average_raters_absolute ICC1k 0.81 5.1 99 100 4.3e-15 0.71 0.87 #> Average_random_raters ICC2k 0.81 5.1 99 99 5.4e-15 0.71 0.87 #> Average_fixed_raters ICC3k 0.81 5.1 99 99 5.4e-15 0.71 0.87 #> #> Number of subjects = 100 Number of Judges = 2