1
$\begingroup$

I have a set of data that looks like this:

Person 1 [48 total records]
2, 2, 2, 1, 2, 2, 1, 2, 1, 2, 1, 2, 2, 2, 1, 1, 1, 2, 1, 2, 2, 1, 2, 1, 1, 1, 1, 1, 1, 2, 2, 1, 2, 1, 1, 2, 2, 2, 2, 1, 1, 2, 2, 1, 2, 1, 1, 1,
--
Person 2 [56 total records]
1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 1, 2, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 2, 2, 2, 2, 2, 2, 1, 2, 1, 1, 1, 1, 2, 1,
--
Person 3 [18 total records]
1, 2, 1, 2, 1, 1, 1, 1, 2, 2, 1, 2, 2, 2, 1, 1, 1, 1,

A '1' indicates a false answer, and a '2' indicates a correct one. I want to be able to compare records to see if any people in the data set are performing significantly above or below average. I've heard of using z-scores and standard deviation, but I'm not sure if that's the correct approach, or even how I would go about doing the calculation.

I also need to find out the minimum number of records I would need in order to have sufficient confidence in the results.

My math skills are pretty limited, so a simple explanation would be greatly appreciated.

  • 0
    It's easy to find scores that are above and below average, but where to draw the boundary between significant and not significant is a personal opinion. Would you mind clarifying what is significant for your purposes?2012-08-16
  • 0
    I thought significance was a technical term that indicates whether the product indicates a pattern, rather than just chance. Or am I confusing it with confidence levels?2012-08-16

1 Answers 1

0

How about a very simple solution that can give you kind of an overview?

  1. for each person, calculate the percentage of correct answers (= the number of 2's / total records).
  2. group these numbers into groups, e.g. 0-10%, 10-20%, .... Count the numbers in each group. This gives you an overview of how people perform, e.g. 6/10 people belongs to the 80-90% group and 4/10 in the 20-30%.
  • 0
    This approach is more reasonable, but many journals in a wide swatch of sciences - particularly biomedicine and healthcare, sociology &c, favor pre-WW2 methods that involve "significance." Stanford statistician Efron in his new book "Large Scale Inference" wrote in a footnote that he no longer refers to patterns as "significant" but rather as "interesting"2012-08-15
  • 0
    Is there a reason why the "significance" approach is a problem? I thought significance was a technical term that indicates whether the product indicates a pattern, rather than just chance.2012-08-16