2
$\begingroup$

I would like to find out the formula for CandidateAbility used in the European PISA-test, which tests 9th grade pupil's abilities. Unfortunately the agency which publishes the results does not provide many mathematical facts. They say they use a logit-function to determine pupils' abilities in terms of percentage of correctly solved problems from a fixed problem set and the average problem difficulty for that set (never mind the definition of that). Googling for "logit" revealed the following formula:

$$\mathrm{CandidateAbility} = \log \left( \frac{x}{1-x} \right) + \mathrm{AverageDifficulty}$$

where $x$ denotes the fraction of correctly solved problems.

Assuming $\mathrm{AverageDifficulty}=0$ for now, this is centered around 0.5, i.e. a pupil solving half of the problems gets assigned ability zero. However, the PISA-agency says that they center the scale around 0.625, i.e. a pupil solving 62.5 percent of the problems gets assigned 0.

Now I can imagine many ways of modifying the above formula to achieve this. The first that come to my mind are:

$$\mathrm{CandidateAbility} = \log \left( \frac{x}{1-x} \right) + \mathrm{AverageDifficulty} - \log \left( \frac{0.625}{1-0.625} \right),$$ just shifting the outcome of the formula, and

$$\mathrm{CandidateAbility} = \log \left( \frac{x-c}{1-(x-c)} \right) + \mathrm{AverageDifficulty}$$

where c=0.625-0.5 (the difference between the new and the old center), modifying the input into the log-term.

My question is: Is there any modification of the formula, either one of the above or something entirely different, which is most natural from a statistician's point of view? Any suggestion would be welcome and could be used to counter-check against the data that is provided by the PISA-agency. Thanks!

  • 0
    That description they provide is almost devoid of mathematical content, I fear. My guess is what they're trying to describe is some version/variant of a Rasch model. They may be trying to "invert" the model to recover a particular pupil's strength coefficient.2011-03-28
  • 0
    Sorry for not mentioning it: They do indeed use a sort of Rasch model.2011-03-28

3 Answers 3