0
$\begingroup$

Assume I want to pass a test with 98 % probability, or 95 % probability or 90 % probability or even as low as with 75 % probability.

We say that I will take the test if the probability of success is at least 75 %.

The test has 80 questions. (It is the OCAJP 8 test.) I have a practice test with 77 questions. Now if I'm lazy I don't answer all 77 questions on the practice (mock) test but only as many as I need to estimate my probability of success.

For instance, I answer 4 mock questions and get 3 answers right, then it is too little sample size to assume that I will pass the test (where passing score is 65 %) and too little sample size to generalize to 75 %.

I know there is a "magic sample size" of about 30 questions that seems reasonable here. If I answer 30 questions and get 28 or 29 answers right then it will seem "reasonable" that I will get at least 65 % on the real test.

But how should I make a more exact and correct estimate? How many questions do I have to answer to generalize my mock score?

  • 0
    This is an interesting question! However, I don't understand the meaning of the first two sentences. What grade do you need to pass the test? 50%? – 2017-01-14
  • 0
    Passing is 65 % – 2017-01-14
  • 0
    The answer depends on both the sample size of the practice exam and the proportion of correctly answered question on the practice exam. – 2017-01-14

1 Answers 1

0

First, we need to make two important assumptions: 1) all questions on a given exam are equal in terms of difficulty, and 2) the questions on the practice and real exams are equal in terms of difficulty.

Let $ X_i $ be a Bernoulli random variable that denotes whether or not the $ i_{th} $ answer on the practice exam is correct, i.e. $ X_i = 1 $ if the $ i_{th} $ answer is correct and $ X_i = 0 $ if the $ i_{th} $ answer is wrong.

Let $ p $ denote the proportion of correctly answered questions on the practice exam. Then $ \hat{p} $, the MLE of p, is equal to $ \hat{p} = \frac{\sum_{i=1}^{n}X_i}{n} = \bar{X}_{n} $.

Let $ Y $ denote the total number of correct answers on the real exam. Then $ Y \approx 80\hat{p} $. Since the minimum passing grade is 65%, we want to find:

$ P(Y\geq52) = P(80\hat{p} \geq 52) = P\big(\hat{p}\geq\frac{52}{80}\big) = P\big(\hat{p}\geq0.65\big) $

By the CLT, $ \frac{\hat{p}-E(\hat{p})}{\sqrt{Var(\hat{p})}} \overset{d}{\to} Z $. Furthermore, $ E(\hat{p}) = \bar{X}_n $ and $ Var(\hat{p}) = n\bar{X}_n(1-\bar{X}_n) $, which means that we get:

$ P\bigg(Z \geq \frac{0.65-\hat{p}}{\sqrt{Var(\hat{p})}} \bigg) = P\bigg(Z \geq \frac{0.65-\bar{X}_n}{\sqrt{n\bar{X}_n(1-\bar{X}_n)}} \bigg) $

In order to pass with probability $ (1-\alpha)100\% $, we need:

$ P\bigg(Z \geq Z_{\alpha} \bigg) = 1-\alpha $, which means that we need: $ Z_{\alpha} = \frac{0.65-\bar{X}_n}{\sqrt{n\bar{X}_n(1-\bar{X}_n)}} \rightarrow n = \Bigg(\frac{0.65-\bar{X}_n}{Z_{\alpha}\sqrt{\bar{X}_n(1-\bar{X}_n)}}\Bigg)^2 $

Note: in this formula, $ n $ is number of questions on the practice exam, while $ \bar{X}_n $ is the proportion of correctly answered questions on the practice exam.