Let's say that you have a probability $p$ of getting the correct answer to each question. Then if you answer $n$ of the questions, the number that you answer correctly is distributed as a binomial random variable $X\sim B(n,p)$.
Your score on the exam, $Y$, is related to this. You score 1 for each correct answer and lose 1/2 for each incorrect answer, which means that
$Y = X - \tfrac{1}{2}(n-X) = \tfrac{1}{2}(3X-n)$
You are asking for the probability that $Y\geq 5$, which reduces to
$\begin{align} P(Y\geq 5) & = P\left(\frac{3X-n}{2} \geq 5\right) \\ & = P\left(X \geq \frac{10+n}{3}\right) \end{align}$
This probability can be easily calculated from the binomial density, or (cheating) using the following R code:
> f <- function(n,p) {pbinom((10+n)/3 - 0.001, n, p, lower.tail=FALSE)} > plot(0:10, f(0:10, 0.25), type='l')
which generates this chart, from which you can see that you should answer eight questions:
[image upload not working at the moment, I'll come back and edit the image in later]