You might be interested to read about Chaitin's constant $\Omega$, which is the probability that a randomly chosen computer program will eventually terminate (you may worry a bit about the idea of a 'randomly chosen' computer program, and you'd be right to, but trust me that it can be made precise).
Chaitin's constant is a well-defined probability, so it doesn't quite answer your question, but it is non-computable, in the sense that you can't write a computer program which will output the value of the constant.
With a little bit of thinking, the reason for this becomes clear: In order to compute the probability, you would have to examine every possible program and decide if it terminates or not. Obviously this is impossible as there are infinitely many programs, but let's say that we just have to examine a sufficiently large number $N$ of possible programs to get a good estimate of the probability.
Call the program we're currently examining A. That means that you need some other program B, which can look at A and decide in a finite amount of time if A will terminate or not. But Turing proved that it is impossible to write program B so that it works on every input A - that is, there is always an input A which will cause B to run forever. So as we demand more and more accurate estimates of $\Omega$ (i.e. we increase $N$) we will eventually come across an input that causes program B to run forever, meaning that we can never compute the value of $\Omega$.