In the above problem, I can't seem to understand why we include the cosθ term which changes the measure of the integral. The only thing I am able to note is that this corresponds to the dot product of the lines of force with the unit normal of the plane. However, I can't seem to think of a rationale for why this is necessary.
I suspected it was a mathematical one and therefore I have posted this question here.
That's because the force is an vector quantity. The vector is $\cdots\cos\theta$ downwards and $\cdots\sin\theta$ sideways, but the later component is canceled by the charge at the opposite side.
To elaborate:
$${4\pi\epsilon_0F\over\sigma} = \int {(x,y,r)\over|x,y,r|^3}dS = \int {(R\cos\phi,R\sin\phi,r)\over|R,r|^3}d\phi dR = \int{2\pi R(0,0,r)\over (R^2+r^2)^{3/2}}dR$$
But you have from the definition that $\cos\theta = r/\sqrt{R^2+r^2}$ which makes $(0,0,r)/\sqrt{R^2+r^2} = (0,0,1)\cos\theta$ from which the result follows.