Upstairs it seems you are counting unordered outcomes and downstairs it seems you are counting ordered outcomes. This is not right, you need to be consistent...
I'll solve the problem both ways.
With ordered outcomes and thinking of all cards as distinct:
The total number of ordered outcomes is $10\cdot9\cdot 8\cdot 7\cdot 6$.
The number of ordered outcomes in which exactly 3 cards are red is:
${5\choose3} \cdot( 5\cdot4\cdot 3 )\cdot 5\cdot4$.
The $5\choose 3$ is the number of ways to choose which of the three draws are red (it seems you had an error here). Then the number of ways to choose the cards for the "red slots" is $5\cdot4\cdot3$ and the number of ways to choose the other two cards is $5\cdot4$.
So the probability is ${{5\choose3} \cdot( 5\cdot4\cdot 3 )\cdot 5\cdot4\over10\cdot9\cdot 8\cdot 7\cdot 6 } ={ 10\cdot 25\cdot16\cdot3\over 10\cdot 9\cdot 8\cdot 7\cdot 6 }={25\over63}.$
With unordered outcomes:
There are $10\choose 5$ total outcomes and ${5\choose3}\cdot{5\choose2}$ desired outcomes. So the probability is ${{5\choose3}\cdot{5\choose2} \over {10\choose 5}} ={25\over63}.$