For what it's worth, here's the outline using the "derivative approach". Since you tagged this as analytic geometry, I would think the splendid answers by Americo and Didier would be the better approach.
Rewriting your equation of the line for $B\ne0$ gives: $ y={-Ax-C\over B}. $ (If $B=0$, the line is vertical, and the problem is simple.)
If the point $(x, {-Ax-C\over B})$ is on the line, its distance to $(x_1,y_1)$ is $ \tag{1}D=\sqrt{ (x_1-x)^2 + {\Bigl(y_1- { \textstyle{-Ax-C\over B}}\Bigr)^2 }}. $
You want to find the smallest value of $D$. This is equivalent to finding the smallest value of $ P(x)=D^2= (x_1-x)^2 + {\Bigl(y_1- { \textstyle{-Ax-C\over B}}\Bigr)^2 } $
Now, it should be obvious that the distance from a point $p$ on the line to $(x_1,y_1)$ is big, if $p$ is "on an extreme end of the line". So, the minimum distance is attained at a point where P'(x)=0.
So, you need to find P'(x), then solve P'(x)=0. I'll leave this for you...
You should only get one solution to P'(x)=0. The minimum of $P$, and hence the minimum of $D$, would then have to occur at this point . If you then plug this solution into (1), your formula will result after a bit of simplification.