The medial triangle is the [internal] Cevian triangle of largest area.
To see this, use an affine transform to map the given triangle into the triangle (0,0), (0,1), (1,0). Cevians through $(x,y)$ then yield the points $(\frac{x}{x+y},\frac{y}{x+y}), (\frac{x}{1-y},0), (0,\frac{y}{1-x})$, giving an area of $\frac{1}{2}(\frac{xy}{(1-y)(x+y)} + \frac{xy}{(1-x)(x+y)} - \frac{xy}{(1-x)(1-y)})$ = $\frac{xy}{2}\frac{(1-x)+(1-y)-(x+y)}{(1-x)(1-y)(x+y)}$ = $\frac{1}{2}\frac{xy(2-2x-2y)}{(1-x)(1-y)(x+y)}$.
Defining a=1-x, b=1-y, c=x+y, we can rewrite this as $\frac{1-a}{a}\frac{1-b}{b}\frac{1-c}{c}$ subject to $a+b+c=2$.
(Perhaps there is an easier way to get to this point, since these three fractions are exactly the ratios of the segment lengths in the three Cevians — a nice result in itself.)
Maximizing the log, we want to maximize $\sum log(\frac{1-a_i}{a_i})$ given that $\sum a_i$ is fixed. If the $a_i$ are all above 1/2, then Jensen's inequality gives us what we want, since $log(\frac{1-a}{a})$ is concave above 1/2. If any $a_i$ is below 1/2, there must be another $a_j$ which is above 1-$a_i$ (even above 1-$a_i$/2), so raising $a_i$ and lowering $a_j$ will increase $\sum log(\frac{1-a_i}{a_i})$ in that case as well, since the derivative of $log(\frac{1-a}{a})$ is more extreme the farther $a$ is from 1/2. Therefore the maximum area is attained at a=b=c=2/3, which is the medial triangle (which was also the medial triangle before the affine transformation).
There are at most 6 Cevian triangles $\triangle P_A P_B P_C$ (through Cevian point $P$ ) similar to $\triangle ABC$, as Henry points out in the comment above.
This is because once you pick the shape (angles) of $\triangle P_A P_B P_C$, then there is only a single point $P$ yielding a triangle of that shape (ugly boring proof omitted), and there are 6 ways $\triangle P_A P_B P_C$ could be similar to $\triangle A B C$ (six ways to map the three points of one onto the three points of the other). If there are fewer than 6 ways, it is due to symmetry of the original triangle: there are 3 ways for an isosceles triangle and 1 for an equilateral triangle.
Conclusion:
If your "application" requires a specific one of these 6 possible similarities (presumably the $\triangle P_A P_B P_C \sim \triangle ABC$ one), then there is only one solution (the medial triangle).
Otherwise, if solutions such as in Henry's comment above are acceptable, they will always yield smaller triangles than the medial triangle. Since there are only 5 such non-medial triangles to try, and no clean way for a formula to pick one of them, your best bet is to just try them all.