One "algebraically" correct solution is explained by svenkatr, but I found a different solution that doesn't involve degenerate lines.
Please look at the following image for reference.

Haha, sorry it is very poorly drawn. As you can see, $XB$ is $n$, $BY$ is $m$, and $XY$ is $w$.
Now using the Pythagorean theorem, we know that:
$n^2 + m^2 = w^2$ and
$(8-n)^2+(6-m)^2=64$.
Method 1 (the first one I thought of...)
Now please refer to my second picture:

Pretend the lower, left corner of the rectangle is the origin. Now the line with the negative slope contains the points $(0,m)$ and $(n,0)$. This means that our slope is $\frac{m-0}{0-n} = \frac{-m}{n}$. That means that the slope of the perpendicular line is $\frac{n}{m}$. Since the positive slope line intersects $x=0$ at $(0,m)$, then the equation of that line is $y=\frac{n}{m}x+m$. Note that a point on that line is $(8-n,m)$, so let us plug that into the equation and get: $6 = \frac{n(8-n)}{m} + m$
And there you have your three equations:
$n^2 + m^2 = w^2$
$(8-n)^2+(6-m)^2=64$
$6 = \frac{n(8-n)}{m} + m$
My assumption is that you only need help with the algebra, so I let wolfram handle the arithmetic, and I got this result.
We now know that $m=1.89573, n=1.13305$ and most importantly, your answer $w=WZ=2.20853$ =)
Method 2 (probably the fastest)
Triangles AWX and $XBY$ are similar due to the ASA property. Therefore, the sides are proportional, and $\frac{XB}{BY}=\frac{AW}{AX}$. That was the key to this problem.
Method 3 (suggested by svenkatr)
The third equation can involve the sum of the areas. $nm + (8-n)(6-m) + 8z = 6*8 = 48$
http://www.wolframalpha.com/input/?i=n%5E2%2Bm%5E2+%3D+w%5E2%3B+%288-n%29%5E2%2B%286-m%29%5E2%3D64%3B+mn%2B%288-n%29%286-m%29+%2B+8w+%3D+48