If you have learned how to visualize the concept of conditional densities,
the answer
to part (c) (and indeed parts (a) and (b) as well) can be obtained with a
little bit visualization and hardly any calculation. I will explain below,
but it takes longer to write down than to simply draw a simple sketch
and deduce the answers. A picture is indeed worth the next thousand words!
The joint density can be thought of as a right prism of height $1$ with
triangular base sitting on the $x$-$y$ plane. The base has vertices are $(0,0)$, $(1,0)$ and $(1,2)$, and thus area $1$. The conditional density of
$Y$ given that $X$ has taken on value $x$ is proportional
to the cross-section of
this prism by the plane at $x$. For $0 < x < 1$, this cross-section is a rectangle with
base of length $2x$, and so the conditional density of $Y$ is a uniform
density $U(0,2x)$, that is,
$$f_{Y\mid X}(y\mid X=x)
= \begin{cases}\frac{1}{2x}, & 0 \leq y \leq 2x,\\
& \\
0, & \text{otherwise.}\end{cases}$$
Since the conditional density is uniform on $(0,2x)$, it follows immediately that
$E[Y \mid X=x] = x$.
Turning to the matter of calculating $\rho$, note that by very similar
visualization, the conditional density of $X$ given $Y = y$ is
a uniform density on $(y/2, 1)$ and thus $E[X \mid Y = y] = 1/2 + y/4$,
the midpoint of the base of the uniform density.
Now, $E[Y\mid X = x]$ and $E[X\mid Y = y]$
are the minimum-mean-square-error (MMSE) estimators for $Y$ and $X$
respectively given the value of the other random variable, and since
these are linear functions, they are also the MMSE
linear estimators for $Y$ and $X$ respectively. Now, the
MMSE linear estimators are lines through the mean point $(\mu_X,\mu_Y)$
given by
$$\begin{align*}
\frac{y-\mu_Y}{\sigma_Y} &= \rho\frac{x-\mu_X}{\sigma_X}\qquad \equiv
\quad y = x\\
\frac{x-\mu_X}{\sigma_X} &= \rho \frac{y-\mu_Y}{\sigma_Y}
\qquad \equiv \quad x = \frac{1}{2}+\frac{y}{4}
\end{align*}$$
Clearing fractions, we see that $\rho\frac{\sigma_Y}{\sigma_X} = 1$
and $\rho\frac{\sigma_X}{\sigma_Y} = \frac{1}{4}$, giving
$\rho^2 = \frac{1}{4}$ and $\rho=\frac{1}{2}$. (The solution
$\rho = -\frac{1}{2}$ can be discarded since we know that
$\rho$ is positive.)