Once you realize that one should (always) draw a picture, the following might help:
Grand Principle 1: Include the conditions on the ranges in the densities as indicator functions.
In your case, the density $f_{(X,Y)}$ of $(X,Y)$ is defined for every $(x,y)$ in $\mathbb R\times\mathbb R$ by $ f_{(X,Y)}(x,y)=\frac12\mathbf 1_{0\le |x|\le |y|}\mathbf 1_{0\le |y|\le 1}, $ and the question is to know whether there exists some functions $g$ and $h$ defined on $\mathbb R$ such that, for every $(x,y)$ in $\mathbb R\times\mathbb R$, $ f_{(X,Y)}(x,y)=g(x)h(y). $ If this is so, you know that (1) $g$ is a multiple of the density of $X$, (2) $h$ is a multiple of the density of $Y$, and (3) $X$ and $Y$ are independent. Otherwise, $X$ and $Y$ are not independent.
Second remark:
Grand Principle 2: A way to disprove independence is to find a product $B_1\times B_2$ such that $f_{(X,Y)}$ is zero on $B_1\times B_2$ but neither zero on $B_1\times\mathbb R$ nor zero on $\mathbb R\times B_2$.
In other words, one is looking for a zone where $f_{(X,Y)}$ is zero but ought not to.
In your case, $B_1=(\frac12,1)$ and $B_2=(0,\frac12)$ would do. To see this, note that $f_{(X,Y)}$ is zero on $B_1\times B_2$ but that $B_1\times\mathbb R\supset(\frac12,\frac34)\times(\frac34,1)$ and $\mathbb R\times B_2\supset(0,\frac14)\times(\frac14,\frac12)$, which both have positive probability. Hence you are done.