For part 2: As a start, let's try and see why we should expect the integral to be infinity. A toy example is the case where $F=(-\infty, 0] \cup [1, \infty)$ and $x=1/2$. In this case $\delta(y)$ is $0$ for $y \in F$, and otherwise $\delta(y)=\min\{y, 1-y\}$. We can then write down the integral defining $I(1/2)$ explicitly as $I(1/2)=2 \int_0^{1/2} \frac{y}{(y-1/2)^2}.$ To see this diverges we could compute it directly. But we could also bound it by saying $\int_{1/4}^{1/2} \frac{y}{(y-1/2)^2} \geq \int_{1/4}^{1/2} \frac{1/4}{(y-1/2)^2},$ replacing $I(t)$ by a simpler integral we know diverges.
Now let's see if we can try and do something in similar in general. Our goal will be to bound our integral by the integral of something which looks like $c/(y-x)^2$. A rough sketch might be to show that for $x \notin F$ we have:
Step 1. $\delta(x)>0.$
Step 2. For some interval $[x-\epsilon, x+\epsilon]$ and some $c>0$ we have $\delta(y)>c$ everywhere in that interval.
Step 3. The integral diverges even if we only consider it on that interval.
For Part 3: Fubini sounds like a reasonable plan. If you set it up right, you should have gotten $\int_{x \in F} \int_y \frac{\delta(y)}{|x-y|^2} dy dx= \int_y \delta(y) \int_{x \in F} \frac{1}{|x-y|^2}dx dy.$
The key question is how to bound the inner integral in terms of $y$. Suppose you knew that the only points in $F$ were, say, the points at distance at least $2$ from $y$. Do you see how to use this to get an upper bound on that inner integral?