You are on the correct path, it is very similar to the previous one. As you noted, we have $$\sum_{n\leq x}\frac{\phi(n)}{n^{2}}=\sum_{n\leq x}\frac{1}{n}\sum_{d|n}\frac{\mu(d)}{d}.$$ Switching the order of summation yields $$\sum_{d\leq x}\frac{\mu(d)}{d}\sum_{r\leq\frac{x}{d}}\frac{1}{rd}=\sum_{d\leq x}\frac{\mu(d)}{d^{2}}\sum_{r\leq \frac{x}{d}}\frac{1}{r}=\sum_{d\leq x}\frac{\mu(d)}{d^{2}}H_{\left[\frac{x}{d}\right]}.$$
Where $H_m$ is the $m^{th}$ harmonic number. Now this is possible to evaluate since $$\sum_{n\leq x}\frac{1}{n}=\log x +\gamma+O\left(\frac{1}{x}\right).$$ We then get $$\sum_{d\leq x}\frac{\phi(d)}{d^2}=\sum_{d\leq x}\frac{\mu(d)}{d^{2}}\left(\log\frac{x}{d}+\gamma+O\left(\frac{d}{x}\right)\right).$$
We may rewrite this sum as
$$\log x\sum_{d\leq x}\frac{\mu(d)}{d^{2}}-\sum_{d\leq x}\frac{\mu(d)\log d}{d^{2}}+\gamma\sum_{d\leq x}\frac{\mu(d)}{d^{2}}+O\left(\frac{1}{x}\sum_{d\leq x}\frac{|\mu(d)|}{d}\right)$$
Since $\sum_{d> x}\frac{\mu(d)\log d}{d^2}=O(\log x/x)$, and $\sum_{d\leq x}\frac{\mu (d)}{d}=O(\log x)$, we see that
$$\sum_{n\leq x}\frac{\phi(n)}{n^2}=\log x \sum_{d=1}^\infty \frac{\mu(d)}{d^2}+\gamma \sum_{d=1}^\infty \frac{\mu(d)}{d^2}-\sum_{d=1}^\infty \frac{\mu(d)\log d}{d^2}+O\left(\frac{\log x}{x}\right).$$
Looking at these series, $\sum_{d=1}^\infty \frac{\mu(d)}{d^2}=\frac{1}{\zeta(2)}$ and $\sum_{d=1}^\infty \frac{\mu(d)\log d}{d^2}=\frac{\zeta^{'}(2)}{\zeta(2)^2}$, and so
$$\sum_{n\leq x}\frac{\phi(n)}{n^2}=\frac{\log x}{\zeta(2)}+\frac{\gamma}{\zeta(2)} -\frac{\zeta^{'}(2)}{\zeta(2)^2}+O\left(\frac{\log x}{x}\right).$$
Hope that helps,
Edit: Fixed error with the constant term. I originally missed the $-\log d$ term.
Edit 2: Changed the bound $\sum_{d> x}\frac{\mu(d)\log d}{d^2}=O(1/x)$ to $\sum_{d> x}\frac{\mu(d)\log d}{d^2}=O(\log x/x).$ While the previous one is also true, it requires the prime number theorem. For this result the $\log x$ does not change anything, and there is a much simpler proof. See this answer.