Unfortunately, we don't have a functional equation connecting $\zeta(s)$ and $\zeta(s-1)$. The reason being that $\{x\}$ is not perfectly periodic and cannot be developed into a Fourier series.
In case you are familiar with the Euler-Maclaurin summation, I can give a brief idea of how we have \begin{equation} \zeta(s)=2(2\pi)^{s-1}\Gamma(1-s)\sin\left(\frac{\pi s}{2}\right)\zeta(1-s). \end{equation}
From Euler-Maclaurin, we have
\begin{align} \zeta(s) & = \frac{1}{s-1}+\frac{1}{2}+\sum_{r=2}^{q}\frac{B_r}{r!}(s)(s+1)\cdots(s+r-2) \\ & \phantom{=} -\frac{(s)(s+1)\cdots(s+q-1)}{q!}\int_{1}^{\infty}B_{q}(x-[x])x^{-s-q} ~dx \end{align}
here $B_{q}(x-[x])$ are the periodic Bernoulli polynomials and have a Fourier expansion for $q\geq 2$.
Putting $q=3$, and using Fourier expansion, we can have the Riemann functional equation.
Just to make a slight correction, your formula is valid for $\Re(s)>2$ as
\begin{equation} -s\int_{1}^{n}\frac{(t-[t])^{2}}{t^{s+1}}dt=\sum_{k=1}^{n-1}\int_{k}^{k+1}(t-k)^{2}d(t^{-s})=\\\sum_{k=1}^{n-1}[(k+1)^{-s}+\frac{2}{s-1}(k+1)^{1-s}]+\frac{2}{(s-1)(s-2)}(n^{2-s}-1) \end{equation}
But since the integral
\begin{equation} -s\int_{1}^{\infty}\frac{(t-[t])^{2}}{t^{s+1}}dt \end{equation}
converges absolutely in the plane $\Re(s) >0$, therefore you can have your formula valid for $\Re(s)>0$ {analytic continuation} (excluding the poles $s=2$ and $s=1$ in the equation).
PS: I think you should try again for Indian Statistical Institute, Chennai Mathematical Institute and Institute of Mathematics and Applications next year. Where are you studying currently?
Best Wishes,
Sumit