Suppose that $f(n) + (n+1)^2 = f(n+1),$
- How could I find the original (or family of) function(s) that satisfies this property?
- What is the branch of mathematics that deals with equations like this?
Suppose that $f(n) + (n+1)^2 = f(n+1),$
That is a recurrence relation. There will be a family of solutions, depending on the starting point. For example, starting with $f(0)=0$, it will simply be the sum of squares, which is $f(n)=n(n+1)(2n+1)/6$. If you started from $f(0)=1$ it would be $(n+2)(2n^2-n+3)/6$.
There are various ways of solving it. One is spot that since the first difference $f(n+1)-f(n) = n^2+2n+1$ is a quadratic polynomial, $f(n)$ will be a cubic polynomial, so you can use the early values to find the coefficients.
Another would be to use generating functions, for example described in generatingfunctionology.
And there are more, such as methods associated with functional equations.
Let $x=n+1$ ,
Then $f(x)=f(x-1)+x^2$
In fact this belongs to a functional equation of the form http://eqworld.ipmnet.ru/en/solutions/fe/fe1108.pdf.
The general solution of this functional equation is $f(x)=\Theta(x)+f_p(x)$ , where $\Theta(x)$ is an arbitrary periodic function with unit period
Luckily we can find $f_p(x)$ by method of undetermined coefficients:
Let $f_p(x)=Ax^3+Bx^2+Cx$ ,
Then $f_p(x-1)=A(x-1)^3+B(x-1)^2+C(x-1)=Ax^3-3Ax^2+3Ax-A+Bx^2-2Bx+B+Cx-C=Ax^3+(B-3A)x^2+(3A-2B+C)x-A+B-C$
$\therefore Ax^3+Bx^2+Cx-(Ax^3+(B-3A)x^2+(3A-2B+C)x-A+B-C)\equiv x^2$
$3Ax^2+(2B-3A)x+A-B+C\equiv x^2$
$\therefore\begin{cases}3A=1\\2B-3A=0\\A-B+C=0\end{cases}$
$\begin{cases}A=\dfrac{1}{3}\\B=\dfrac{1}{2}\\C=\dfrac{1}{6}\end{cases}$
$\therefore f(x)=\Theta(x)+\dfrac{x^3}{3}+\dfrac{x^2}{2}+\dfrac{x}{6}=\Theta(x)+\dfrac{2x^3+3x^2+x}{6}=\Theta(x)+\dfrac{x(x+1)(2x+1)}{6}$ , where $\Theta(x)$ is an arbitrary periodic function with unit period
$f(n)=\Theta(n)+\dfrac{n(n+1)(2n+1)}{6}$ , where $\Theta(n)$ is an arbitrary periodic function with unit period