I calculated the same polynomial and I got
$P(X)= X^2 (X-1)^3 (X-4) \,.$
Note that $tr(A)=7$ has to be the sum of eigenvalues.
Just to get you started:
$\det(A-tI)= \det \pmatrix{2-t & 1 & 1 & 1 & 1 & 1 \\ 1 & 1-t & 0 & 1 & 0 & 1 \\ 1 & 0 & 1-t & 0 & 0 & 1 \\ 1 & 0 & 0 & 1-t & 0 & 0 \\ 1 & 0 & 0 & 0 & 1-t & 0 \\ 1 & 0 & 0 & 0 & 0 & 1-t}$
Subtract the 6th row from 4th and 5th: $\det(A-tI)= \det \pmatrix{2-t & 1 & 1 & 1 & 1 & 1 \\ 1 & 1-t & 0 & 1 & 0 & 1 \\ 1 & 0 & 1-t & 0 & 0 & 1 \\ 0 & 0 & 0 & 1-t & 0 & t-1 \\ 0 & 0 & 0 & 0 & 1-t & t-1 \\ 1 & 0 & 0 & 0 & 0 & 1-t}$
Now, $(1-t)$ common factor on rows 4 and 5.
$\det(A-tI)= (t-1)^2\det \pmatrix{2-t & 1 & 1 & 1 & 1 & 1 \\ 1 & 1-t & 0 & 1 & 0 & 1 \\ 1 & 0 & 1-t & 0 & 0 & 1 \\ 0 & 0 & 0 & 1 & 0 & -1 \\ 0 & 0 & 0 & 0 & 1 & -1 \\ 1 & 0 & 0 & 0 & 0 & 1-t}$
Next add Column 4 and column 5 to Column 6, and you can get a smaller $4 \times 4$ determinant....