let $SO(3, \mathbb{R})$ be the group of special orthogonal 3x3 matrices, i.e., $A \in SO(3, \mathbb{R})$ if $A^tA=I$ and $det(A)=1$.
I want to show $-1\leq trace(A)\leq 3$ by considering the eigenvalues of $A$, any hint for how to make a start?
let $SO(3, \mathbb{R})$ be the group of special orthogonal 3x3 matrices, i.e., $A \in SO(3, \mathbb{R})$ if $A^tA=I$ and $det(A)=1$.
I want to show $-1\leq trace(A)\leq 3$ by considering the eigenvalues of $A$, any hint for how to make a start?
Let $A\in SO_3(\mathbb{R})$ $A$ has $3$ eigenvalues, so it must have a real eigenvalue.
Case 1: $A$ has $3$ real eigenvalues. Since an orthogonal matrix preserves lengths and can only have $\pm1$ as eigenvalues, and $\det A=1$, we must have either all three eigenvalues equal to $1$, or two equal to $-1$ and the other equal to $1$.
Case 2: $A$ has a complex eigenvalue. Complex eigenvalues appear in conjuate pairs, and since the matrix is orthogonal we again have that the absolute value of each eigenvalue is 1. So these two eigenvalues multiply to 1, and since the product of the eigenvalues is $\det A=1$, the remaining eigenvalue equals $1$. Adding up the real parts of the eigenvalues (we can ignore the imaginary parts, since they will cancel), we see that the sum of the eigenvalues is in the desired range.
I know you said you want to do it by considering the eigenvalues, but it seems worth having this answer here as well nevertheless: A matrix in $SO(3,\mathbb R)$ describes a rotation through some angle $\phi$, which is conjugate to a rotation about one of the coordinate axes through that angle. Conjugate matrices have the same trace, and a matrix describing a rotation through $\phi$ about a coordinate axis has trace $1+2\cos\phi$, which ranges from $-1$ to $3$.
Since $A$ is orthonormal, all eigenvalues lie on the unit circle. Since $A$ is real, any complex eigenvalues must occur in conjugate pairs. Since $\det A = 1$, we have $\lambda_1 \lambda_2 \lambda_3 = 1$ where $\lambda_i$ are the eigenvalues.
The trace is given by $\mathbb{tr} A = \lambda_1 + \lambda_2+\lambda_3$.
Split into two cases: (1) Some eigenvalue is not real. (2) All eigenvalues are real.
Case 1: Some $\lambda_1$ is not real. Then one of the other eigenvalues (suppose $\lambda_2$) must be its conjugate. Then $\lambda_1 \lambda_2 \lambda_3 = \lambda_3 = 1$, and $\mathbb{tr} A = 2 \; \mathbb{re}(\lambda_1)+1 $. Since $|\mathbb{re}(\lambda_1)| \leq 1$, the result follows in this case.
Case 2: All eigenvalues are real. Then all eigenvalues are $\pm 1$. It is easy to see that $\mathbb{tr} A \leq 3$, only the other bound needs attention. Since the product of the eigenvalues is $1$, any negative eigenvalue must have even multiplicity, so at least one eigenvalue is $1$. Consequently, $\mathbb{tr} A \geq -1$.