If I understood Dinesh's intent correctly he wants an example for each prime $p$. Henning and Zev have covered the case $p=2$.
Let first $p=3$.
Let $\zeta=e^{2\pi i/9}$. Then $L=\mathbf{Q}(\zeta)$ is the ninth cyclotomic field. It is well known that $Gal(L/\mathbf{Q})$ is cyclic of degree six. A generator $\sigma$ is determined by $\sigma(\zeta)=\zeta^2$. The subgroup generated by $\sigma^3$ has thus order two, and its fixed field $K=\mathbf{Q}(\zeta+\zeta^{-1})$ is a cyclic cubic Galois extension of $\mathbf{Q}$. I claim that $\alpha=\zeta+\zeta^{-1}=2\cos\left(2\pi/9\right)$ works. We can calculate that $ N^K_{\mathbf{Q}}(\alpha)=(\zeta+\zeta^{-1})(\zeta^2+\zeta^{-2})(\zeta^4+\zeta^{-4})=-1, $ so $\alpha$ is actually a unit of the ring of integers of $K$. But also note that $\alpha$ is a real number $>1$, so it cannot be a root of unity. Together these facts imply that no power of $\alpha$ will be rational. Namely, let $n>0$ be any integer.Then $\alpha^n$ is an algebraic integer of $K$, so if it were rational, it would have to be a rational integer $m$. On the other hand $N^K_{\mathbf{Q}}(\alpha^n)=(-1)^n$, so $m$ would have to be $\pm1$, and $\alpha$ would have to be a root of unity of order at most $2n$. A contradiction. Because $[K:\mathbf{Q}]=3$ is a prime, and $\alpha^n\in K\setminus\mathbf{Q}$, we have $K=\mathbf{Q}(\alpha^n)$ as desired.
Let then $p$ be any fixed prime $>3$. A way to generalize the above argument is to look at a bigger cyclotomic field. Let $\zeta=e^{2\pi i/p^2}$, and $L=\mathbf{Q}(\zeta)$. It is known that $L/\mathbf{Q}$ is a cyclic Galois extension of degree $\phi(p^2)=p(p-1)$. Let $H$ be the unique subgroup of this Galois group that is of order $p-1$. Then the fixed field $K$ of $H$ is a degree $p$ cyclic extension of $\mathbf{Q}$. Furthermore, the restriction of the usual complex conjugation to $L$ belongs to $H$, because it is of order two, and $2\mid p-1$. Therefore we can deduce that the numbers in the field $K$ are all real (the field $K$ is, in fact totally real). From Dirichlet's theorem of units we know that the group of units of the ring of integers of $K$ has a free abelian part of rank $p-1$. Let us pick a non-torsion element $u$ from that group. As $u$ is real, and not a root of unity, we can repeat the previous argument, and conclude that $K=\mathbf{Q}(u^n)$ for all positive integers $n$.
I would love to give you an explicit $u$ as in the case $p=3$, but I can't find a suitable cyclotomic unit now that would belong to the field $K$.