The answer is yes.
You know that $E$ is a field, which implies that $E[x]$ is a Unique Factorization Domain. Therefore $p(x)$ dividing $g(x)^n$ implies $p(x)$ divides $g(x)$ if and only if $p(x)$ is squarefree in $E[x]$ (otherwise let $g(x)$ be the product of the prime divisors of $p(x)$ and let $n$ be the largest of the powers with which any of those primes appear in $p(x)$).
Now, if $a^p=b$, then what you want is that $b\in L$, but $a\not\in E$, which is equivalent to saying that $p(x)=x^p-b$ does not factor in $E[x]$ (or in $L[x]$). Hence, $p(x)$ is in fact irreducible in $E[x]$, and irreducible elements in a UFD are certainly square-free (and also of course prime).