The equivalence in the exercise is indeed false: since $A$ is a $K$-algebra (multiply elements of $K\otimes_k K$ by elements of $K$ through the left factor, say) , so is $A/J(A)$ which can thus never equal $k$ if the extension $k\subset K$ is not trivial.
You are also right that for any finite-dimensional extension $k\to K$, purely inseparable or not, $A/J(A)$ is isomorphic to the product $\Pi F_i$ of a finite number (maybe equal to one!) of field extensions $K\to F_i$ (this follows from the Chinese Remainder Theorem).
Notice also that since $A$ is finite-dimensional over $K$, it is an artinian ring, so that the Jacobson radical coincides with the nilpotent radical : $J(A)=Nil(A)$.
The correct implication is (as you suggested) : $\quad K/k \:\;\text {is purely inseparable} \Leftrightarrow A/Nil(A) =K$
Edit
Since I find this equivalence rather difficult, I'll upgrade my preceding indications to a concise proof.
$\boxed {\Longleftarrow}$
Notice first that the hypothesis forces $A=K\oplus Nil(A)$.
Consider the separable part of your extension: $k\subset K_{sep }\subset K$.
Since $K\otimes _k K_{sep }\subset K\otimes _k K$ is reduced (separable extensions are universally reduced!) , we must have $ K\otimes _k K_{sep }=K \subset K\oplus Nil(A)$ and thus $K_{sep }=k$, so that $k\subset K$ is indeed purely inseparable.
$\boxed {\Longrightarrow}$
It suffices to show that every element of $ K\otimes_k K\setminus K\otimes_k k$ is nilpotent.
And for that it suffices to show that for any $b\in K$ the set $ K\otimes_k k(b) \setminus K\otimes_k k$ is composed of nilpotents of $K\otimes_k k(b) $.
But $k(b) \simeq k[T]/(T^{p^r}-q) \;\; (q\in k, q=b^{p^r})$ by pure inseparability and so $K\otimes_k k(b) \simeq K[T]/(T^{p^r}-q) =K[T]/(T-b)^{p^r}$
In that last isomorphism the set $ K\otimes_k k(b) \setminus K\otimes_k k$ corresponds to the set $K[T]/(T-b)^{p^r} \setminus K$, and this last set is indeed composed of nilpotents. More precisely, it is the nilpotent radical $(\bar T-b)$ of $K[T]/(T-b)^{p^r}$.