$\def\a{\alpha} \def\at{\widetilde\a} \def\b{\beta} \def\d{\delta} \def\j{\psi} \def\J{\Psi} \def\JT{\widetilde\J} \def\l{\lambda} \def\L{\Lambda} \def\p{\pi} \def\w{\omega} \def\det{\mathrm{det'}\,} \def\dt{\frac{\partial}{\partial t}} \def\diag{\mathrm{diag}} \def\su{\mathfrak{su}}$ A simpler determinant
Let's first have a look at a related problem, to determine $\det\left(\dt - i \a\right)$ where $\a$ is some scalar. (Recall that the analog of commutation is multiplication.) Some of the difficulties of the full problem exist already in this simpler one.
We are looking for the eigenvectors of the operator $\dt - i \a$, that is, for solutions to the equation $\left(\dt - i \a\right)\j = \L \j.$ The eigenvectors are $\j(t) = e^{i\w t}$, with eigenvalues $\L = i(\w-\a)$. Imposing periodic boundary conditions, $\j(\b) = \j(0)$, we find $\w_m = 2m\p/\b$, where $m\in\mathbb{Z}$. Thus, the eigenvalues are quantized, $\L_m = i\left(\frac{2 m\p}{\b}-\a\right).$
We assume that $\a\ne 2m\p/\b$ for any $m\in\mathbb{Z}$, so $\L_m \ne 0$. In particular, $\L_0 = -i\a \ne 0$. Then the product $\det$ ranges over all $m$, $\begin{eqnarray*} \det\left(\dt - i \a\right) &=& \prod_{m=-\infty}^\infty \L_m \\ &=& \L_0 \prod_{m\ne 0} \L_m \\ &=& \L_0 \prod_{m\ne 0} i\left(\frac{2 m\p}{\b}-\a\right) \\ &=& \L_0 \prod_{m\ne 0} \frac{2m\p i}{\b} \left(1-\frac{\a\b}{2\p m}\right) \\ &=& \L_0 \left(\prod_{m=1}^\infty \left(\frac{2m\p}{\b}\right)^2 \right) \prod_{m=1}^\infty \left(1-\left(\frac{\a\b}{2\p m}\right)^2\right) \\ &=& \L_0 \frac{2}{\a\b}\sin\frac{\a\b}{2} \prod_{m=1}^\infty \left(\frac{2m\p}{\b}\right)^2 \\ &=& -\frac{2i}{\b}\sin\frac{\a\b}{2} \prod_{m=1}^\infty \left(\frac{2m\p}{\b}\right)^2. \end{eqnarray*}$ Above we have used the infinite product representation for the sine function, $\sin x = x\prod_{m=1}^\infty \left(1-\frac{x^2}{m^2\pi^2}\right)$.
The determinant in general
We assume the algebra is a finite dimensional semisimple Lie algebra over $\mathbb{R}$ or $\mathbb{C}$.
Consider the equation $\begin{equation*} \dt\J - i [\a,\J] = \L \J.\tag{1} \end{equation*}$ The objects $\a$ and $\J$ are now $n\times n$ matrices in the adjoint representation of the Lie algebra where $n$ is the number of elements in the algebra. We are free to apply a similarity transformation to (1), and instead solve $\begin{equation*} \dt\JT - i [\at,\JT] = \L \JT,\tag{2} \end{equation*}$ where $\widetilde A = U^{-1}AU$, and $U$ is independent of time. In fact, we can diagonalize $\alpha$ with this transformation, $\at = \diag(\l_1,\ldots,\l_n)$, so $\at$ is in the Cartan subalgebra. (See the example for $\su(3)$ below.)
Let's choose as our basis for matrices $e_{ab}$, where $a,b=1,\ldots,n$, such that the components are $(e_{ab})_{ij} = \d_{ai} \d_{bj},$ where $\d$ is the Kronecker delta. Then $[\at,e_{ab}] = (\l_a - \l_b) e_{ab},$ so the $e_{ab}$s are eigenvectors of $\at$. (This is straightforward to prove using the fact that $\at = \sum_a \l_a e_{aa}$.)
Thus, the eigenvectors of (2) are $\JT = e_{ab}e^{i\w t}$, since $\begin{eqnarray*} \dt\JT - i [\at,\JT] &=& \left(i\w - i(\l_a-\l_b)\right)\JT. \end{eqnarray*}$ Imposing the boundary condition $\JT(\beta) = \JT(0)$ we find $\L_m^{ab} = i\left(\frac{2m\pi}{\beta} - (\l_a-\l_b)\right).$ For every $a,b$ there is an infinite tower of eigenvalues corresponding to the index $m$.
Notice that $\L\ne 0$ if and only if $\l_a-\l_b \ne 2m\p/\b$. For $m\ne0$ and a generic $\b$, $\l_a-\l_b \ne 2m\p/\b$. For $m=0$, $\L \ne 0$ if and only if $\l_a - \l_b \ne 0$.
Therefore, $\begin{eqnarray*} \det\left(\dt - i [\a,\,]\right) &=& \left(\prod_{\l_a\ne\l_b} \L_0^{ab} \right) \left(\prod_{m\ne0\atop a,b} \L_m^{ab}\right) \\ &=& \left(\prod_{\l_a\ne\l_b} \L_0^{ab}\right) \left(\prod_{m\ne0\atop a,b} i\left(\frac{2m\pi}{\beta} - (\l_a-\l_b)\right) \right) \\ &=& \left(\prod_{\l_a\ne\l_b} \L_0^{ab}\right) \left(\prod_{a,b} \frac{2}{(\l_a-\l_b)\b}\sin\frac{(\l_a-\l_b)\b}{2} \prod_{m=1}^\infty \left(\frac{2m\p}{\b}\right)^2\right) \\ &=& \left(\prod_{\l_a\ne\l_b} \L_0^{ab}\right) \left(\prod_{\l_a\ne\l_b} \frac{2}{(\l_a-\l_b)\b}\sin\frac{(\l_a-\l_b)\b}{2}\right) \left(\prod_{m=1\atop a,b}^\infty \left(\frac{2m\p}{\b}\right)^2\right) \\ &=& \left(\prod_{\l_a\ne\l_b} -\frac{2i}{\b}\sin\frac{(\l_a-\l_b)\b}{2} \right) \left(\prod_{m=1\atop a,b}^\infty \left(\frac{2m\p}{\b}\right)^2\right). \end{eqnarray*}$ Convince yourself that the correct way to interpret $\prod_{\l_a=\l_b} \frac{2}{(\l_a-\l_b)\b}\sin\frac{(\l_a-\l_b)\b}{2}$ is $\prod_{\l_a=\l_b}1 = 1$. Intuitively, $\lim_{x\to 0}\frac{\sin x}{x} = 1$.
Notice the similarity between this result and the previous one. The formula for $\det\left(\dt-i[\a,\,]\right)$ given in the question statement is missing some factors.
For readers unfamiliar with such calculations, which appear in physics quite regularly, and disturbed by the product $\prod_{m=1\atop a,b}^\infty \left(\frac{2m\p}{\b}\right)^2$, it may relieve you (a little) to know that this is just $\det\left(\dt\right)$ and typically we are interested in objects of the form $\frac{\det\left(\dt-i[\a,\,]\right)}{\det\left(\dt\right)}.$
Example: Eigenvalues of $\at$ for $\su(3)$
We use the structure constants typical to physics, $f_{abc}$, to construct the adjoint representation $(t_a)_{bc} = -i f_{abc}$.
The Cartan subalgebra is two dimensional. Simultaneously diagonalize $t_3$ and $t_8$ so $\at = x \widetilde t_3 + y \widetilde t_8$, where $x$ and $y$ are some constants. To be explicit, $t_3 = \textstyle\left( \begin{array}{cccccccc} 0 & -i & 0 & 0 & 0 & 0 & 0 & 0 \\ i & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & -\frac{i}{2} & 0 & 0 & 0 \\ 0 & 0 & 0 & \frac{i}{2} & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & \frac{i}{2} & 0 \\ 0 & 0 & 0 & 0 & 0 & -\frac{i}{2} & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right)$ $t_8 = \textstyle\left( \begin{array}{cccccccc} 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & -\frac{i \sqrt{3}}{2} & 0 & 0 & 0 \\ 0 & 0 & 0 & \frac{i \sqrt{3}}{2} & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & -\frac{i \sqrt{3}}{2} & 0 \\ 0 & 0 & 0 & 0 & 0 & \frac{i \sqrt{3}}{2} & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right)$ Up to permutation, $\at = \diag\left( x, -x, \frac{1}{2}(x+\sqrt{3} y), \frac{1}{2}(x-\sqrt{3} y), -\frac{1}{2}(x+\sqrt{3} y), -\frac{1}{2}(x-\sqrt{3} y), 0,0\right).$ The diagonal elements above are the eigenvalues $\l_a$.