This is essentially Exercise 1.21 in Folland's Real Analysis, which states the following: If $\mu^*$ is an outer measure induced by a premeasure and $\overline{\mu}$ is the restriction of $\mu^*$ to the $\mu^*$-measurable sets, then $\overline{\mu}$ is saturated.
Definition: Folland says a measure $\overline{\mu}$ on a space $(X, \mathcal{M})$ is saturated if every locally measurable set is measurable, where a set $E$ is locally measurable if and only if $E \cap A$ is measurable for every $A \in \mathcal{M}$ with $\overline{\mu}(A) < \infty$.
I'm having trouble showing that when $E$ is a locally measurable set with $\mu^*(E) = \infty$, then $E$ is measurable. (The finite case is not hard.)
Here's what I have so far. Write $\mu_0$ for the premeasure, $\mathcal{A} \subset \mathcal{P}(X)$ for the algebra on which $\mu_0$ is defined, and $\mathcal{M}$ for the collection of all $\mu^*$-measurable sets. Also, let $\mathcal{A}_{\sigma}$ be the collection of countable unions of sets in $\mathcal{A}$. The hint is to use an earlier exercise: for any $\varepsilon > 0$, there is $A \in \mathcal{A}_{\sigma}$ with $E \subset A$ and $\mu^*(A) \leq \mu^*(E) + \varepsilon$. So I obtain $E = \bigcup_{j=1}^{\infty} E \cap A_j$, where each $A_j \in \mathcal{A}$. Then I want to use the locally measurable property, but it may be the case that $\mu_0(A_j) = \infty$ for some $j_0$.
Any ideas? We don't have any assumption that $\overline{\mu}$ is $\sigma$-finite, for example...