Daniell proved a theorem on the existence of random sequences (see page 13 of these notes):
Let $(S_n,\mathbf{S_n})$ be a sequence of Borel spaces and let $\mu_n$ be a projective sequence of probability measures on $ (S_n:n\in > \mathbb{N})$. Then, there's a unique probability measure $ \mathbb{P}$ on the product $ \sigma$-algebra of $ \prod_{n\geq 0}S_n$, such that, for all $ n$, for all $ E \in S_0\otimes \cdots\otimes S_n$, $\mathbb{P}[E\times \prod_{i\geq n+1}S_i] = \mathbb{P}[E]$
I'm trying to find a counterexample when we don't assume that $(S_n,\mathbf{S_n})$ are Borel.
Perhaps the simplest examples of non-Borel spaces are finite measurable spaces for which not all singleton sets are measurable.
Non-Borel sets are also non-Borel spaces, but this seems like a complicated way to go. (Quick, relevant question: are all probability measures on such spaces inner regular?)
The main idea of the proof of this theorem seems to be that we can approximate measurable sets on $([0,1],\mathcal{B}[0,1])$ by compact sets, and use the fact that $(S_n:n\in \mathbb{N})$ have the same structure, since they are Borel.
I still don't have a clear idea of which properties any counterexample must satisfy, which is probably why I haven't got very far!
Thank you.