Let $X$ be a compact Hausdorff space. Let $\mu$ be a Borel, probability measure on $X$. Does it automatically follow that $\mu$ is regular? That is, for all Borel $E \subset X$, must we have $\mu(E) = \inf \mu(U) = \sup \mu(K)$ where the infimum is taken over open $U \supset E$ and the supremum is taken over compact $K \subset E$? I know this is true when $X$ is a metric space or, more generally, when every open subset of $X$ is $\sigma$-compact. I imagine this fails in general though.
I have a proposed counterexample which is a long way from being complete, but I'll put it here anyway. Let $I$ be an uncountable index set. Denote the product of $I$ many copies of the discrete space $\{0,1\}$ by $2^I$ and give it the (compact) product topology. I would like to give $2^I$ it's "product measure" as well, but I don't understand products of infinite families of measure spaces. Luckily, $2^I$ is, in a natural way, a compact group, so we can use the Haar measure which does what I wanted the product measure to do. Let $E$ be the set of elements of $2^I$ with countable support.
Issue 1: Is $E$ Borel? I don't see why it should be...
Assuming $E$ is Borel, it must have measure zero. Consider the translates $xE = \{ y \in 2^I : y_i = x_i \text{ for all but countably many } i \in I\}$ of $E$. By choosing a sequence of points $x_1,x_2,x_3,\ldots \in X$ such that any two differ in uncountably many coordinates (this is possible), we see that the translates $x_1E,x_2E,x_3E,\ldots$ are disjoint. So, if $E$ had positive measure, we get that $\mu(x_1E \cup x_2E \cup \ldots) = \mu(E) + \mu(E) + \ldots = \infty$ by translation invariance of the measure. This contradicts $\mu(2^I) =1$.
Now, I would like to use $E$ to contradict outer regularity. So how is it that one can find open $U \supset E$? For $F \subset I$ finite, let $U_F = \{x \in 2^I : x_i = 0 \text{ for all } i \in F\}$ (basic open set). I think essentially the only way to cover $E$ by an open set is to choose an uncountable family $F_j$ of disjoint finite subsets of $I$ and to consider $U = \bigcup_j U_{F_j}$. This covers $E$ since, if $x \in E$, then the support of $x$ is countable, so $x$ is nonzero on countably many of the $F_j$, so $x$ is zero on some particular $F_j$ and $x \in U$.
Issue 2: Is it true that $U$ should have $\mu(U)=1$? For some reason I feel maybe it should.
If both of the above issues are resolved, then $2^I$ is not regular since $\mu(E) = 0$ and $\inf \mu(U) = 1$.