Rejecting $H_0: \mu = 0$ is a statement about the data. The data are not
consistent with drawing from a population with mean $0.$
$1.$ Strictly speaking, rejecting $H_0$ does not mean $H_1$ is true. The sample
mean $\bar X$ must have been relatively 'far' from $0$ as measured by the test statistic.
For example, suppose you have a random sample of size $n = 16$ from a normal population with $\sigma = 12$ and sample mean $\bar X = 6.75.$ Then
the test statistic is $Z = \frac{\bar X - 0}{\sigma/\sqrt{n}} = 6.75/3 = 2.25.$
You reject $H_0$ against $H_1$ at the 5% level when $|Z| > 1.96.$ And here
$|Z| = 2.25 > 1.96,$ so we reject $H_0.$
While rejecting $H_0$ does not say anything specific about how far $\mu$ is from $0,$ the information used in the test of this hypothesis allows you to make
a 95% confidence interval for $\mu:$
$$ \bar X \pm 1.96\sigma/\sqrt{n}.$$
In my example above, this computes to the interval $6.75 \pm 5.88$ or $(0.87, 12.63),$ which gives some information about how far from $0$ the population mean
$\mu$ must lie.
The truth is that $\mu$ either lies in this interval or not.
In this case it seems best to act as if $\mu \ne 0$ because 0 does not lie in the interval. If you go through life believing such intervals are correct, you
will be right 95% of the time; this time--probably Yes, maybe No.
$2.$ You cannot use the significance level 5% = 0.05 to make probability
statements about $\mu.$ The significance level is the probability your
decision to reject $H_0$ was wrong.
In my example, assuming $\mu = 0,$ the P-value is the probability of seeing
a value of the sample mean that makes $|Z| > 2.25.$ That computes to
a P-value of 0.024, which is smaller than 0.05. But $\bar X$ was
used in computing $Z$, so the P-value amounts to another probability statement about the
sample mean $\bar X,$ not about the population mean $\mu.$
In summary, you need to realize that statistical testing is an inductive
process, not a deductive one. In many parts of mathematics, you are given
axioms, from which you deduce other statements using logic. Statistics uses
mathematics, but not in exactly in that way for interpreting results of hypothesis tests and confidence intervals.
By contrast, in statistics
we begin with data and perhaps some assumptions about the population from
which the data came. We can say, "It is very unlikely your 16 observations are
a random sample from a normal population with mean 0 and SD 12." It is
up to you to decide to take the hint that $\mu \ne 0$ or to believe that
an event of probability 0.024 has occurred and insist that $\mu = 0$
in spite of the evidence from your 16 observations. We used mathematics
to find the probability 0.024, but we can't make true or false statements
whether nor not $\mu = 0.$
Statisticians make statements in very careful ways, trying to make statements
about data that help people make decisions about the populations the data
are sampled from. Some of the statements about hypothesis testing and
confidence intervals may seem vague and convoluted. These statements are intended to be
helpful in making decisions, but it takes some practice getting used to them.
It may help you to realize that statisticians see only the data and not
the whole populations. They can make valid statements about the data, but
only help you make informed guesses about populations.