From the comments I understood that your main concern is that $g(\alpha)$ is not defined and hence we can't talk of the symbol $$\int_{a}^{\alpha}g(x)\,dx$$ You are right here. But we have to talk of $\int_{a}^{\alpha}f(x)\,dx$ and not about integral of $g$ here.
The problem is handled by introducing another function $a(x)$ on $[a,\alpha]$ such that $a(x) = g(x)$ for $x \in [a, \alpha)$ and $a(\alpha) = h(\alpha)$. Now we can talk of the integral $\int_{a}^{\alpha}a(x)\,dx$ and note that we have $f(x) = a(x)$ if $x\in [a, \alpha]$ and $f(x) = h(x)$ if $x \in [\alpha, b]$ and then we can write $$\int_{a}^{b}f(x)\,dx = \int_{a}^{\alpha}a(x)\,dx + \int_{\alpha}^{b}h(x)\,dx$$ provided that both integrals on the right exist.
In reality we are not given functions like $g(x), h(x)$ but rather expressions in $x$ like $x + 1, 2x^{2} + \sin x$ in place of $g(x), h(x)$ and for these functions we do know that they are defined for the intermediate point $\alpha$.
Still suppose that there is a function $f$ which is defined and bounded on $[a, b)$ and we don't have any definition of $f$ for point $b$. Then as per definition of Riemann integral we can't talk about $\int_{a}^{b}f(x)\,dx$. What can we do now? Well we can define $f(b)$ to take any value (as per our wish) and then then talk of $\int_{a}^{b}f(x)\,dx$. As I mentioned in my comments, the values of a function at a finite number of points in an interval do not affect the existence/value of the Riemann integral of the function, hence we are at a liberty to choose any value of $f(b)$ and talk about integral of $f$ on $[a, b]$. This is precisely the way we think about integrals like $$\int_{0}^{1}\cos (1/t)\,dt$$ Limits are used for a different setup which goes by the name of Improper Riemann Integral. This setup is used when either the interval of integration is unbounded or if the function is unbounded on a bounded interval. Examples of both types are $$\int_{0}^{\infty}\frac{\sin x}{x} \,dx = \lim_{t \to \infty}\int_{0}^{t}\frac{\sin x}{x}\,dx,\, \int_{0}^{1}\frac{dx}{\sqrt{1 - x^{2}}} = \lim_{t \to 1^{-}}\int_{0}^{t}\frac{dx}{\sqrt{1 - x^{2}}}$$