I'm not sure I understand all of what you're saying, but from what I understand it seems that you've misunderstood the definition of convexity. In the slides you link to, convexity is introduced by the condition
$f_i(\alpha x + \beta y) \le \alpha f_i(x) + \beta f_i(y)$
for $\alpha+\beta=1$, $\alpha\ge0$, $\beta\ge0$ on the constraint functions $f_i$ in the constraints
$f_i(x)\le b_i\;.$
That is, if the constraints are fulfilled for $x$ and $y$, they are also fulfilled for all convex combinations of $x$ and $y$, that is, for all points on the line segment joining $x$ and $y$. Thus, the constraints define a convex set, which is a set containing all convex combinations of its points.
Thus, you don't have to think about whether you can approach or reach the boundary to decide whether a set is convex. A cube without its corners and a ball without its boundary are both convex, since these boundary points don't lie on line segments joining any other points. By contrast, if you were to remove the midpoints of the sides of a cube, it would no longer be convex, since those midpoints do lie on line segments joining other points, e.g. on the face diagonals – this is similar to the third example in the image.