I think this is easier with the definition of faces as the intersection with supporting hyperplanes. But let's give it a shot with this definition.
Obviously, the set $\DeclareMathOperator{\conv}{conv}F = \conv(\widehat B \cup \{a\})$ is a subset of $\conv(B \cup \{a\}) = P$.
Since $P$ is a $d$-polytope, $a$ is not in the $d-1$-space affinely spanned by $B$. That is, assuming we are in $d$-space, $a$ is not in the hyperplane $H$ containing $B$. It follows that each point $x \in P$ (except the apex $a$) can be written as $\alpha x' + (1 - \alpha)a$ for a unique $x' \in B$ and a unique $\alpha \in [0,1]$. Why? The line $L$ from $a$ to $x$, together with $H$, spans the whole $d$-space, and the intersection of a transverse codimension-$(d-1)$ subspace and a codimension-$1$ subspace has codimension $d$, i.e., is a point.
Now suppose $x, y \in P$ and $\alpha \in (0,1)$ such that the point $z = \alpha x + (1 - \alpha) y$ is in $F$. Since $x, y \in P$, we can write $x = \beta x' + (1 - \beta) a$ and $y = \gamma y' + (1-\gamma)a$ for some $x', y' \in B$ and some $\beta, \gamma \in [0,1]$. So \begin{align} z &= \alpha (\beta x' + (1 - \beta) a) + (1 - \alpha)(\gamma y' + (1-\gamma)a) \\ &= \alpha \beta x' + \gamma y' - \alpha \gamma y' + (1 - \gamma - \alpha \beta + \alpha \gamma) a. \end{align} Let $\delta = \alpha \beta + \gamma - \alpha \gamma$. (You can check that $\delta = 0$ only if $x = y = z = a$.) \begin{align} z &= \delta \left( \frac{\alpha \beta}{\delta} x' + \frac{\gamma - \alpha \gamma}{\delta} y' \right) + (1 - \delta) a. \end{align} But $\frac{\alpha \beta}{\delta} x' + \frac{\gamma - \alpha \gamma}{\delta} y'$ is a convex combination of points in $B$, so it is a point in $B$. Since $z \in F$, $z = \kappa z' + (1 - \kappa) a$ for some $z' \in \widehat B$. So $\kappa = \delta$ and $z' = \frac{\alpha \beta}{\delta} x' + \frac{\gamma - \alpha \gamma}{\delta} y'$. But since $\widehat B$ is a face, we must have $x', y' \in \widehat B$. Therefore, $x, y \in F$.
This shows that $F$ is a face.