In each of these cases, you have a group $G$ (with various additional structures), and a space of functions on that group, which is therefore naturally a $G \times G$ module under left and right translation.
But "where there's a function space, there's a distribution space". In each case the function space has a natural topology, and you can consider the space of continuous linear maps from the function space to the base field. This doesn't see the pointwise product structure on functions, but it has a new product structure (generally noncommutative) given by convolution; and this new space also has a $G \times G$ action by transport of structure. If $G$ is compact, then both the function space and its dual will (in all the examples I can think of) look like some kind of completion of the direct sum of $V \otimes V^*$ over all finite-dimensional irreps, but the functions and the distributions will generally use different notions of completion.
For $\mathbb{C}[G]$ when $G$ is finite, there is the (commutative) pointwise-product-of-functions structure, but the linear dual of $\mathbb{C}[G]$ is canonically $\mathbb{C}[G]$ itself and this picks up the convolution product structure. And you get the same action of $G \times G$ both ways round. For $L^2$ functions on a compact Lie group you get the same thing happening: the continuous dual of $L^2(G)$ is $L^2(G)$ itself, so you just get the same space, with the same topology. But when $G$ has positive dimension there's no natural pointwise product of two $L^2$ functions, so you miss part of the picture.
For $\mathcal{O}[G]$ where $G$ is a reductive algebraic group, you have a function space and a distribution space, and the functions have a pointwise product and the distributions a convolution product. But the two spaces have very different topologies. The topology on $\mathcal{O}[G]$ is the finest locally convex topology, for which every subspace is closed and every linear functional is continuous. Since this is an insanely strong topology, the dual topology will have to be very weak, and if you chase through definitions for a while, you can convince yourself that the dual of $ \mathcal{O}[G] = \bigoplus_V V \otimes V^*$ is the direct product $ \mathcal{O}[G]^* = \prod_V V \otimes V^*. $
In general function spaces won't be self-dual -- that's a really special thing about $L^2$ spaces. So the lack of symmetry you mention, that there are two natural objects which are dual to each other but not isomorphic, is the generic case.