Heisenberg's uncertainty principle can be given a precise mathematical formulation that, as far as I can tell, has nothing to do (mathematically) with the incompleteness theorem.
One version is a statement about how spread out a function on $\mathbb{R}$ can be relative to how spread out its Fourier transform can be; roughly speaking, a function and its Fourier transform cannot simultaneously be localized (physically the function can be interpreted as describing the position of some particle and its Fourier transform can be interpreted as describing its momentum, but the mathematical statement is independent of this interpretation). A more general version is a statement about the variances of noncommuting random variables. In this form it is essentially an application of the Cauchy-Schwarz inequality.
If there are similarities to the incompleteness theorem, they are philosophical, not mathematical.