Usually in probability theory, for a random variable whose value is in $\mathbb{R}$, we talk about its cumulative distribution function $F(x)$ and then its density $f(x)$, in good enough cases $F'(x)=f(x)$.
That's the setup I'm familiar with, so I got annoyed when physicists talk about unnormalized "densities". E.g. if the probabilistic density of the position of a particle on $\mathbb{R}$ is equal to 1 everywhere, that means it is equally likely to appear anywhere. More generally you can imagine them talking about a non-negative function $f(x)$ being the density of something with the density $f(x)$ is not integrable on $\mathbb{R}$ but locally integrable, i.e. $\int_{[a,b]}f(x)dx$ make sense for $a\leq b, a, b\in\mathbb{R}$. As $\int_{\mathbb{R}}f(x)dx$ is undefined ($=\infty$), one cannot divide by it to normalize.
Is there a mathematical way to make sense of such statements? There is one I have in mind, namely, one can talk about the density of some random variable $X$ up to a scalar multiple, such that for any intervals $[a,b]\subset [c,d]$ we can express the conditional probability as a quotient of integrals:
$P(X\in[a,b] \big| X\in[c,d])=\dfrac{\int_{[a,b]}f(x)dx}{\int_{[c,d]}f(x)dx}.$
I only know very basic probability theory so I don't know if this makes sense. Am I allowed to interpret unnormalized probabilistic density functions this way? Is this what physicists mean? Or are there any other interpretations? Do I have to worry about something else when thinking about things in this way?