Let $(\Omega ,\mathcal F,\mathbb P)$ a probability space. If $V\geq 0$ a.s. and integrable, Is it true that $$\forall \varepsilon>0, \exists \delta>0: \forall A\in \mathcal F, \mathbb P(A)<\delta\implies \int_A V\mathrm d \mathbb P\ \ ?$$
And if yes, how can I show it ? (this result looks strange to me)
Yes, it is true.
Since $V$ is integrable, it is in particular finite a.s. and hence $Y_n:=1_{\{V>n\}}V\stackrel{a.s.}{\rightarrow}0$, as $n\rightarrow\infty$. Dominated convergence then gives $EY_n\rightarrow0$.
Let $\varepsilon>0$ be given. For $A\in \mathcal{F}$, \begin{eqnarray*} \int_AVdP&=&\int_A(V-Y_n)dP+\int_AY_ndP\\ &\leq&nP(A\cap\{V\leq n\})+\int Y_n dP\\ &\leq&nP(A)+EY_n. \end{eqnarray*} Choose $n$ big enough as to make $EY_n<\varepsilon/2$. Then the claim follows by setting $\delta=\frac{\varepsilon}{2n}$.