To say that speed and delay are inversely proportional is to say that there's a constant $c$ such that speed is $c$ divided by delay. If that were the case, then you'd have speed times delay always equal to this constant, $c$. But you don't have that, e.g., $20\times.064=1.28$ while $30\times.036=1.08$. So I would say they aren't inversely proportional.
But from your mention of an "offset", I take it you are using some different meaning for the phrase, "inversely proportional." Perhaps you mean, ${\rm speed\ }={{\rm factor}\over{\rm delay\ }-{\rm\ offset}}$ Let me write that as $s={f\over d-p}$ where I'm using $p$ for offset because using $o$ for a variable is too creepy. Do some algebra to get $s(d-p)=f,\qquad sd-sp=f,\qquad sp+f=sd$ Now if you put in your four known pairs of $s$ and $d$ values, you get four equations for the two unknowns $p$ and $f$. If you're lucky, these four equations will have a single solution, and you win. If they don't, then there are ways of finding values of $p$ and $f$ that won't work exactly but won't be off by very much. But I don't propose to go into that until/unless you agree that $s=f/(d-p)$ is what you want, and you find the four equations have no solution, and you express some interest in knowing about approximate solutions.