This is a pinhole camera model
(I don't get, is there [R t], or (R, t)). This formula is used to model the projection from a space point M to an image point m. Projection drawing:
Tilde over vector, means that "1" is added to that vector as the element. M is the coordinate of the point in the 3d space, and m is the coordinate of the point in the picture, f is the focal length of the camera, and s is the pixel aspect ratio. (R, t) describes the 3D transformation between the world coordinate system, in which the rectangle is described, and the camera coordinate system.
It is unclear to me, what does "t" letter and (R, t) after A mean, and how, by inserting the 3D coordinates (pixel aspect ratio = 1) of the rectangle corners to the formula we get this: $ \begin{aligned} \lambda_1\tilde{\mathbf{m}}_1&=\mathbf{A}\mathbf{t} \\ \lambda_2\tilde{\mathbf{m}}_2&=w\mathbf{A}\mathbf{r}_1+\mathbf{A}\mathbf{t} \\ \lambda_3\tilde{\mathbf{m}}_3&=h\mathbf{A}\mathbf{r}_2+\mathbf{A}\mathbf{t} \\ \lambda_4\tilde{\mathbf{m}}_4&=w\mathbf{A}\mathbf{r}_1+h\mathbf{A}\mathbf{r}_2+\mathbf{A}\mathbf{t} \end{aligned} $
I found this formula in this document (page 13).