One possible approach would be to first assume that $f(x,y) = A(x) B(y)$, $g(x,y) = C(x) D(y)$. This turns the problem into ODE system which easier to solve. If we are in luck, this will give us a base for the solution space. Let's try it.
Combining the equations we get $\beta A'(x) / C(x) = - \alpha D'(y) / B(y).$ The only way this can happen is when both sides are equal to constant. It will turn out that this constant doesn't matter so simply set it to one. We get $g(x,y) = \beta A'(x)D(y),\quad f(x,y) = - \alpha A(x) D'(y).$ Plugging this into the first equation, we obtain
$A'(x) D'(y) = - \alpha A(x) D'(y) - \beta A'(x) D(y)$ so that $ {A'(x) + \alpha A(x) \over A'(x) } = -\beta{ D(y) \over D'(y)}.$ Again pick a constant, say $\gamma$, that both sides are equal to. We get $D(y) = E_{\gamma} \exp(-{\beta \over \gamma} y)$ and $A(x) = F_{\gamma} \exp({\alpha \over \gamma - 1} x)$. Therefore, the solution is just some exponential linear in $x$ and $y$.
Having obtained this intuition, let us use a more informed ansatz $f(x, y) = A \exp(-i p x -i q y),\quad g(x, y) = B \exp(-i p x -i q y)$ from which we get directly $-i A p = \alpha (A - B), \quad -i B q = \beta (B - A)$ $\beta A p = - \alpha B q $ $-i p q = \alpha q + \beta p $
that determines the amplitudes $A(p, q)$ and $B(p, q)$ as functions of momenta $p, q$ (and also the couplings $\alpha$ and $\beta$ of course), and also enforces relationship between the momenta. General solution is then obtained by adding (an infinite) number of these so called plane-wave solutions with coefficients picked so that they satisfy boundary conditions.