Extraction of coefficients of a two-variable generating function

Question

The generating function for a certain two-variable recurrence relation that I'm working on is $$\sum_{m,n}a_{m,n}\,x^my^n=\frac{xy(1-x)(1-y)}{(1-x)^2(1-y)^2-xy}.$$

Question: How can you compute $$[x^my^n]\frac{xy(1-x)(1-y)}{(1-x)^2(1-y)^2-xy}$$ using basic calculus or complex analysis?

One reason that I'm having problems extracting the coefficients is that I can't come up with a factorization of the denominator (and I can't get Maple to do it, either). Maple does say that the zeros of the denominator look like:

Since there are two branches in the graph, maybe the denominator is factorable after all.

Another reason that I'm having problems is that I just don't have a lot of experience in dealing with generating functions in more than one variable. My only real idea has been to think of the denominator as a quadratic polynomial in $x$ with coefficients as polynomials in $y$, and then try a partial fraction decomposition first with respect to $x$ and then try to extract the coefficient of $y$. This quickly turns into an algebraic nightmare.

In any case, by a completely separate combinatorial analysis of the recurrence relation (which I'm deliberately not stating), I have found that $$a_{m,n}=\sum_{\ell}\binom{\ell+m-1}{m-\ell-1}\binom{\ell+n-1}{n-\ell-1}.$$ What I'd really like to do is learn how to obtain this result directly from the generating function.

Any ideas?

Answer 1

Using the geometric series expansion we obtain

\begin{align*} &\frac{xy(1-x)(1-y)}{(1-x)^2(1-y)^2-xy}\\ &\qquad=\frac{\frac{xy}{(1-x)(1-y)}}{1-\frac{xy}{(1-x)^2(1-y)^2}}\\ &\qquad=\frac{xy}{(1-x)(1-y)}\sum_{l=0}^\infty\left(\frac{xy}{(1-x)^{2}(1-y)^{2}}\right)^l\\ &\qquad=\sum_{l=0}^\infty\frac{x^{l+1}y^{l+1}}{(1-x)^{2l+1}(1-y)^{2l+1}} \end{align*}

We continue applying the binomial series expansion and using the binomial identity $$\binom{-p}{q}(-1)^q=\binom{p+q-1}{q}$$

We obtain \begin{align*} [x^my^n]&\sum_{l=0}^\infty\frac{x^{l+1}y^{l+1}}{(1-x)^{2l+1}(1-y)^{2l+1}}\\ &=\sum_{l=0}^\infty[x^{m-l-1}y^{n-l-1}](1-x)^{-2l-1}(1-y)^{-2l-1}\\ &=\sum_{l}\binom{-2l-1}{m-l-1}(-1)^{m-l-1}\binom{-2l-1}{n-l-1}(-1)^{n-l-1}\\ &=\sum_{l}\binom{l+m-1}{m-l-1}\binom{l+n-1}{n-l-1} \end{align*} and the claim follows.

Answer 2

This is a comment to follow Markus Scheuer's answer, but too long to fit there...

In the one-dimensional case, the corresponding generating function would be $\sum \alpha_nx^n=\frac{x(1-x)}{(1-x)^2-x}$, which has singularities at $x=\frac{3\pm\sqrt5}{2}$. So, by partial fractions this generating function can be written $$\frac{\frac{5-\sqrt5}{10}}{1-\frac{3+\sqrt5}{2}x}+\frac{\frac{5+\sqrt5}{10}}{1-\frac{3-\sqrt5}{2}x}-1,$$ and then the coefficients would be $$\alpha_n= \frac{5-\sqrt5}{10}\left(\frac{3+\sqrt5}{2}\right)^n+\frac{5+\sqrt5}{10}\left(\frac{3-\sqrt5}{2}\right)^n.$$ Therefore, in the one-dimensional case the explicit representation of the coefficients has no summation; it's just a linear combination of two powers (which, incidentally, consist of the odd Fibonacci numbers). The obvious follow-up question is this---is there any chance of expressing the coefficients of the two-variable generating function without a summation, just as a linear combination of powers?