Generalization of the well-known combinatorial identity $\sum_{j=0}^n (-1)^j {n \choose j} j^k = 0$

Question

The well known identity $\sum_{j=0}^n (-1)^j {n \choose j} j^k =0$ when $k\lt n$ can be viewed as a particular case, applied to a uniform distribution, of a more general relation, defined for a general distribution $\{p_i\}, 1\le i\le n, \sum_{i=1}^np_i=1$, for $k \lt n$: $$\sum_{j=0}^n (-1)^j \sum_{|J|=j} P_J^k = 0$$ where $\sum_{|J|=j}$ is the sum over all the subsets of size j of $\{1..n\}$ , and $P_J=\sum_{j\in J}p_j$. This more general relation can be proven by proving an intermediate result, then the general relation follows. The intermediate result is that, for any distribution, $\sum_{|J|=j} P_J^k$ can be expressed as a weighted sum of binomial coefficients $\sum_{u=1}^k {n-k \choose j-u} \alpha_{k,u}$, where $\alpha_{k,u}$ are independent of j and $\alpha_{k,k}=1$. For example, one can show quite easily that: $\sum_{|J|=j} P_J = {n-1 \choose j-1}$ , i.e., the same as a uniform distribution, and that: $\sum_{|J|=j} P_J^2 = {n-2 \choose j-2} + {n-2 \choose j-1} \sum_{i=1}^n p_i^2 $. However the rest of the proof based on induction on the exponent is quite tedious, several pages long. Is there another proof of this relation ?

If an analytic form exists in the general case, when applied to a uniform probability, the result should be such that: $\alpha_{k,u}n^k=\sum_{i=1}^nS(k,i)i!{n \choose i}{k-i \choose u-i}$ where S is the Stirling number of 2nd order. This can be proven with the well-known relation of Stirling numbers $j^k=\sum_{i=0}^jS(k,i)i!{j \choose i}$ and Chu-Vandermonde identity.

Proof by induction can be found in: https://www.researchgate.net/publication/314243075_On_Von_Schelling_Formula_for_the_Generalized_Coupon_Collector_Problem

Answer 1

The following calculation is likely to be based on the wrong interpretation of the question since the OP informs us that this is a difficult combinatorial identity.

The sum in question is given by

$$k! [z^k] \prod_{q=1}^n (1-\exp(p_q z)).$$

Now observe that $1-\exp(p_q z)$ starts at $z$ namely with $-p_q z - p_q^2 z^2/2-\cdots$ so the product starts at $[z^n]$ and hence the coefficients on $[z^k]$ where $k\lt n$ are zero, which is the claim.

Answer 2

It appears that what the OP is seeking is the following formula, which follows by inspection, namely that when $j\ge k$ we have

$$\sum_{|J|=j} P_J^k = \sum_{q=1}^k \sum_{|J|=j} \sum_{Q\subseteq J, \; |Q|=q} k! [z^k] \prod_{r\in Q} (\exp(rz)-1) \\ = k! [z^k] \sum_{q=1}^k \sum_{|J|=j} \sum_{Q\subseteq J, \; |Q|=q} \prod_{r\in Q} (\exp(rz)-1) \\ = k! [z^k] \sum_{q=1}^k \sum_{|J|=j} [u^q] \prod_{r\in J} (1+ u(\exp(rz)-1)) \\ = k! [z^k] \sum_{q=1}^k [v^j] [u^q] \prod_{r\in P} (1+ v(1+ u(\exp(rz)-1))) \\ = k! [z^k] \sum_{q=1}^k [v^j] [u^q] \prod_{r\in P} (1+ v + uv(\exp(rz)-1)) \\ = k! [z^k] \sum_{q=1}^k [v^j] [u^q] \prod_{r\in P} (1+ v -uv + uv\exp(rz)).$$

This is

$$k! [z^k] \sum_{q=1}^k [v^j] [u^q] (1+v-uv)^n \prod_{r\in P} \left(1 + \frac{uv}{1+v-uv} \exp(rz)\right) \\ = k! [z^k] \sum_{q=1}^k [v^j] [u^q] (1+v-uv)^n \sum_{Q\subseteq P} \left( \frac{uv}{1+v-uv}\right)^{|Q|} \prod_{q\in Q} \exp(qz) \\ = \sum_{q=1}^k [v^j] [u^q] (1+v-uv)^n \sum_{Q\subseteq P} \left( \frac{uv}{1+v-uv}\right)^{|Q|} \left(\sum_{l\in Q} l\right)^k.$$

This becomes

$$\sum_{q=1}^k [v^j] [u^q] \sum_{Q\subseteq P} (uv)^{|Q|} (1+v-uv)^{n-|Q|} \left(\sum_{l\in Q} l\right)^k \\ = \sum_{Q\subseteq P, \; |Q|\le j} \left(\sum_{l\in Q} l\right)^k [v^{j-|Q|}] \sum_{q=|Q|}^k [u^{q-|Q|}] (1+v-uv)^{n-|Q|} \\ = \sum_{Q\subseteq P, \; |Q|\le j} \left(\sum_{l\in Q} l\right)^k [v^{j-|Q|}] \sum_{q=|Q|}^k {n-|Q|\choose q-|Q|} (-1)^{q-|Q|} v^{q-|Q|} (1+v)^{n-q} \\ = \sum_{Q\subseteq P, \; |Q|\le j} \left(\sum_{l\in Q} l\right)^k \sum_{q=|Q|}^k {n-|Q|\choose q-|Q|} (-1)^{q-|Q|} [v^{j-q}] (1+v)^{n-q} \\ = \sum_{Q\subseteq P, \; |Q|\le j} \left(\sum_{l\in Q} l\right)^k \sum_{q=|Q|}^k {n-|Q|\choose q-|Q|} (-1)^{q-|Q|} {n-q\choose j-q}.$$

Note that the subsets with $j\ge |Q|\gt k$ do not contribute because the range of the second inner sum is up to $k.$ We re-write one more time to isolate the dependency on $j$ and get

$$\bbox[5px,border:2px solid #00A000]{ \sum_{q=1}^k {n-q\choose j-q} \sum_{Q\subseteq P, \; 1\le |Q|\le q} {n-|Q|\choose q-|Q|} (-1)^{q-|Q|} \left(\sum_{l\in Q} l\right)^k.}$$

Here we have started $q$ at one because the empty set does not contribute, with the power sum being zero. It thus appears that the queried constants are given by

$$\alpha_{k,q} = \sum_{Q\subseteq P, \; 1\le |Q|\le q} {n-|Q|\choose q-|Q|} (-1)^{q-|Q|} \left(\sum_{l\in Q} l\right)^k.$$

Answer 3

This is Exercise 6.50 (b) in my Notes on the combinatorial fundamentals of algebra, version of 10 January 2019. (The numbering of the exercises might change, but you can find the version of 10 January 2019 archived on github.)

Note that the proof I give there is really written up "from scratch" (and I spend a while ruminating on trivialities about sums of products because my ring is noncommutative; not sure if you need that generality). You probably will find Lemma 7.222 trivial, and maybe Lemma 7.223 known. Parts (c) and (d) of that exercise give a formula for the sum in the case when $k=n$ (rather than $k). Part (a) gives a formula in the general case, but your mileage may vary about how useful it is.