An optimal stopping problem on Durrett's book

Question

This is Exercise 4.1.14. on page 187 on Rick Durrett's book Probability: Theory and Examples.
Let $X_n,n\ge 1$ be i.i.d. with $EX_1^+<\infty$ and let $Y_n=\max_{1\le m\le n}X_m-cn$.

(i) Let $T=\inf\{n:X_n>a\},p=P(X_n>a)$ and compute $EY_T.$

(ii) Let $\alpha$ be the unique solution of $E(X_1-\alpha)^+=c.$ Show that $EY_T=\alpha$ in this case and use the inequality $Y_n\le\alpha+\sum_{m=1}^n((X_m-\alpha)^+-c)$ for $n\ge 1$ to conclude that if $\tau\ge1$ is a stopping time with $E\tau<\infty,$ then $EY_\tau\le\alpha$.

By conditional on $T$, I calculate $EY_T=E[X_1-a]^+-\frac{c}{p}$. When plug $\alpha$ in, it equals to $c-c/p$. I don't know why it equals to $\alpha$. And the last question is just by Wald's equation.

Answer 1

You really need to separate the two parts of this exercise. Part (i) has nothing to do with Wald's formula, it is just general probability.

Let's start with the general formula $$E[(W-a)^+]=\int_a^\infty \{1-F_W(t)\}\,dt.\tag1$$

Now in part (a) of the exercise assume $P(X>a)>0$, and $T=\inf\{n:X_n>a\}$ so that $$P(X_T>t)=\sum_{n=1}^\infty P(X_n>t, X_1\leq a,\dots, X_{n-1}\leq a) =\sum_{n=1}^\infty (1-F(t))F(a)^{n-1}={1-F(t)\over 1-F(a)}.$$ Using the formula (1) we get $$E(X_T-a)=\int_a^\infty {1-F(t)\over 1-F(a)}\,dt,$$ so that $E(X_T)=a+{E[(X_1-a)^+]\over 1-F(a)}.$

Also, $T$ is a geometric random variable with $p=P(X>a)=1-F(a)$ so that $E(T)=1/(1-F(a))$.

Finally $\max_{1\leq m\leq T}X_m=X_T,$ since $X_m$ takes its maximum value at time $T$. Therefore $$E(Y_T)=E(X_T)-cE(T)=a+{E[(X_1-a)^+]-c\over 1-F(a)}.$$

Can you do part (ii) now?