Interesting question - just some thoughts:
I'd guess it's impossible to speak of a usual infinite sequence $(w_0, w_1, \ldots)$ as being palindromic. That's because knowledge of the first character would - by definition - imply knowing the value of some "last" character, which is a problem: There is usually no definite last character to speak of, as the sequence continues infinitely. Tell me a last char and I go further in the sequence and tell you a different one.
Take ABABABAB...
. Is this palindromic or not? Whenever the last character was an A
, it surely is, with a B
, it isn't. Now which one to take? It's feels to me kind of like asking what is $\lim_{x \to \infty} \sin x$.
The only case where it would make sense to speak of knowing the last char of a sequence is when it was constant from some point on. But in order to be palindromic, that means that our sequence has to be constant at all, because again, if it changed in the front, it would have to change at the end as well and thus wouldn't be constant from some point -> Contradiction. (Or simply because the beginning resembles the end!).
Your grammar example is even more tricky, because it doesn't even generate a well-defined infinite sequence, that is, converge to some function $w : \mathbb N \to \{0,1\}$. It produces partial, finite strings, but in the infinite case, the definition is meaningless. Every production step taken may change the whole sequence - except we'd always use either $0$ or $1$ - and thus the result is again constant.
Now what you might want to use is not an indexing $w : \mathbb N \to \{0, 1\}$ but a bi-infinite sequence $w : \mathbb Z \to \{0, 1\}$ such that e.g. $w(-z) = w(z)$ for all $z \in \mathbb Z$. Such a thing could rightfully be called palindromic. However remember that a palindrome only contains half information, the second half is irrelevant as it's just a copy. You may have p-adic numbers encoded in the forward half, perform usual operations with them, reverse and built your new palindrome. In such a way you could have operations defined, though I see little use in it.