Let $P$ be a (partially) ordered set, and let $A$ be a subset of $P$.
- We say $x\in P$ is an upper bound for $A$ if and only if $a\leq x$ for all $a\in A$.
- We say $s\in P$ is the supremum of $A$ if and only if two things happen:
- $s$ is an upper bound for $A$; that is, $a\leq s$ for all $a\in A$; and
- $s$ is the smallest upper bound for $A$; that is, if $a\leq x$ for all $a\in A$, then $s\leq x$.
The definitions for lower bound and infimum are dual:
- We say $y\in P$ is a lower bound for $A$ if and only if $y\leq a$ for all $a\in A$.
- We say $t\in P$ is the infimum of $A$ if and only if two things happen:
- $t$ is a lower bound for $A$; that is, $t\leq a$ for all $a\in A$; and
- $t$ is the largest lower bound for $A$; that is, if $y\leq a$ for all $a\in A$, then $y\leq t$.
Note that neither the supremum nor the infinmum is required to be in $A$. If they are, then they are also called the maximum and the minimum, respectively. The difference between the supremum and the maximum is that the maximum must be in the set, and the supremum does not have to. (In the real numbers, every set that has upper bound must have a supremum, but it does not have to have a maximum.)
You should be able to verify that you are correct that $\sqrt{2}$ is the supremum of your set $A$. And that any number larger than $\sqrt{2}$ is also an upper bound for $A$.
Note also that $0$ is the infimum of $A$, so you are incorrect in saying that $A$ does not have an infimum.
For $B$, the supremum is $2$, and the infimum is $-2$. Check that.
Now, in the case of the real numbers there are other ways to express the condition of being the smallest upper bound for $A$. It's possible you have seen it defined a different way. Here is a typical one:
Proposition. Let $A\subseteq \mathbb{R}$. Then $s$ is the supremum of $A$ if and only if:
for all $a\in A$, we have $a\leq s$; and
For all $\epsilon\gt 0$, there exists $a\in A$ (which may depend on $\epsilon$) such that $s-\epsilon\lt a\leq s$.
Proof. Suppose that $s$ is the supremum of $A$. Then certainly $a\leq s$ for all $a\in A$. Now let $\epsilon\gt 0$. Then, using the contrapositive of the second part of the definition of supremum, we see that since $s-\epsilon\lt s$, then $s-\epsilon$ cannot be an upper bound for $A$, so therefore there exists $a\in A$ such that $s-\epsilon\lt a$. This gives the condition in the proposition.
Conversely, suppose that $s$ satisfies the conditions in the theorem. To show that it is the supremum of $A$, we must show it is the smallest upper bound for $A$. Indeed, it is an upper bound by the first condition. Now, if $x\lt s$, then there exists $\epsilon\gt 0$ such that $x=s-\epsilon$. By the second condition of the proposition, there exists $a\in A$ such that $x\lt a\leq s$. Thus $x$ is not an upper bound for $A$. So we have shown: if $x$ is smaller than $s$, then $x$ is not an upper bound for $A$. The contrapositive of this is: if $x$ is an upper bound for $A$, then $x$ is not smaller than $s$ (i.e.,, $s\leq x$). This proves $s$ is the supremum. $\Box$