I had minor problems convincing myself of the fact that the group described by Geoff exists (See also Derek Holt's answer to the previous version of this question), most notably that it has the prescribed order. So I spent some time on it, and want to share this more concrete version. Hopefully I didn't fumble this.
Inside the group of upper triangular 3x3 matrices (entries from $F_p$) we have the (often used) matrices
$$
A=\left(\begin{array}{ccc}1&1&0\\0&1&0\\0&0&1\end{array}\right),\quad
B=\left(\begin{array}{ccc}1&0&0\\0&1&1\\0&0&1\end{array}\right),\quad
C=\left(\begin{array}{ccc}1&0&1\\0&1&0\\0&0&1\end{array}\right),
$$
satisfying the relations $A^p=B^p=C^p=1, [A,B]=C, [A,C]=[B,C]=1$.
Using these we can realize that group as $m\times m$ upper triangular matrices, where $m=3n(n-1)/2$, using $n(n-1)/2$ blocks (sized 3x3) along the diagonal. Label the blocks with pairs of indices $(i,j), 1\le ii$, matrix $B$ in any block with label $(x,i),x
The entire group $G$ then consists of matrices with blocks
$$
g_{i,j}=\left(\begin{array}{ccc}1&u_i&v_{i,j}\\0&1&u_j\\0&0&1\end{array}\right)=A^{u_i}C^{v_{i,j}-u_ju_i}B^{u_j},
$$
where the $n(n+1)/2$ coefficients $u_i,v_{i,j}$ are arbitrary elements of $F_p$.
If there is a simpler concrete description of this group, I'm all ears :-).
Edit: Anyway, we have $x_i^p=1$ for all $i$, $[x_i,x_j]^p=1$ for $i