Gerry Myerson gave a recipe for finding a symmetric solution, also providing a solution in a comment. I wanted to implement that while preserving the structure of the Don Giles matrix for order 2 (Fano plane). That is, I wanted the first four rows and first four columns to be the analogous pattern to the 7 by 7. Gerry's method amounts to this: take all 13 lines in $\mathbb F_3^3,$ where $\mathbb F_3$ is the field with three elements. The 13 lines can be parametrized by triples of $1,0,-1,$ where the first nonzero entry is $+1.$ There is a $1$ in the matrix for a pair of vectors out of the 13 if their dot product is $0 \pmod 3.$ I decided to label the 13 points using capital letters. Notice that $A \cdot A \equiv G \cdot G \equiv J \cdot J \equiv M \cdot M \equiv 0 \pmod 3.$
LABELS:
$$
\begin{array}{rrrr}
0 & 0 & 1 & K \\
0 & 1 & -1 & B \\
0 & 1 & 0 & H \\
0 & 1 & 1 & F \\
1 & -1 & -1 & G \\
1 & -1 & 0 & D \\
1 & -1 & 1 & J \\
1 & 0 & -1 & C \\
1 & 0 & 0 & E \\
1 & 0 & 1 & I \\
1 & 1 & -1 & M \\
1 & 1 & 0 & L \\
1 & 1 & 1 & A
\end{array}
$$
The order of both the rows and the columns of the matrix is alphabetical, $ABCDEFGHIJKLM.$ As I said, there is a $1$ in row $X,$ column $Y$ (and the reverse) when $X \cdot Y \equiv 0 \pmod 3.$
MATRIX:
$$ P \; = \;
\left( \begin{array}{cccc|ccc|ccc|ccc}
1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
1 & 0 & 0 & 0 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\
1 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 & 0 & 0 & 0 \\
1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\ \hline
0 & 1 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 1 & 0 & 0 \\
0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 \\
0 & 1 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 1 & 0 \\ \hline
0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 \\
0 & 0 & 1 & 0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 & 0 & 1 \\
0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 1 & 0 & 1 & 0 \\ \hline
0 & 0 & 0 & 1 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 1 & 1 & 0 & 0 \\
0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 1
\end{array}
\right).
$$
The result is that $P$ is symmetric and $P^2$ is the matrix with all 4's on the main diagonal and 1's everywhere else.