We have a problem the arises in asset management. I think (and hope) it raises interesting enough questions for this forum to consider it. We made pretty extensive searches of the literature and find things that talk around this question, but nothing directly dealing with the issue.
Background
We have time series data for assets, from which we calculate a correlation matrix. For 5 assets using Mathematica it might look something like this:
m = Correlation[data]
{{1.0, 0.635562, 0.698852, 0.404792, -0.32746}, {0.635562, 1.0, 0.410075, 0.314375, -0.0636438}, {0.698852, 0.410075, 1.0, 0.374416, -0.260137}, {0.404792, 0.314375, 0.374416, 1.0, 0.293135}, {-0.32746, -0.0636438, -0.260137, 0.293135, 1.0}}
m //TableForm
1.000000, 0.635562, 0.698852, 0.404792, -0.32746
0.635562, 1.000000, 0.410075, 0.314375, -0.0636438
0.698852, 0.410075, 1.000000, 0.374416, -0.260137
0.404792, 0.314375, 0.374416, 1.000000, 0.293135
-0.32746, -0.0636438, -0.260137, 0.293135, 1.000000
In asset management one wants diversification. If two or more assets in a portfolio correlate too highly it concentrates rather than diversifying the risk.
What we want
We want an approach or method to construct a portfolio of assets that minimizes the portfolio’s “concentration” risk.
I’ll illustrate concentration risk with a couple of examples below, but first...
Why this is an interesting problem or question?
A couple of things make this an interesting and challenging question:
While related to “efficient frontier” we have no assumptions about future performance for the individual instruments to use.
Minimizing variance gives an answer, but not an intuitively satisfying or even useful one.
Principal Components Analysis seems a natural way to look at this, but also doesn’t appear to give us what we need.
We’ve looked at using entropy maximization, but while one of our guys familiar with discrete entropy thought it seemed promising, when we tried thinking about this in terms of continuous entropy it proved a dead end.
A couple of simple examples to illustrate concentration risk
It's easiest to understand what we want to achieve if we look at a portfolio of 3 assets in a thought experiment. Assume 2 of the instruments have correlations of 1 (100%) and the third at 0, it's correlation matrix would look like this:
1, 1, 0
1, 1, 0
0, 0, 1
From our perspective in this case, it would make sense to put 25% in each of the 2 correlated stocks and 50% in the uncorrelated one.
25%, 25%, 50%
This offsets the risk of concentrating in correlated instruments, while recognizing that the 100% correlated assets are in fact still different instruments whose correlation in the future may change.
One might make the case that as the two assets that have 100% correlation move the same, then a wide range of possible allocations could equally serve our purposes e.g:
50%, 0%, 50%
0%, 50%, 50%
10%, 40%, 50%
... or any such variations on the theme.
But, as we don’t know how their future correlation will evolve, we think the best and most intuitive solution remains at:
25%, 25%, 50%
Another example
In a portfolio of 5 assets with 4 having 100% correlation and 1 having 0% correlation the correlation matrix would look like the following:
1, 1, 1, 1, 0
1, 1, 1, 1, 0
1, 1, 1, 1, 0
1, 1, 1, 1, 0
0, 0, 0, 0, 1
and the portfolio allocation we want would have the following proportions:
12.5%, 12.5%, 12.5%, 12.5%, 50%
Of course the real world presents us with greater complication.
Things we’ve tried
Minimizing variance (promising but doesn’t work)
Someone suggested minimizing variance to do this, but as one can see it doesn’t produce an intuitive solution:
Some Mathematica code illustrating this follows:
For 3 assets:
m3 = {{1, 1, 0}, {1, 1, 0 }, { 0, 0 , 1}};
Array[x, Length@m3];
Minimize[{p.m3.p, Tr@p == 1 && And @@ Thread[p >= 0]}, p]
{1/2, {x[1] -> 1/4, x[2] -> 1/4, x[3] -> 1/2}}
This looks good. It gives us:
25%, 25%, 50%
but...
For 5 assets:
m5 = {{1, 1, 1, 1, 0}, {1 , 1, 1, 1, 0 }, {1 , 1, 1, 1, 0 }, {1 , 1,1, 1, 0 }, { 0, 0 , 0, 0, 1}};
p = Array[x, Length@m5];
Minimize[{p.m5.p, Tr@p == 1 && And @@ Thread[p >= 0]}, p]
{1/2, {x[1] -> 1/16, x[2] -> 1/4, x[3] -> 1/8, x[4] -> 1/16, x[5] ->1/2}}
Not so good as it gives us:
6.25%, 25%, 12.50%, 6.25%, 50%
So, minimizing variance doesn’t work even for this simple (if artificial) case let alone something more realistic.
A promising solution
One contributor to our discussion suggested a promising approach - at least for cases that do not have any negative correlations. Perhaps it would lead someone to suggest a more complete solution.
Again with Mathematica code: m = {{1, 1, 0}, {1, 1, 0}, {0, 0, 1}};
Tr /@ PseudoInverse[m]/Tr[ Tr /@ PseudoInverse[m]]
{1/4, 1/4, 1/2}
Exactly what we would want. Note: For those not familiar with Mathematica code the functions: “Tr” finds the trace of a matrix and “/@” maps a function to a list or matrix. The rest probably makes sense.
Another example for four assets: m = {{1, 1, 1, 0}, {1, 1, 1, 0}, {1, 1, 1, 0}, {0, 0, 0, 1}};
Tr /@ PseudoInverse[m]/Tr[ Tr /@ PseudoInverse[m]]
{1/6, 1/6, 1/6, 1/2}
Again, exactly what we want.
This works better than minimizing variance, but in a more real world example (the first one described in the post) we get something that doesn’t work:
m = {{1.0, 0.635562, 0.698852, 0.404792, -0.32746}, {0.635562, 1.0, 0.410075, 0.314375, -0.0636438}, {0.698852, 0.410075, 1.0, 0.374416, -0.260137}, {0.404792, 0.314375, 0.374416, 1.0, 0.293135}, {-0.32746, -0.0636438, -0.260137, 0.293135, 1.0}}
Tr /@ PseudoInverse[m]/Tr[ Tr /@ PseudoInverse[m]]
{0.267806, 0.0898877, 0.22403, -0.0541658, 0.472441}
In this case we have a negative allocation (-0.0541658) for the 4th asset, something that doesn’t make sense for what we want to achieve.
Conclusion
So, we need a kind of MinimizeVarianceIntuitively function. I hope all of this describes what we want to achieve clearly enough. Any suggestions or ideas for attacking this problem in completely different ways or for extending any of the things we’ve tried already much appreciated.
Many thanks,
Arun Garapata