Portfolio optimisation

Portfolio optimisation is how you decide how much money to put in each asset. This post explores the idea by drawing thousands of random portfolios, which shows the shape of the problem and makes the intuition easier to see than an optimiser would.

This post pulls the whole Finance stats series together: it needs expected returns, standard deviations, correlations and the Sharpe ratio.

The question

You have money to split across four asset classes. Each has an expected yearly return, a standard deviation, and correlations with the others. What mix should you hold?

Made-up yearly figures:

Equities move closely together (correlation 0.8). Bonds move slightly against equities (−0.05 to −0.10). Gold is nearly unrelated to the rest.

The two formulas you need

For weights $\key{w}$ (one per asset, adding up to 1), expected returns $\good{\mu}$ and covariance matrix $\risk{\Sigma}$. Romero and Balch’s What Hedge Funds Really Do (chapter 10) writes these as $\good{r_p} = \sum_i \key{w_i} \good{r_i}$ and $\risk{\sigma_p^2} = \sum_i \sum_j \key{w_i} \key{w_j} \risk{\sigma_i} \risk{\sigma_j} \rho_{ij}$; in vector form:

the weights we choosereturnsrisk

$$\text{expected return: } \good{\mu_p} = \key{w^\top} \good{\mu}$$ $$\text{risk: } \risk{\sigma_p} = \sqrt{\key{w^\top} \risk{\Sigma}\, \key{w}}$$

The return is just the weighted average of the assets’ returns. The risk is not a weighted average: as the correlation post showed, the correlations inside $\risk{\Sigma}$ decide how much of the risk cancels out. That asymmetry is the whole reason optimisation is worth doing.

50,000 random portfolios

I drew 50,000 random long-only mixes (all weights between 0 and 1, adding up to 1) and computed the return and risk of each:

Three things to notice:

1. The cloud has a sharp upper-left edge. No mix gets above it. That edge is the efficient frontier: for each level of risk, the highest expected return available. Anything below the edge is beaten by a portfolio straight above it, with the same risk and more return.
2. The lowest-risk mix (blue) has 5.44% risk, less than bonds alone (6%). Adding a little equity and gold to bonds lowers risk, because they don’t move together.
3. The best mix beats every single asset on Sharpe ratio. The best of the 50,000 has a Sharpe of 0.301, higher than home equities alone (0.278), the best single asset.

(Random search only finds points near the true optimum. An optimiser would find the exact edge. With 50,000 draws in four assets, we’re close.)

What “optimal” means

Romero and Balch state the problem like this: choose the weights with the least risk that still meets a target return $r^*$, fully invested:

$$\begin{array}{ll} \underset{\key{w}}{\text{minimise}} \quad & \risk{\sigma_p}(\key{w}) \\[12pt] \text{subject to} \quad & \good{r_p}(\key{w}) \ge r^*,\; \textstyle\sum_i \key{w_i} = 1 \end{array}$$

Solve it once for every possible target $r^*$, and the answers trace out the efficient frontier: the lowest-risk portfolio for each achievable return. In practice you add limits too, such as no more than 5% in any one asset. Two special cases come up again and again:

Minimum variance: the mix with the least risk, whatever the return.

$$\begin{array}{ll} \underset{\key{w}}{\text{minimise}} \quad & \key{w^\top} \risk{\Sigma}\, \key{w} \\[12pt] \text{subject to} \quad & \textstyle\sum_i \key{w_i} = 1 \end{array}$$

If short selling is allowed (weights can be negative), this has an exact answer:

$$\key{w_{\text{min var}}} = \frac{\risk{\Sigma^{-1}} \mathbf{1}}{\mathbf{1}^\top \risk{\Sigma^{-1}} \mathbf{1}}$$

where $\mathbf{1}$ is a vector of ones. Note that it doesn’t use expected returns at all, which turns out to be an advantage (see below).

Maximum Sharpe ratio: the mix with the most excess return per unit of risk.

$$\begin{array}{ll} \underset{\key{w}}{\text{maximise}} \quad & \displaystyle\frac{\key{w^\top} \good{\mu} - r_f}{\sqrt{\key{w^\top} \risk{\Sigma}\, \key{w}}} \\[12pt] \text{subject to} \quad & \textstyle\sum_i \key{w_i} = 1,\; \key{w_i} \ge 0 \end{array}$$

Without the long-only limit, the answer points in the direction $\risk{\Sigma^{-1}}(\good{\mu} - r_f \mathbf{1})$, rescaled so the weights add up to 1. With limits like $\key{w_i} \ge 0$ there’s no neat formula, and you need a numerical optimiser.

The capital market line

The dashed line on the chart starts at the risk-free rate (3%, zero risk) and just touches the cloud at the best-Sharpe portfolio. Its slope is that Sharpe ratio:

$$\good{E[R]} = r_f + \text{Sharpe}_{\text{best}} \times \risk{\sigma}$$

Every point on the line is a mix of the best-Sharpe portfolio and cash. Want less risk? Hold some cash. More? Borrow (if you can). The line lies above the frontier everywhere except where it touches, so in theory everyone should hold the same risky mix, and only change how much cash they hold beside it. That’s a surprising and elegant result, and it’s the foundation of index investing.

Why real optimisers disappoint

The maths is exact. The inputs are not. Expected returns are very hard to estimate: a few years of data tell you almost nothing reliable about a stock’s average return. And the optimiser takes your guesses completely literally.

I changed one input: global equities expected to return 8.5% instead of 7.5%. Everything else the same.

A one-point change in a number nobody can actually forecast to one point moved a fifth of the portfolio. Optimisers concentrate money in whatever assets have the most optimistic estimates, which are often the ones with the biggest estimation errors. Richard Michaud called mean-variance optimisers “error maximizers”.

What practitioners do about it:

• Prefer minimum variance, which doesn’t need expected returns at all.
• Add limits: no asset above some maximum, or close to a benchmark.
• Shrink estimates towards something sensible, like equal returns, before optimising.
• Check stability: if a small change to the inputs flips the answer, don’t trust the answer.

The code

This only scores random portfolios. It doesn’t search for the best one.

import numpy as np

mu = np.array([0.08, 0.075, 0.035, 0.04])       # expected yearly returns
sd = np.array([0.18, 0.20, 0.06, 0.15])         # yearly standard deviations
corr = np.array([[1.00, 0.80, -0.10, 0.05],
                 [0.80, 1.00, -0.05, 0.10],
                 [-0.10, -0.05, 1.00, 0.20],
                 [0.05, 0.10, 0.20, 1.00]])
cov = np.outer(sd, sd) * corr                   # covariance matrix

W = np.random.default_rng(17).dirichlet(np.ones(4), 50_000)   # random long-only weights
ret = W @ mu
risk = np.sqrt(np.einsum("ij,jk,ik->i", W, cov, W))           # sqrt(w' cov w) for each row
sharpe = (ret - 0.03) / risk
W[sharpe.argmax()]                              # best mix among the random ones

Summary

That completes the Finance stats series: probability, expected value, mean and standard deviation, covariance and correlation, the risk-adjusted ratios, and portfolio optimisation.

References

• Philip J. Romero and Tucker Balch, What Hedge Funds Really Do: An Introduction to Portfolio Management (Business Expert Press, 2014), chapter 10: modern portfolio theory, the efficient frontier and the optimisation problem
• Harry Markowitz, “Portfolio Selection”, The Journal of Finance 7(1), 77–91, 1952
• Richard O. Michaud, “The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal?”, Financial Analysts Journal 45(1), 31–42, 1989