Odds-On Imperfection: Monte Carlo Simulation (WSJ article)

Monte Carlo simulates random events. The housing mess is the result of a bunch of crooks selling homes to people who couldn't afford them combined with "insurance" that had no reserve.

The outcome, in hindsite, is fully deterministic.

Frank

And all that was necessary was a random trigger, even an unrelated one, in this case, high oil and food prices, was enough to make the castle of cards crumble down. Talk about a "hind" sight...

My comment is that it is a logical fallacy to apply the term "standard deviation" to market behavior.

The argument is much more complicated than I can make in this small space, but if you're intersted, you can learn more than you ever wanted to know about it here:

Agree 100%. Or as I like to put it, any list of numbers has a standard deviation, but that doesn't mean it's meaningful.

Coincidentally I was just reading a white paper by one of the MC vendors, that speaks of a mean absolute error of 11.5% in predicting returns in one of the stock indices (the Nasdaq 100 - a whole issue in itself, but that's another post). In the world of science and engineering the methodology would then be discarded, as unworkable - imagine you were trying to use MCS to predict the failure rate for a part on an aircraft to determine manufacturing processes and came up with an error like that. In finance it becomes a "best practice" in some circles because while it's so far out of bounds as to be arguably useless, it's slightly better than using the more-invalid "project the past few years forward to the next year."

That said I think the technique is helpful just to illustrate variable returns - that visual, showing all those trajectories where sometimes it goes to $0, sometimes sky-high, and mostly in the middle. But god forbid you then pretend that the probabilities it spits out have meaning for decision-making. Even a layperson should be able to see the problem with running "tens of thousands or hundreds of thousands of scenarios, which help gauge extreme events at the tail end of the distribution." You could run it a billion times, you're still limited by the lack of data and (more fundamentally) the fact that "hijacked jet flies into building" and "laws change to facilitate bad lending" and "bad guy with bad mustache takes over Europe" and "for no apparent reason stocks drop" can't be described using probability theory.

-Tad

MCS is a methodology used in engineering all the time, with great effectiveness, just not in the way you describe. Google. You might remember that engineering design is not generally subject to psychological whim or fraud, so in fact MCS has much more application to it. Remember the meaning of "factor of safety," too. To the layperson, it must seem a little astounding that a factor of safety in determining say the required dimensions of a building, bridge, automotive, nuclear reactor etc. structural component can easily run on the order of 5 or 10 or more.

ISTM that these rare events most certainly are reflected in MCS. No one says MCS is perfect but the fact is it does note the historical rarity of the Crash of '29, Black Monday '87, and so on. General counsel to increase allocation to high grade bonds as one ages is, after all, based simply on the fact that we have seen crashes in the past; we have no idea whether they will occur in the future; but they could.

Without the impressions that MCS and related notions leave, what is the basis for allocating? Guessing?

I do not want to get into a debate with anyone on this. More that I think MCS and other models do incorporate history, and ISTM history does say something about economies and can help a person understand what it means to own stock. What do you say to someone when s/he says, "Why buy stock"?

IMO the above statement is oxymoronic. Outliers by definition are not highlighted but they must be acknowledged by those using MCS.

The WSJ author seems to be saying, oh so because they are happening more often, they have a higher probability of happening again in the future. Does she realize she is making an inference based on statistics?

OK. I can't resist. The way I read this is that you are saying "Past performance IS indicative of future results". Right? -- Doug

Monte Carlo simulation is a very simple process. It is a process of making various random selections of input variables, _distributed according to the modeler's assumptions_, and then average values of outcomes are calculated by averaging results of very many tries.

It is especially helpful in cases of valuing financial derivative products, where no closed form of partial differential type solutions exist.

The big disadvantage of Monte Carlo simulation is that it is usually very expensive computationally.

Monte Carlo simulation is not at all a magic solution whereby a closed minded modeler can suddenly "think out of the box". For instance, if the modeler's assumption calls for normally distributed inputs, the simulation will dutifully comply with normal distribution of those inputs and will not in any way indicate what would happen if inputs were not normally distributed, or if correlations or standard deviations were different from those assumed.

I think you misunderstood Tad's point. MCS is used in engineering all the time, and effectively at that. But if the model you are running MCS over churns out the errors at the rate Tad suggests, you've got the wrong model.

MCS itself is a technique (giving me difficulty parsing what "it" is in your last sentence above) and thus does not take anything into account in and of itself. MCS must be applied to a model that may or may not take these things into account. A common criticism of financial MCS tools is that they typically use Gaussian probability distributions, but the markets seem to behave within fat-tailed distributions. Another criticism I've seen is similar to B's, market volatility changes dramatically over time, but these tools almost always use constant volatility.

Here, I agree with you, and I think you agreed with Tad. Financial MCS tools are instructive, but it seems foolish to use them to develop exact asset allocations as they typically exist today.

-Will

william dot trice at ngc dot com

IMO, it depends. Posting for the archives and laypeople, not to get into what would probably be a useless row over this.

It depends. It used to be very expensive computationally. With today's computer power, it's all relative.

AFAICT Tad's post was saying that someone might come up with a failure rate of an airplane part of 25% + or - some 11%. This is useful information, assuming the sample size is large enough.

I would amend this to: "There is no tool that will model the future returns of different allocation categories exactly. In fact, the consumer should not delude him/herself into thinking that one tool is necessarily superior to any other. The consumer should strive to understand and question as appropriate the assumptions the tool is using."

AFAIC, this is a lead-in to an existential discussion. The fact is we always use some information from the past to make future decisions, whether they are financial or otherwise. So a sound bite will not do. Why buy stock? I do not think there is a rational answer that does not rely in part on the past.

That's why I see it best used (in finance) for just an illustration of "results can vary...a lot!" And in a sense, your inputs don't really matter too much, as long as you show an illustration with a big range of results.

But I don't believe that it's possible to model, in a meaningful way, the distribution of investment returns (in particular, factoring in their interactions). Just adding in fat tails or skewing the shape of the curve isn't enough. The derivatives example is a perfect illustration of how bad the models can be. It violates basic principles of common sense, but apparently "home prices have not dropped" was an input used in some models for valuing mortgages and derivatives based on them.

-Tad

I suppose it depends on how broadly you define "relying on the past" but one very good reason has little to do with past returns, and just looks at capitalism and how it works. The reason to own stocks is that you believe businesses exist to earn profits, and that the owners of those businesses (ie shareholders) get the benefit of that. If you don't believe that, I don't know why you'd invest in stocks.

But you don't need a stock market with a history of return figures to believe this. Take a guy off the street with $10,000 and offer him

In what I think is an absurd illustration of MCS's flaws, I recall a discussion about what inputs to use for "private equity" - i.e. that small business I'm talking about above. As if there is such a thing. But somewhere out there is somebody typing figures into MCS software and spitting out just that kind of analysis.

-Tad

That's all about how it /has/ worked in the past, though.

Zero percent info on past price history is an impossibility, because we all have been raised of some awareness of costs.

I agree with the rest of your post. I think way too many buy stocks "just because they go up over the long term," without understanding the principles (of which your post is a decent Q&D summary AFAIC) behind why they go up. Though one caveat: the MCS example you gave is an abuse. Yet I think there is some agreement here that uses of MCS in finances are not always an abuse.

I keep thinking of the Trinity study and how it spews out a low probability of running out of money if one has X blah blah in bonds, Y yada in stocks, for a certain amount of years. If the unwashed focus on the high probability and do not admit the risk that the lower probability certainly could still happen, then sure Trinity etc. are useless. Right now people are all upset because the "less likely" happened. Well that's partly their fault for not understanding say statistics or letting greed blind them.

It could be useful to estimate the value, or the partial derivatives of value, of a complex financial instrument. For example, a complicated life insurance policy would be a good example.

I agree. Seeking precision beyong vague statements like "markets fluctuate, sometimes wildly", may lead to very wrong thinking.

I believe that Moody's, or another bond rater, used some software package to rate various mortgage derivatives. An input to their risk model was "expected HPA" (housing price appreciation). The program would not accept a negative number, it woulod report a syntax error.

Here's a related thought: I wouldn't give up on the Trinity Study just yet. My experience is that assumptions about the future we are using today (during a depressing economic contraction) are just as likely to prove fallacious as those we made back in the euphoric stage of the previous economic expansion. The truth is likely somewhat in the middle (50-50 stocks/bonds, not far from what the Trinity folks suggested).

-HW "Skip" Weldon Columbia, SC

Big snip

Milevsky ( papers published at

Milevsky's papers are worth reading even if they are skewed towards the Canadian experience. For those so inclined some of his papers include all the math.

You can't take a bunch of 1 sigma (standard deviation) information and grind it together and then believe that you have a 6 sigma result

The key point is that "financial data" does not have a Gaussian (normal) distribution. There are a lot of black swans out there. Finally "past results do not ....."

Aside, because maybe I was not clear: I also think the Trinity Study remains helpful. I tend to agree with Skip's comments on this. Concur with Avrum's comments, too.