When a Monte Carlo risk estimate looks noisy, “just double the number of scenarios” sounds like an easy fix. But what did those extra 756 simulated observations actually add? I wanted to measure the difference between getting a more repeatable number and preserving the losses that matter.

I compared six scenario generators on the same historical sample of 12 ETFs, using 50 paired random seeds. Doubling each simulated series from 756 to 1,512 observations reduced the variation of some estimates. Yet Gaussian scenarios still had a downside co-crash rate of 0.3180, against 0.4211 in the fixed training history.

Think of “co-crash” as a simple question: when SPY has a really bad day, how many other ETFs are having a really bad day too? Each ETF’s “bad day” threshold is its own bottom 5% return from the training sample; I do not move those thresholds to flatter the simulation.

There are three different questions hiding behind the headline: Does another batch make the estimate more stable? Does the generator preserve same-day joint losses and consecutive bad days? Does a risk threshold hold up out of sample? The first two are evaluated on a fixed history below. The third is a separate, narrower one-day VaR exercise—not an OOS contest for the six generators.

Twelve ETFs and one fixed 756-day training window

The prices come from Boris Marjanovic’s Huge Stock Market Dataset. Its description marks the data CC0 and says prices account for dividends and splits. I picked 12 ETFs covering equities, government bonds, credit, gold, commodities, the dollar and real estate. Looking only at SPY would miss much of the cross-asset behavior I wanted to inspect; the full ticker list is at the end.

I intersected the dates of their adjusted Close series and did not fill missing days. In this Kaggle snapshot, the HYG file starts on 2007-04-11, so the common window spans 2,668 price dates from 2007-04-11 through 2017-11-10. I converted prices to daily simple returns and took the equally weighted return of all 12 ETFs each day. That portfolio definition never changes.

The scenario experiment trains on the first 756 daily returns, roughly three trading years that include the 2008 shock. Holding that history fixed puts all six generators in front of the same evidence. I also fixed Student-t degrees of freedom at five; I did not tune it after seeing the out-of-sample results.

Method	How the return scenarios are generated
Gaussian	Multivariate normal draws from training means and covariance
Student-t df=5	Multivariate t draws with fixed df=5 and training-matched covariance
IID Bootstrap	Sample whole daily ETF-return rows independently, with replacement
Moving-block 5 / 20 / 60	Sample consecutive stretches of 5, 20 or 60 return days

A comparison of two different random draws could easily confuse luck with a sample-size effect. So, for each generator, I used the same 50 seeds, 1000–1049: first draw a series of 1,512 observations, then reuse its first 756 as the smaller sample. In plain terms, I am reshuffling the same 756 days of historical information; generating another 756 simulated days does not create another 756 days of market history.

Student-t hides another trap. Calling a distribution “fat-tailed” is not enough if you accidentally inflate its variance as well. My implementation uses a Cholesky factor of the training covariance to produce correlated normal draws, then multiplies by √(3/χ²₅), where χ²₅ is chi-squared with five degrees of freedom. Equivalently, the multivariate t scale matrix is 3/5 of the training covariance. Omit that factor and generated covariance grows to 5/3 of the intended level.

What doubling scenarios actually stabilized

Start with one-day ES(5%): the average return among the worst 5% of observations. I computed it independently for every seed, then compared the across-seed standard deviation of ES at N=1,512 with N=756. This measures sampling variability, not how closely the model knows an unknown future market.

Generator	Across-seed ES standard deviation ratio: 1,512 / 756
Gaussian	0.718
Student-t df=5	0.671
IID Bootstrap	0.746
Moving-block 5	0.768
Moving-block 20	0.681
Moving-block 60	0.734

Every ratio is below one (0.671–0.768). In these 50 paired runs, the ES numbers clustered more tightly after the simulated sample doubled. The familiar 1/√2≈0.707 is a useful sampling-noise intuition, though a tail statistic such as ES need not follow it exactly.

The co-crash calculation tells a different story. On every simulated day when SPY breaches its fixed training-period 5% threshold, I count what fraction of the other 11 ETFs also cross their own fixed thresholds, then average over those SPY-triggered days. Gaussian’s mean co-crash error relative to the training benchmark barely moves: around −0.1036 at N=756 versus −0.1031 at N=1,512.

Running more draws made the estimate steadier. It did not make Gaussian suddenly reproduce the joint bad days in the training data.

IID Bootstrap remembers which assets fell together, not when bad days cluster

Matching a one-day co-crash statistic is not the same as reproducing several bad days in a row. All the metrics below are calculated from the equally weighted 12-ETF daily portfolio return, not by calculating each ETF separately and averaging statistics. The lag-one autocorrelation of absolute returns, ACF(1), gives a rough measure of clustered volatility. Sixty-day ES and drawdown look at longer return paths.

There is a compounding pitfall here: a 60-day loss cannot be recovered by simply adding 60 daily percentages. I slide a 60-day window forward one day at a time along the generated return sequence and calculate the compound window return as ∏(1+rₜ)-1. Sixty-day ES is the mean of the worst 5% of those window returns. For maximum drawdown, I calculate the drawdown along each window’s compounded wealth path and then take the 5% quantile across windows.

These are 50-seed means at N=1,512, compared with the same fixed training history:

Scenario source	Downside co-crash	Absolute-return ACF(1)	60-day ES(5%)	5% quantile of 60-day max drawdown
Training reference	0.4211	0.3000	-21.01%	-25.45%
Gaussian	0.3180	-0.0011	-15.77%	-16.53%
Student-t df=5	0.3549	-0.0005	-15.97%	-17.00%
IID Bootstrap	0.4265	0.0059	-15.15%	-16.26%
Block 5	0.4209	0.2408	-14.45%	-15.75%
Block 20	0.4227	0.2842	-16.54%	-18.56%
Block 60	0.4113	0.2684	-17.49%	-20.48%

You do not have to read every cell to see the problem. IID Bootstrap samples an entire daily row of ETF returns, preserving which assets fell on the same day. Its co-crash value, 0.4265, is close to the training reference 0.4211. But the days themselves are sampled independently: its ACF(1) is 0.0059, versus 0.3000 in the training history.

It remembers which assets can fall together, while nearly forgetting whether big moves happen on consecutive days.

Block 20 brings ACF(1) up to 0.2842, close to that historical reference. Block 60 pushes 60-day ES from IID’s −15.15% to −17.49%, yet the training reference is −21.01%. Longer blocks are not automatically better: Block 5 actually has a shallower 60-day ES than IID in this sample.

Why use overlapping 60-day windows? Without overlap, the 756-day training history gives me only about 12 separate 60-day stretches; the 1,512-day generated path gives about 25. I would not put much weight on a bottom-5% estimate from barely a dozen independent blocks. Sliding daily yields 697 training windows and 1,453 generated windows, but they overlap and must not be counted as independent risk events. The training period includes 2008, so −21.01% is a historical reference for this particular sample, not an exact target every generator must hit.

Moving-block Bootstrap: block seams and sample-edge weights are different effects

When the Block ACF rose, my first explanation was straightforward: it preserves the time order. Looking at the actual sampler made me separate two mechanisms I had lumped together.

First are the seams between sampled blocks. Consecutive days within a block are real historical neighbors; the next block can jump to a distant date. With block length L, about 1/L of consecutive pairs cross a seam. Under a simplified independent-block approximation, lag-one dependence gets damped on the order of 1−1/L. Multiplying the reference ACF of 0.3000 by that heuristic gives about 0.240 for L=5 and 0.285 for L=20—close to the measured 0.2408 and 0.2842. Block 60 does not fit that simplistic scaling, which is exactly why the rule of thumb is not a substitute for data.

The second mechanism is unequal selection of dates near the training sample’s ends. Standard non-circular Moving-block sampling allows block starting positions from 0 to 756−L. Dates near either edge have fewer chances to appear than interior dates; their selection weights can differ by as much as a factor of L. To see what this actually changes here, I reran the matched 50-seed comparison with Circular Block sampling, which allows a block to wrap around the historical sample’s end.

Block length	ACF(1): standard / circular	60-day ES: standard / circular
5	0.2408 / 0.2411	-14.45% / -14.35%
20	0.2842 / 0.2884	-16.54% / -16.37%
60	0.2684 / 0.2792	-17.49% / -17.23%

The differences are small. For Block 20, ACF goes from 0.2842 to 0.2884; 60-day ES from −16.54% to −16.37%. Edge weights differ mechanically, but in this data they do not explain most of the gap between Block and IID. Circular sampling also invents a connection between the last and first training day. I would not call it better just because it fixes one edge effect.

That extra check stopped me from attributing every improvement in Block to a single cause. For the method’s background, see Circular Block Resampling.

One-day VaR: three single-period methods fail calibration over the full OOS window

Now I am asking a narrower question: when the next real trading day arrives, is the predicted tail threshold calibrated? This is not an OOS ranking of the six scenario generators above. It compares just three one-day VaR methods—the empirical historical quantile, Gaussian and Student-t df=5. For each target day I refit the threshold on the preceding rolling 756 actual portfolio returns, not the fixed initial training window used for scenario simulation.

The t5 threshold still matches training variance. A VaR “breach” means the next realized portfolio return falls below the model’s predicted left-tail 5% cutoff. The rolling evaluation has 1,911 distinct one-day targets:

One-day 95% VaR method	Breaches	Breach rate	Kupiec p-value	Christoffersen independence p-value
Historical empirical	52	2.72%	6.16e-7	0.0145
Gaussian	51	2.67%	3.16e-7	0.0122
Student-t df=5	62	3.24%	1.75e-4	0.0171

At a nominal 5%, 1,911 × 0.05 = 95.55 breaches would be expected on average. All three methods produced only 51–62. Fewer breaches do not mean the models passed calibration. Kupiec tests unconditional breach frequency; Christoffersen tests independence. Both sets of p-values here are small.

That finding belongs to the whole 1,911-day period. Early rolling windows still include the 2008 crisis, which gradually leaves the training window later on. A regime-split test might tell another story, but I did not run that separate split. Consecutive double breaches number only 5 / 5 / 6 for the three methods, another useful fact when interpreting the independence test.

I deliberately did not force Block 60’s 60-day ES into this table. A multi-day OOS test needs a properly defined 60-day prediction target, and sliding 60-day targets overlap heavily. I have not finished a defensible backtesting design for that dependence. Copying one-day Kupiec/Christoffersen tests over would only create a false sense of completeness, so this section reports one-day VaR.

One more counterintuitive detail: “Student-t has fat tails” does not imply a more conservative VaR threshold at every confidence level. With df=5 and matched variance, its standardized left 5% quantile is about −1.56085, closer to zero than the Gaussian −1.64485. The distribution label alone does not tell you which threshold is stricter.

Where I would spend the next simulation budget

The next time somebody says, “These risk numbers are noisy, so double the scenarios,” I will ask three questions before buying more compute: Is the noise just random sampling? Did the generator preserve joint losses and strings of bad days? Has anyone checked calibration at the actual prediction horizon? Doubling scenarios mainly answers the first.

If I had to pick an online risk fallback between Block 20, Block 60 and t5, I would not crown a winner from this fixed-history comparison. Block 20’s ACF resembles one piece of the past; that is not evidence of future coverage. Student-t’s heavier tails do not guarantee a stricter 95% VaR. Choose the risk horizon first, then test it out of sample.

The two findings I want to remember are that Gaussian’s co-crash gap survived the doubling, and that the extra Circular Block run forced me to narrow my initial explanation. When I see a vendor advertising “a million risk scenarios,” the question I now ask is: which real observations are being reused, and which risk structures survived?

For a smaller hands-on starting point, see my Python example of how NaN changes the CVaR tail-sample denominator in vectorbt. Understanding how a single missing value alters a tail average makes the overlapping windows and risk statistics here less daunting.

---

Data source and fixed settings: Kaggle Huge Stock Market Dataset v3; SPY, IWM, EFA, EEM, TLT, IEF, LQD, HYG, GLD, DBC, UUP and VNQ. The intersection of adjusted-close dates is 2007-04-11 to 2017-11-10 (2,668 price rows), with no missing-date imputation. Fixed CSV SHA-256: 0a20c095f0d9400fbf7d30e17a0c664716f147c460b03bf5439cd980ef8bf38a. The scenario study uses 756 training return days and 50 paired seeds; the separate rolling one-day VaR study has 1,911 targets. This is a historical risk-modeling methods comparison, not investment advice.