# Double the Monte Carlo Scenarios—Does Tail Risk Get More Real? A 12-ETF Experiment > Canonical: https://vxi.me/en/posts/monte-carlo-scenarios-tail-risk-12-etf/ > Locale: en-us > Published: 2026-10-23 > Updated: 2026-10-23 > Markdown: https://vxi.me/en/posts/monte-carlo-scenarios-tail-risk-12-etf.md
12-ETF comparison: six ES variability ratios 0.671–0.768; Gaussian co-crash 0.3180 versus training reference 0.4211.
One fixed training sample: more scenarios reduce sampling noise, not Gaussian’s co-crash gap. The separate OOS tests only three one-day VaR methods.
When a Monte Carlo risk estimate looks noisy, “just double the number of scenarios” sounds like an easy fix. But what did those extra 756 simulated observations actually add? I wanted to measure the difference between getting a more repeatable number and preserving the losses that matter. I compared **six scenario generators on the same historical sample of 12 ETFs**, using 50 paired random seeds. Doubling each simulated series from 756 to 1,512 observations reduced the variation of some estimates. Yet Gaussian scenarios still had a downside co-crash rate of **0.3180**, against **0.4211** in the fixed training history. Think of “co-crash” as a simple question: *when SPY has a really bad day, how many other ETFs are having a really bad day too?* Each ETF’s “bad day” threshold is its own bottom 5% return from the **training sample**; I do not move those thresholds to flatter the simulation. There are three different questions hiding behind the headline: **Does another batch make the estimate more stable? Does the generator preserve same-day joint losses and consecutive bad days? Does a risk threshold hold up out of sample?** The first two are evaluated on a fixed history below. The third is a separate, narrower one-day VaR exercise—not an OOS contest for the six generators. ## Twelve ETFs and one fixed 756-day training window The prices come from Boris Marjanovic’s [Huge Stock Market Dataset](https://www.kaggle.com/datasets/borismarjanovic/price-volume-data-for-all-us-stocks-etfs). Its description marks the data CC0 and says prices account for dividends and splits. I picked 12 ETFs covering equities, government bonds, credit, gold, commodities, the dollar and real estate. Looking only at SPY would miss much of the cross-asset behavior I wanted to inspect; the full ticker list is at the end. I intersected the dates of their adjusted Close series and did not fill missing days. In this Kaggle snapshot, the HYG file starts on 2007-04-11, so the common window spans **2,668 price dates from 2007-04-11 through 2017-11-10**. I converted prices to daily simple returns and took the equally weighted return of all 12 ETFs each day. That portfolio definition never changes. The scenario experiment trains on the **first 756 daily returns**, roughly three trading years that include the 2008 shock. Holding that history fixed puts all six generators in front of the same evidence. I also fixed Student-t degrees of freedom at five; I did not tune it after seeing the out-of-sample results. | Method | How the return scenarios are generated | | --- | --- | | Gaussian | Multivariate normal draws from training means and covariance | | Student-t df=5 | Multivariate t draws with fixed df=5 and training-matched covariance | | IID Bootstrap | Sample whole daily ETF-return rows independently, with replacement | | Moving-block 5 / 20 / 60 | Sample consecutive stretches of 5, 20 or 60 return days | A comparison of two different random draws could easily confuse luck with a sample-size effect. So, for each generator, I used the **same 50 seeds, 1000–1049**: first draw a series of 1,512 observations, then reuse its *first 756* as the smaller sample. In plain terms, I am reshuffling the same 756 days of historical information; generating another 756 simulated days does not create another 756 days of market history. Student-t hides another trap. Calling a distribution “fat-tailed” is not enough if you accidentally inflate its variance as well. My implementation uses a Cholesky factor of the training covariance to produce correlated normal draws, then multiplies by `√(3/χ²₅)`, where χ²₅ is chi-squared with five degrees of freedom. Equivalently, the multivariate t scale matrix is **3/5 of the training covariance**. Omit that factor and generated covariance grows to **5/3** of the intended level. ## What doubling scenarios actually stabilized Start with one-day ES(5%): the average return among the worst 5% of observations. I computed it independently for every seed, then compared the *across-seed standard deviation* of ES at N=1,512 with N=756. This measures sampling variability, not how closely the model knows an unknown future market. | Generator | Across-seed ES standard deviation ratio: 1,512 / 756 | | --- | ---: | | Gaussian | 0.718 | | Student-t df=5 | 0.671 | | IID Bootstrap | 0.746 | | Moving-block 5 | 0.768 | | Moving-block 20 | 0.681 | | Moving-block 60 | 0.734 | Every ratio is below one (0.671–0.768). In these 50 paired runs, the ES numbers clustered more tightly after the simulated sample doubled. The familiar `1/√2≈0.707` is a useful sampling-noise intuition, though a tail statistic such as ES need not follow it exactly. The co-crash calculation tells a different story. On every simulated day when SPY breaches its **fixed training-period 5% threshold**, I count what fraction of the other 11 ETFs also cross their own fixed thresholds, then average over those SPY-triggered days. Gaussian’s mean co-crash error relative to the training benchmark barely moves: around **−0.1036 at N=756** versus **−0.1031 at N=1,512**. Running more draws made the estimate steadier. It did not make Gaussian suddenly reproduce the joint bad days in the training data. ## IID Bootstrap remembers which assets fell together, not when bad days cluster Matching a one-day co-crash statistic is not the same as reproducing several bad days in a row. All the metrics below are calculated from the **equally weighted 12-ETF daily portfolio return**, not by calculating each ETF separately and averaging statistics. The lag-one autocorrelation of absolute returns, **ACF(1)**, gives a rough measure of clustered volatility. Sixty-day ES and drawdown look at longer return paths. There is a compounding pitfall here: a 60-day loss cannot be recovered by simply adding 60 daily percentages. I slide a 60-day window forward one day at a time along the generated return sequence and calculate the compound window return as `∏(1+rₜ)-1`. Sixty-day ES is the mean of the worst 5% of those window returns. For maximum drawdown, I calculate the drawdown along each window’s compounded wealth path and then take the 5% quantile across windows. These are **50-seed means at N=1,512**, compared with the same fixed training history: | Scenario source | Downside co-crash | Absolute-return ACF(1) | 60-day ES(5%) | 5% quantile of 60-day max drawdown | | --- | ---: | ---: | ---: | ---: | | Training reference | 0.4211 | 0.3000 | -21.01% | -25.45% | | Gaussian | 0.3180 | -0.0011 | -15.77% | -16.53% | | Student-t df=5 | 0.3549 | -0.0005 | -15.97% | -17.00% | | IID Bootstrap | 0.4265 | 0.0059 | -15.15% | -16.26% | | Block 5 | 0.4209 | 0.2408 | -14.45% | -15.75% | | Block 20 | 0.4227 | 0.2842 | -16.54% | -18.56% | | Block 60 | 0.4113 | 0.2684 | -17.49% | -20.48% | You do not have to read every cell to see the problem. IID Bootstrap samples an *entire daily row* of ETF returns, preserving which assets fell on the same day. Its co-crash value, **0.4265**, is close to the training reference **0.4211**. But the days themselves are sampled independently: its ACF(1) is **0.0059**, versus **0.3000** in the training history. It remembers **which assets can fall together**, while nearly forgetting **whether big moves happen on consecutive days**. Block 20 brings ACF(1) up to **0.2842**, close to that historical reference. Block 60 pushes 60-day ES from IID’s **−15.15%** to **−17.49%**, yet the training reference is **−21.01%**. Longer blocks are not automatically better: Block 5 actually has a shallower 60-day ES than IID in this sample. Why use overlapping 60-day windows? Without overlap, the 756-day training history gives me only about **12** separate 60-day stretches; the 1,512-day generated path gives about **25**. I would not put much weight on a bottom-5% estimate from barely a dozen independent blocks. Sliding daily yields **697** training windows and **1,453** generated windows, but they overlap and must not be counted as independent risk events. The training period includes 2008, so **−21.01% is a historical reference for this particular sample, not an exact target every generator must hit**. ## Moving-block Bootstrap: block seams and sample-edge weights are different effects When the Block ACF rose, my first explanation was straightforward: it preserves the time order. Looking at the actual sampler made me separate two mechanisms I had lumped together. First are the **seams between sampled blocks**. Consecutive days within a block are real historical neighbors; the next block can jump to a distant date. With block length L, about `1/L` of consecutive pairs cross a seam. Under a simplified independent-block approximation, lag-one dependence gets damped on the order of `1−1/L`. Multiplying the reference ACF of 0.3000 by that heuristic gives about **0.240 for L=5** and **0.285 for L=20**—close to the measured **0.2408** and **0.2842**. Block 60 does not fit that simplistic scaling, which is exactly why the rule of thumb is not a substitute for data. The second mechanism is **unequal selection of dates near the training sample’s ends**. Standard non-circular Moving-block sampling allows block starting positions from `0` to `756−L`. Dates near either edge have fewer chances to appear than interior dates; their selection weights can differ by as much as a factor of L. To see what this actually changes here, I reran the matched 50-seed comparison with Circular Block sampling, which allows a block to wrap around the historical sample’s end. | Block length | ACF(1): standard / circular | 60-day ES: standard / circular | | --- | --- | --- | | 5 | 0.2408 / 0.2411 | -14.45% / -14.35% | | 20 | 0.2842 / 0.2884 | -16.54% / -16.37% | | 60 | 0.2684 / 0.2792 | -17.49% / -17.23% | The differences are small. For Block 20, ACF goes from **0.2842 to 0.2884**; 60-day ES from **−16.54% to −16.37%**. Edge weights differ mechanically, but **in this data they do not explain most of the gap between Block and IID**. Circular sampling also invents a connection between the last and first training day. I would not call it better just because it fixes one edge effect. That extra check stopped me from attributing every improvement in Block to a single cause. For the method’s background, see [Circular Block Resampling](https://statistics.stanford.edu/technical-reports/circular-block-resampling-procedure-stationary-data). ## One-day VaR: three single-period methods fail calibration over the full OOS window Now I am asking a narrower question: **when the next real trading day arrives, is the predicted tail threshold calibrated?** This is *not* an OOS ranking of the six scenario generators above. It compares just three **one-day VaR methods**—the empirical historical quantile, Gaussian and Student-t df=5. For each target day I refit the threshold on the **preceding rolling 756 actual portfolio returns**, not the fixed initial training window used for scenario simulation. The t5 threshold still matches training variance. A VaR “breach” means the next realized portfolio return falls below the model’s predicted left-tail 5% cutoff. The rolling evaluation has **1,911 distinct one-day targets**: | One-day 95% VaR method | Breaches | Breach rate | Kupiec p-value | Christoffersen independence p-value | | --- | ---: | ---: | ---: | ---: | | Historical empirical | 52 | 2.72% | 6.16e-7 | 0.0145 | | Gaussian | 51 | 2.67% | 3.16e-7 | 0.0122 | | Student-t df=5 | 62 | 3.24% | 1.75e-4 | 0.0171 | At a nominal 5%, **1,911 × 0.05 = 95.55** breaches would be expected on average. All three methods produced only **51–62**. Fewer breaches do **not** mean the models passed calibration. Kupiec tests unconditional breach frequency; Christoffersen tests independence. Both sets of p-values here are small. That finding belongs to the **whole 1,911-day period**. Early rolling windows still include the 2008 crisis, which gradually leaves the training window later on. A regime-split test might tell another story, but I did not run that separate split. Consecutive double breaches number only **5 / 5 / 6** for the three methods, another useful fact when interpreting the independence test. I deliberately did not force Block 60’s 60-day ES into this table. A multi-day OOS test needs a properly defined 60-day prediction target, and sliding 60-day targets overlap heavily. I have not finished a defensible backtesting design for that dependence. Copying one-day Kupiec/Christoffersen tests over would only create a false sense of completeness, so this section reports one-day VaR. One more counterintuitive detail: “Student-t has fat tails” does not imply a more conservative VaR threshold at every confidence level. With df=5 and matched variance, its standardized **left 5% quantile is about −1.56085**, *closer to zero* than the Gaussian **−1.64485**. The distribution label alone does not tell you which threshold is stricter. ## Where I would spend the next simulation budget The next time somebody says, “These risk numbers are noisy, so double the scenarios,” I will ask three questions before buying more compute: **Is the noise just random sampling? Did the generator preserve joint losses and strings of bad days? Has anyone checked calibration at the actual prediction horizon?** Doubling scenarios mainly answers the first. If I had to pick an online risk fallback between Block 20, Block 60 and t5, **I would not crown a winner from this fixed-history comparison**. Block 20’s ACF resembles one piece of the past; that is not evidence of future coverage. Student-t’s heavier tails do not guarantee a stricter 95% VaR. Choose the risk horizon first, then test it out of sample. The two findings I want to remember are that Gaussian’s co-crash gap survived the doubling, and that the extra Circular Block run forced me to narrow my initial explanation. When I see a vendor advertising “a million risk scenarios,” the question I now ask is: **which real observations are being reused, and which risk structures survived?** For a smaller hands-on starting point, see my [Python example of how NaN changes the CVaR tail-sample denominator in vectorbt](/posts/vectorbt-nan-cvar-tail-sample-test-assertions/). Understanding how a single missing value alters a tail average makes the overlapping windows and risk statistics here less daunting. --- *Data source and fixed settings: Kaggle Huge Stock Market Dataset v3; SPY, IWM, EFA, EEM, TLT, IEF, LQD, HYG, GLD, DBC, UUP and VNQ. The intersection of adjusted-close dates is 2007-04-11 to 2017-11-10 (2,668 price rows), with no missing-date imputation. Fixed CSV SHA-256: `0a20c095f0d9400fbf7d30e17a0c664716f147c460b03bf5439cd980ef8bf38a`. The scenario study uses 756 training return days and 50 paired seeds; the separate rolling one-day VaR study has 1,911 targets. This is a historical risk-modeling methods comparison, not investment advice.*