# Order Book Backtest Fills Don't Match? Debugging NautilusTrader batch_deltas and F_LAST
> Canonical: https://vxi.me/en/posts/nautilustrader-orderbook-batch-deltas-f-last/
> Locale: en-us
> Published: 2026-10-18
> Updated: 2026-10-18
> Markdown: https://vxi.me/en/posts/nautilustrader-orderbook-batch-deltas-f-last.md
If you replay L2/L3 order-book data through NautilusTrader's `OrderBookDelta` and the fills or position counts change after an upgrade, don't immediately retune the strategy. Two different mechanisms deserve investigation: **`batch_deltas` determines whether matching can see an intermediate order-book state; and cutting off the start of an event can go undetected even if its last row still carries `F_LAST`.**
That second case is deceptively easy to miss. `F_LAST` identifies the *end* of a group; it does not prove that the first row of that group was present. A time window or `filter_expr` can remove an initial ADD while leaving the final DELETE marked `F_LAST`. The batcher may close what is now only a fragment. The [version-pinned backtesting documentation](https://github.com/nautechsystems/nautilus_trader/blob/55c0e82053142bce3a5cba4de4221d3929c2d3bc/docs/concepts/backtesting/apis-and-runs.md#L154-L187) explicitly warns about this failure to detect a truncated beginning.
I inspected the source and upstream tests for NautilusTrader `v2.0.0rc7` and independently reproduced the event-boundary behavior using Python's standard library. Use the configuration comparison below on your own Catalog; the appendix scripts illustrate the grouping mechanism with synthetic data rather than running the NautilusTrader engine.
## Before changing the strategy: a four-step reconciliation checklist
This is the order in which I would investigate a backtest that changed despite apparently using the same data Catalog.
1. **Freeze the baseline.** Record both framework versions, Catalog contents, time ranges, instrument precision, strategy version, and order/matching parameters. Do not regenerate the underlying historical data while investigating.
2. **Inspect event completeness.** For each `instrument_id`, check `F_LAST` and especially whether the first event in the selected window was cut in half. A valid closing marker says nothing about the missing head.
3. **Change one setting only.** Run the same strategy, dataset and matching assumptions with `batch_deltas=True` and `batch_deltas=False`. Locate the *first* fill or callback difference, then inspect its exact deltas.
4. **Decide on the baseline afterward.** Choose the replay granularity according to the data provider's event semantics, not according to which curve shows a higher return.
Adapt the [documented `BacktestDataConfig` pattern](https://github.com/nautechsystems/nautilus_trader/blob/55c0e82053142bce3a5cba4de4221d3929c2d3bc/docs/concepts/backtesting/apis-and-runs.md#L176-L187) in your existing backtest project. First configure `batch_deltas=True`, then change only that parameter to `False` for comparison; keep the real Catalog path and `instrument_id` identical between runs:
```python
data = BacktestDataConfig(
data_type=NautilusDataType.OrderBookDelta,
catalog_path="/data/catalog",
instrument_id=instrument_id,
batch_deltas=True, # rc7 默认：按完整事件回放
)
# 对照组保持其余配置不变，仅将 batch_deltas 改为 False
```
The [pinned Rust configuration](https://github.com/nautechsystems/nautilus_trader/blob/55c0e82053142bce3a5cba4de4221d3929c2d3bc/crates/backtest/src/config.rs#L838-L844) sets the rc7 default to `true`. The [Python configuration tests](https://github.com/nautechsystems/nautilus_trader/blob/55c0e82053142bce3a5cba4de4221d3929c2d3bc/crates/backtest/src/python/config.rs#L1080-L1119) cover default, enabled, and disabled values. Setting `False` is a diagnostic control, **not** a repair for an incomplete historical book.
## F_LAST can detect a missing ending, not necessarily a missing beginning
Suppose one logical market-data event adds an ask at 999.95 and then deletes it. Only the final DELETE carries `F_LAST`.
| Read window | Delta rows retained | What the event batcher can observe |
| --- | --- | --- |
| Complete | ADD 999.95 → DELETE 999.95 (`F_LAST`) | Two rows grouped into a complete event |
| **Tail truncated** | ADD 999.95 without `F_LAST` | Pending group remains; an end-of-input error can be raised |
| **Head truncated** | DELETE 999.95 (`F_LAST`) only | **The one-row fragment can still be closed** |
Why does the last row pass through the grouping stage? The [`BookDeltasBatcher` implementation](https://github.com/nautechsystems/nautilus_trader/blob/55c0e82053142bce3a5cba4de4221d3929c2d3bc/crates/backtest/src/node.rs#L744-L829) maintains pending deltas by instrument, emits `OrderBookDeltas` when a row carries `F_LAST`, and checks for *unclosed groups* at the end of the input. It does not have enough information to reconstruct an ADD row already removed by the query.
“Can close” here refers specifically to the **`F_LAST` grouping check**; an invalid DELETE or a missing snapshot might fail a different downstream check. The practical fix is to verify event boundaries in the original Catalog or data feed, widening the window or starting from a trusted snapshot as needed. Counting how many rows contain `F_LAST` is not an adequate integrity check.
## Why identical order-book deltas can produce different fills
The replay input may be complete and identical, yet the timing of matching still changes the outcome. Imagine a resting limit BUY at 1000.00 when the best ask is 1000.10. One event then adds an ask at 999.95 and removes it immediately; the DELETE carries `F_LAST`.
If the engine attempts matching after each individual delta, the buy can encounter the temporary 999.95 ask. If it applies the whole event first, that ask is gone by the time matching runs. This is a **visibility-of-intermediate-state** issue, not merely a batching performance optimization.
| Replay mode | Book state at the matching point | Synthetic toy fills |
| --- | --- | ---: |
| Individual deltas | Temporary 999.95 ask is visible | 1 |
| Entire `F_LAST` event | 999.95 ask has already been removed | 0 |
The [upstream Rust integration test](https://github.com/nautechsystems/nautilus_trader/blob/55c0e82053142bce3a5cba4de4221d3929c2d3bc/crates/backtest/tests/integration/backtest_node.rs#L2169-L2240) builds a comparable temporary-ask situation and declares `#[case::batched(true, 0)]` and `#[case::individual(false, 1)]` as expected `total_positions` values. That metric comes from upstream *test assertions*. The 1/0 in the table is an independently executed toy model's simulated fill count. Both clarify the same event-boundary mechanism, but they are not the same measurement.
The [backtesting documentation](https://github.com/nautechsystems/nautilus_trader/blob/55c0e82053142bce3a5cba4de4221d3929c2d3bc/docs/concepts/backtesting/apis-and-runs.md#L154-L168) also describes replaying the batch at the closing delta's `ts_init` and changes to `on_book_deltas` callbacks and streaming `chunk_size` accounting. To debug a discrepancy, examine the earliest divergent event and callback rather than comparing only final PnL.
The [v2.0.0rc6 configuration source](https://github.com/nautechsystems/nautilus_trader/blob/v2.0.0rc6/crates/backtest/src/config.rs) lacks the `batch_deltas` field. This establishes a configuration-interface change; it does not prove that every old replay path was literally equivalent to `False`.
## Reproducing the mechanism: full scripts and negative controls
I used Python 3.13.5's standard library to model a deliberately simple book and matching step. The individual replay yields `individual=1` and event-batched replay yields `batched=0`. I also checked missing trailing `F_LAST`, interleaved instruments, and separate events sharing one timestamp. The initial script passed **5/5 checks**; an expanded `unittest` suite passed **10/10**.
The most useful negative control is the case where **the beginning of an event is missing but the remaining fragment still closes**. A green result on that test proves the gap in `F_LAST` validation; it does not mean the damaged feed was recovered. Some checks overlap across the two suites, so fifteen passing assertions should not be advertised as fifteen independent market observations.
The complete runnable code and original output are preserved below. Save the first block as `book_batching_fixture.py` and run `python3 book_batching_fixture.py`. Place the second file, `edge_case_verification.py`, beside it and run `python3 -m unittest -v edge_case_verification`. Both rely only on the standard library and fictional prices: no queue-position logic, fees, historical feed or real exchange fills.
Original synthetic fixture — 5/5: full code and recorded output
```python
"""Self-authored toy model of F_LAST event boundaries, NOT NautilusTrader runtime."""
from dataclasses import dataclass
from decimal import Decimal
@dataclass(frozen=True)
class Delta:
instrument: str
action: str
price: Decimal
last: bool
ts_init: int
def d(instrument, action, price, last, timestamp):
return Delta(instrument, action, Decimal(price), last, timestamp)
def group_by_last(rows):
pending = {}
for row in rows:
event = pending.setdefault(row.instrument, [])
event.append(row)
if row.last:
yield tuple(event)
del pending[row.instrument]
if pending:
detail = ", ".join(sorted(pending))
raise ValueError(f"missing F_LAST for {detail}")
def toy_fills(rows, batch_deltas):
"""One pending BUY at 1000; match after each replay item, not each event row."""
asks = set()
buy_pending = True
fills = 0
replay_items = group_by_last(rows) if batch_deltas else ((r,) for r in rows)
for item in replay_items:
for row in item:
if row.action == "ADD":
asks.add(row.price)
elif row.action == "DELETE":
asks.discard(row.price)
if buy_pending and asks and min(asks) <= Decimal("1000.00"):
buy_pending = False
fills += 1
return fills
def run():
# The same synthetic event shape as the concept behind the pinned upstream test.
rows = [
d("SIM", "ADD", "1000.10", True, 1),
d("SIM", "ADD", "999.95", False, 2),
d("SIM", "DELETE", "999.95", True, 2),
]
groups = list(group_by_last(rows))
assert [len(g) for g in groups] == [1, 2]
print("PASS 1/5 event boundaries: 3 deltas -> groups [1, 2]")
individual, batched = toy_fills(rows, False), toy_fills(rows, True)
assert (individual, batched) == (1, 0)
print(f"PASS 2/5 toy fills: individual={individual}, batched={batched}")
try:
list(group_by_last([d("SIM", "ADD", "1.00", False, 3)]))
except ValueError as exc:
assert "missing F_LAST for SIM" in str(exc)
else:
raise AssertionError("unterminated event did not fail")
print("PASS 3/5 unterminated event rejected")
interleaved = [
d("A", "ADD", "1", False, 4),
d("B", "ADD", "2", True, 4),
d("A", "DELETE", "1", True, 5),
]
received = list(group_by_last(interleaved))
assert [[r.instrument for r in group] for group in received] == [["B"], ["A", "A"]]
print("PASS 4/5 interleaved instruments remain isolated")
same_time = [d("SIM", "ADD", "1", True, 6), d("SIM", "ADD", "2", True, 6)]
assert [len(g) for g in group_by_last(same_time)] == [1, 1]
print("PASS 5/5 equal-timestamp completed events stay separate")
print("SUMMARY: 5/5 PASS, self-authored stdlib model only; no upstream engine run")
if __name__ == "__main__":
run()
```
```text
PASS 1/5 event boundaries: 3 deltas -> groups [1, 2]
PASS 2/5 toy fills: individual=1, batched=0
PASS 3/5 unterminated event rejected
PASS 4/5 interleaved instruments remain isolated
PASS 5/5 equal-timestamp completed events stay separate
SUMMARY: 5/5 PASS, self-authored stdlib model only; no upstream engine run
```
Extended negative controls — 10/10: full code and recorded output
```python
"""Independent negative-control suite for the self-authored F_LAST toy model.
Not NautilusTrader tests and not an exchange/market backtest.
"""
import unittest
from book_batching_fixture import d, group_by_last, toy_fills
class OrderBookEventBoundaryControls(unittest.TestCase):
def test_01_trailing_incomplete_event_is_rejected(self):
rows = [d("SIM", "ADD", "999.95", False, 10)]
with self.assertRaisesRegex(ValueError, "missing F_LAST for SIM"):
list(group_by_last(rows))
def test_02_head_truncated_event_is_not_detected_by_f_last(self):
full_event = [
d("SIM", "ADD", "999.95", False, 10),
d("SIM", "DELETE", "999.95", True, 10),
]
complete = list(group_by_last(full_event))
filtered = list(group_by_last(full_event[1:]))
self.assertEqual([len(g) for g in complete], [2])
# This is a NEGATIVE result: F_LAST alone cannot validate the missing head.
self.assertEqual([len(g) for g in filtered], [1])
self.assertEqual(filtered[0][0].action, "DELETE")
def test_03_unbatched_mode_accepts_unclosed_intermediate_row(self):
rows = [d("SIM", "ADD", "999.95", False, 10)]
self.assertEqual(toy_fills(rows, batch_deltas=False), 1)
with self.assertRaises(ValueError):
toy_fills(rows, batch_deltas=True)
def test_04_persistent_ask_matches_in_both_modes(self):
rows = [d("SIM", "ADD", "999.95", True, 10)]
self.assertEqual(toy_fills(rows, False), 1)
self.assertEqual(toy_fills(rows, True), 1)
def test_05_transient_ask_exposes_event_granularity(self):
rows = [
d("SIM", "ADD", "1000.10", True, 10),
d("SIM", "ADD", "999.95", False, 20),
d("SIM", "DELETE", "999.95", True, 20),
]
self.assertEqual((toy_fills(rows, False), toy_fills(rows, True)), (1, 0))
def test_06_equal_timestamps_do_not_merge_closed_events(self):
rows = [d("SIM", "ADD", "1", True, 10), d("SIM", "ADD", "2", True, 10)]
self.assertEqual([len(g) for g in group_by_last(rows)], [1, 1])
def test_07_interleaved_symbols_keep_independent_buffers(self):
rows = [
d("A", "ADD", "1", False, 10),
d("B", "ADD", "2", True, 10),
d("A", "DELETE", "1", True, 11),
]
self.assertEqual([[r.instrument for r in g] for g in group_by_last(rows)], [["B"], ["A", "A"]])
def test_08_partial_symbol_does_not_disappear_at_end(self):
rows = [d("A", "ADD", "1", False, 10), d("B", "ADD", "2", True, 11)]
iterator = iter(group_by_last(rows))
self.assertEqual([x.instrument for x in next(iterator)], ["B"])
with self.assertRaisesRegex(ValueError, "missing F_LAST for A"):
next(iterator)
def test_09_empty_input_is_accepted(self):
self.assertEqual(list(group_by_last([])), [])
def test_10_closing_delta_has_last_timestamp_in_toy(self):
rows = [d("SIM", "ADD", "1", False, 10), d("SIM", "DELETE", "1", True, 12)]
grouped = list(group_by_last(rows))
self.assertEqual(grouped[0][-1].ts_init, 12)
self.assertEqual(len(grouped[0]), 2)
if __name__ == "__main__":
unittest.main(verbosity=2)
```
```text
test_01_trailing_incomplete_event_is_rejected (edge_case_verification.OrderBookEventBoundaryControls.test_01_trailing_incomplete_event_is_rejected) ... ok
test_02_head_truncated_event_is_not_detected_by_f_last (edge_case_verification.OrderBookEventBoundaryControls.test_02_head_truncated_event_is_not_detected_by_f_last) ... ok
test_03_unbatched_mode_accepts_unclosed_intermediate_row (edge_case_verification.OrderBookEventBoundaryControls.test_03_unbatched_mode_accepts_unclosed_intermediate_row) ... ok
test_04_persistent_ask_matches_in_both_modes (edge_case_verification.OrderBookEventBoundaryControls.test_04_persistent_ask_matches_in_both_modes) ... ok
test_05_transient_ask_exposes_event_granularity (edge_case_verification.OrderBookEventBoundaryControls.test_05_transient_ask_exposes_event_granularity) ... ok
test_06_equal_timestamps_do_not_merge_closed_events (edge_case_verification.OrderBookEventBoundaryControls.test_06_equal_timestamps_do_not_merge_closed_events) ... ok
test_07_interleaved_symbols_keep_independent_buffers (edge_case_verification.OrderBookEventBoundaryControls.test_07_interleaved_symbols_keep_independent_buffers) ... ok
test_08_partial_symbol_does_not_disappear_at_end (edge_case_verification.OrderBookEventBoundaryControls.test_08_partial_symbol_does_not_disappear_at_end) ... ok
test_09_empty_input_is_accepted (edge_case_verification.OrderBookEventBoundaryControls.test_09_empty_input_is_accepted) ... ok
test_10_closing_delta_has_last_timestamp_in_toy (edge_case_verification.OrderBookEventBoundaryControls.test_10_closing_delta_has_last_timestamp_in_toy) ... ok
----------------------------------------------------------------------
Ran 10 tests in 0.001s
OK
```
## Should the new results replace the old backtest baseline?
Fewer fills under `batch_deltas=True` do not, by themselves, mean “more realistic profit,” and setting `False` merely to preserve an attractive equity curve is no better. **First establish the provider's event boundaries; only then interpret the fill differences.** If several deltas form one logical event, consuming them atomically respects that convention. If the window is missing its beginning, neither boolean value restores the lost state.
Keep the previous results, locate the first diverging `OrderBookDelta`, and inspect its `F_LAST`, `ts_init`, and original grouping before analyzing fees, queue assumptions, trading costs or strategy returns. The same underlying lesson appears in the [vectorbt CVaR sample-denominator case](/posts/vectorbt-nan-cvar-tail-sample-test-assertions/) and [repeated-stock-selection win-rate denominators](/posts/quant-winrate-repeated-stock-picks/): a quantitative number is only interpretable after defining what counts as one complete observation.
*Version and verification note: Sources are pinned to NautilusTrader `v2.0.0rc7`, commit `55c0e82053142bce3a5cba4de4221d3929c2d3bc`. The Python 3.13.5 synthetic fixture and 5+10 boundary checks were recorded on October 18, 2026. This is a teaching example, not an upstream NautilusTrader run or a historical/live trading backtest. It covers replay engineering, not investment advice.*