Junoh Kang

Back

In the previous post, we looked at why LOB message generation matters for finance. Here, we introduce existing generation methods, grouped by the unit the model generates at each step:

  • Field-wise generation (LOBS5): generates a message one field token at a time.
    • Lossless except for clipping, and each token is interpretable.
    • Each message takes many positions, so the context covers a short history.
  • Message-wise generation (M3): generates a whole message at each step, and a decoder or projection head recovers the fields.
    • Each message takes one position, so the context covers a long history.
    • Recovering the fields from a message-level output can be lossy.

LOBS5: Field-wise generation#

Nagy et al.’s Generative AI for End-to-End Limit Order Book Modelling1 provides a direct starting point. LOBS5 uses an S5 state-space model to generate message tokens autoregressively, conditioned on message and book history.

Message representation#

Each message spans 22 discrete tokens describing fields such as event type, side, price, size, time, and reference attributes. All fields share a vocabulary of 12,011 tokens, including three special tokens. For new-order additions, no existing order is referenced, so the reference-order fields are filled with <NA> tokens. A training sequence of 500 messages therefore spans 11,000 positions.1

LOBS5 message fields represented by 22 tokens. Each square denotes one token.
  • Price:
    is the tick offset from the tick-aligned mid-price, clipped to .
  • Quantity:
  • Inter-arrival time:
    The nanosecond component is split into three-digit groups.
  • MarketGPT2 uses a decoder-only Transformer to generate tokenized messages from message histories.
  • sigma-03 increases the message length from 22 to 24 tokens by splitting prices into base-100 chunks. A later version uses 26 tokens per message, but the available documentation does not specify which fields account for the increase from 24 to 26.
  • ByteGen4 goes below fields: it predicts the raw 32-byte event, including absolute price, nanosecond timestamp, and order ID, one byte at a time. It trains on sequences of 100–320 events.

M3: Message-wise generation#

M3 places each order at one sequence position by tokenizing it into one discrete token. Causal Transformer blocks then predict each next order token, conditioned on a prefix that encodes the initial LOB. A decoder translates generated tokens back into order fields, but this round trip is lossy.5

An encoder turns the initial LOB snapshot into prefix vectors. A causal Transformer uses the prefix and order tokens for next-token prediction.
Source: M3, Figure 2(b).

Message representation#

Each order is a five-field vector of three continuous fields and two binary flags:

Event type is add or cancel, and side is bid or ask. The continuous fields are clipped at their 0.5th and 99.5th percentiles.

M3 normalizes prices relative to the daily open so that training and inference use the same price anchor.
  • Why the gap arises. Lossy order reconstruction can shift the simulated mid-price, changing the reference used to decode the next order’s price.
  • Trade-off. Prices relative to the daily open span a wider range because they also reflect intraday price movements.5

The learned tokenizer maps each order to one of 32,768 discrete codes and is trained to reconstruct the order using the following steps.5

  1. Encode. A causal encoder maps the order and its history to a latent vector, .
  2. Select a code. Find the codebook vector closest to . Its index is the order token, .
  3. Look up the vector. Retrieve the vector stored at index . This quantized vector is .
  4. Decode. A causal decoder reconstructs the order fields from the quantized vectors, .

The tokenizer loss has four terms.

Order vectors pass through an encoder and vector quantizer to produce discrete order codes.
Source: M3, Figure 2(a).

Limitations#

  • Lossy representation. Of the reconstructed prices, 37.7% match the original price exactly and 66.2% are within one tick.5
  • Stale prefix in long rollouts. M3 trains on windows of at most 1,024 orders, each starting right after its LOB snapshot. Longer rollouts keep the initial prefix but drop the oldest orders, so the prefix no longer matches the start of the window, a combination never seen in training.5
  • MarS6 also generates one token per order, but its token indexes a fixed grid of type, price, volume, and interval bins. A separate model generates minute-level order distributions that steer the order-level generator.
  • LOBERT7 represents each message at one position as a hand-binned token plus continuous price and volume features, and predicts fine values with regression heads. It evaluates next-message prediction rather than long rollouts.
  • TRADES8 also places each order at one position, but instead of quantizing it into a token, it generates the normalized field values directly with a conditional diffusion model, conditioned on recent orders and LOB snapshots.

Evaluating LOB message generators#

  • LOB-Bench9 compares the distributions of market statistics in generated and historical order flow to assess how closely the simulations resemble real trading.
  • LOB-Impact10 injects orders into the simulated market and tests whether the resulting price impact follows patterns observed in real markets, including how impact grows during execution and decays afterward.

Message representations compared#

ModelPositions per messageVocabularyMessages in contextBook input
Field-wise generation
ByteGen432 bytes256100–320—
LOBS5122 tokens12,011500LOB state per message
MarketGPT224 tokens12,111432—
Message-wise generation
M351 learned code32,7681,024Initial LOB prefix only
MarS61 binned token49,1521,024LOB volumes and mid-price per order
LOBERT71 binned token + continuous values293512Optional LOB snapshot per message
TRADES81 continuous 8-D vector—255LOB snapshot per order

References#

Footnotes#

  1. Generative AI for End-to-End Limit Order Book Modelling ↗, Sections 3–6. ↩ ↩2 ↩3

  2. MarketGPT ↗, Materials and Methods; simulation code ↗, commit 070c3ad. ↩ ↩2

  3. sigma-0 project notes ↗, Q12, and sigma-0 synthetic LOB order flow, release 3a ↗, dataset card. ↩

  4. ByteGen: A Tokenizer-Free Generative Model for Orderbook Events in Byte Space ↗, Sections 3.1 and 4.2 and Appendix A.1. ↩ ↩2

  5. M3: A State-Event Generative Foundation Model for Market Microstructure Dynamics ↗, Sections 3–4 and Appendices B–D. ↩ ↩2 ↩3 ↩4 ↩5 ↩6

  6. MarS: a Financial Market Simulation Engine Powered by Generative Foundation Model ↗, Appendices B–D. ↩ ↩2

  7. LOBERT: Generative AI Foundation Model for Limit Order Book Messages ↗, Sections 2.1, 2.3, and 3.2 and Appendix B. ↩ ↩2

  8. TRADES: Generating Realistic Market Simulations with Diffusion Models ↗, Figure 1 and Sections 4.2 and 5; supplementary material, Section 3. ↩ ↩2

  9. LOB-Bench: Benchmarking Generative AI for Finance – an Application to Limit Order Book Data ↗. ↩

  10. LOB-Impact Benchmark: Emergent Market Impact in Generative Limit Order Book Models ↗. ↩

Search