diff --git a/core/CLAUDE.md b/core/CLAUDE.md index d41de0b..0aa0fdf 100644 --- a/core/CLAUDE.md +++ b/core/CLAUDE.md @@ -28,6 +28,10 @@ Per-unit commands and gotchas live in that unit's `AGENTS.md`. Toolchains (ponyc - **S2:** never reclassify a message's provenance. - **S3 / vault:** the COBOL invariant-law vault (Invariant 0, the culpability anchor; 01, "harm" "less";) is immutable at runtime — don't edit it unless explicitly directed and only as such. +## Subagents + +When spawning background agents (Haiku for research, etc.), **keep working on the main task while they run**. Don't wait idle — fold in results as they arrive, edit other files, or advance unrelated build steps. Background agents are cheap parallelism; wasting the main context window on waiting defeats the purpose. + ## Docs map - `README.md` — the project and its intent. diff --git a/core/docs/plans/M3-sims-hub.md b/core/docs/plans/M3-sims-hub.md index 8cd3f13..2dc7ce3 100644 --- a/core/docs/plans/M3-sims-hub.md +++ b/core/docs/plans/M3-sims-hub.md @@ -20,12 +20,23 @@ simulation cores. A query facade accessible to Traders. Each sim type (M3a–M3g different runtime suited to its math. ## 4. Does / does-not -- **Does:** run continuously across multiple time scales (tick-level, hourly, daily, weekly); - maintain populations of Pops whose behaviors emerge from the sim's mathematical model; ingest - live data from Data Feeds (M2) for calibration; respond to Trader queries with bounded - predictions; produce outputs with **explicit upper/lower bounds** on every prediction value. +- **Does:** run continuously at **≥ 360:1 speed** (360 simulated seconds per wall-clock second) + across **six concurrent time horizons** — tick/hourly, daily, weekly, monthly, annual, and + 5-year forecast windows; maintain populations of Pops whose behaviors emerge from the sim's + mathematical model; ingest live data from Data Feeds (M2) for calibration; respond to Trader + queries with bounded predictions; produce outputs with **explicit upper/lower bounds** on + every prediction value. + | Horizon | Window | Sim cadence at 360:1 | + |---------|--------|---------------------| + | Tick–hourly | Next 1–60 min | Real-time (360 sim-sec/s) | + | Daily | Next 24h | 4 sim-minutes per wall-second | + | Weekly | Next 7d | ~28 sim-minutes per wall-second | + | Monthly | Next 30d | ~2 sim-hours per wall-second | + | Annual | Next 365d | ~1 sim-day per wall-second | + | 5-year | Next 1825d | ~5 sim-days per wall-second | - **Does-not:** trade (Traders/Marketplace do); make decisions for traders (it informs, they - decide); enforce laws (Marketplace does); supervise behavior (Conductor/SAE do). + decide); enforce laws (Marketplace does); supervise behavior (Conductor/SAE do); run slower + than 360:1. ## 5. Interface contract - `query(sim_type: SimType, query: PredictionQuery) -> BoundedPrediction`. @@ -49,11 +60,14 @@ different runtime suited to its math. current state; they don't trigger computation. - **L2 (C5):** every prediction output includes **explicit upper and lower bounds** — no unbounded point estimates. Uncertainty is a first-class value, not an afterthought. -- **L3 (C4):** sims are **read-only from traders' perspective** — a query never mutates sim +- **L3 (C5):** sims advance at a **minimum speed of 360:1** — 360 simulated seconds per 1 + wall-clock second. Sims may run faster but never slower. This ensures predictions stay + ahead of real-time market state across all horizons. +- **L4 (C4):** sims are **read-only from traders' perspective** — a query never mutates sim state. Calibration happens only from Data Feeds (M2). -- **L4 (C4):** each sim type is **independent** — failure in one sim does not cascade to others. +- **L5 (C4):** each sim type is **independent** — failure in one sim does not cascade to others. Degraded sims report their status; traders handle missing predictions. -- **L5 (C3):** Pops are **simulation constructs, not AI actors** — they follow mathematical +- **L6 (C3):** Pops are **simulation constructs, not AI actors** — they follow mathematical rules within the sim. Traders (M5) are the AI actors. ## 8. Build steps diff --git a/core/docs/plans/M3a-statistical-sims.md b/core/docs/plans/M3a-statistical-sims.md index 0e26f62..ff7a748 100644 --- a/core/docs/plans/M3a-statistical-sims.md +++ b/core/docs/plans/M3a-statistical-sims.md @@ -2,32 +2,64 @@ ## 1. Component Pure statistical simulation: **Monte Carlo methods, Bayesian inference, time-series forecasting, -and volatility modeling**. The mathematical backbone — no game theory, no sociology, just the -numbers. Operates across multiple time scales (tick to weekly). Pops in this sim represent +stochastic volatility, regime detection, and cross-asset correlation**. The mathematical backbone +— no game theory, no sociology, just the numbers. Operates across **six concurrent time horizons** +(tick → hourly → daily → weekly → monthly → annual → 5-year). Pops in this sim represent **stochastic sample paths**, not behavioral agents. ## 2. Status / certainty -DESIGN-FIRST · ABSENT. Mathematical foundations C4 (standard quant methods); parameterization C1. +DESIGN-FIRST · ABSENT. Core quant methods C5 (GBM, GARCH, ARIMA — textbook). Heston stochastic +volatility C5 (closed-form characteristic function; industry standard since 1993). Rough +volatility C4 (Gatheral et al. 2018, heavily cited; crypto implementations exist). HMM regime +detection C4 (established; crypto-specific copula hybrids emerging 2023–2024). Almgren-Chriss +execution C5 (industry standard since 2001). Jump-diffusion C5 (Merton 1976). Parameterization +for crypto markets C1. ## 3. Language & location TBD · `src/economy/sims/statistical/`. Python (NumPy/SciPy), Julia, or R for numerical -computing. Needs efficient matrix operations and distribution sampling. +computing. Needs efficient matrix operations, SDE solvers, and distribution sampling. Fractional +Brownian motion generation requires specialized libraries (e.g. `fbm` in Python, or spectral +methods). ## 4. Does / does-not -- **Does:** run Monte Carlo price simulations (geometric Brownian motion, jump-diffusion); - Bayesian parameter estimation from live data (M2); time-series forecasting (ARIMA, GARCH for - volatility clustering); Value-at-Risk and Expected Shortfall calculations; produce bounded - predictions with confidence intervals as upper/lower bounds. +- **Does:** run Monte Carlo price simulations (GBM, Merton jump-diffusion, Heston stochastic + volatility); model volatility surface via **Heston SDE**: + $dS_t = \mu S_t dt + \sqrt{\nu_t} S_t dW_t^S$, + $d\nu_t = \kappa(\theta - \nu_t)dt + \xi\sqrt{\nu_t} dW_t^\nu$ + with $\text{corr}(dW^S, dW^\nu) = \rho$ (mean-reversion speed $\kappa$, long-run variance + $\theta$, vol-of-vol $\xi$); model **rough volatility** via fractional Brownian motion + $dS_t = \mu dt + \sigma_t dB_t^H$ with Hurst exponent $H \approx 0.4$ capturing + antipersistent microstructure (Gatheral et al. 2018); detect **regime transitions** via + Hidden Markov Model: $r_t | s_t \sim \mathcal{N}(\mu_{s_t}, \sigma^2_{s_t})$, + $s_t \in \{\text{Bull, Neutral, Bear}\}$ with Viterbi filter updating in <5ms per tick; + model **cross-asset tail dependence** via DCC-GARCH copula hybrid: + $dQ_t/dt = a \cdot (\bar{S} - Q_t) + b \cdot (\varepsilon_t \varepsilon_t^T - Q_t)$ + with $t$-Copula for fat-tailed spillovers (BTC→alts); Bayesian parameter estimation from + live data (M2); time-series forecasting (ARIMA, GARCH for volatility clustering); Value-at-Risk + and Expected Shortfall; produce bounded predictions with confidence intervals. - **Does-not:** model human behavior (M3b does); model protocol mechanics (M3c–M3f do); - trade or recommend (Traders do). + trade or recommend (Traders do); optimize execution routing (M3g does using our vol estimates). ## 5. Interface contract - Implements `query(PredictionQuery) -> BoundedPrediction` per M3 hub. -- **Output bounds:** statistical confidence intervals. - Example: `{ value: 1847.30, lower_bound: 1790.15, upper_bound: 1905.60, confidence: 0.95, +- **Output bounds:** statistical confidence intervals (CI from Monte Carlo), Heston variance + bands (from $\nu_t$ process), rough-vol forecast cones, regime-conditional intervals. +- **Time-horizon mapping** (all run concurrently): + | Horizon | Primary models | Update cadence | + |---------|---------------|----------------| + | Tick–hourly | Rough vol ($H \approx 0.4$), HMM regime filter, realized variance | Every tick | + | Daily | Heston vol surface, GARCH, DCC correlation | Every bar close | + | Weekly–monthly | Jump-diffusion Monte Carlo, regime-conditional forecasts | Hourly roll | + | Annual–5yr | SDE mean-reversion long-run $\theta$, macro regime priors | Daily roll | +- Examples: + `{ value: 1847.30, lower_bound: 1790.15, upper_bound: 1905.60, confidence: 0.95, time_horizon: "24h", sim_type: "statistical" }` — 95% CI on ETH price. -- **Prediction types:** `price_forecast`, `volatility_estimate`, `var_calculation`, - `correlation_matrix`, `regime_detection`. + `{ value: 0.72, lower_bound: 0.58, upper_bound: 0.89, confidence: 0.90, + time_horizon: "1h", sim_type: "statistical" }` — Heston instantaneous vol $\sqrt{\nu_t}$. + `{ value: "bear", lower_bound: null, upper_bound: null, confidence: 0.83, + time_horizon: "current", sim_type: "statistical" }` — HMM regime state. +- **Prediction types:** `price_forecast`, `volatility_surface`, `var_calculation`, + `correlation_matrix`, `regime_state`, `rough_vol_estimate`, `jump_intensity`. - Calibration: ingests `price_tick` and `dex_pool_state` from M2 Data Feeds. ## 6. Dependencies & stubs @@ -35,25 +67,42 @@ computing. Needs efficient matrix operations and distribution sampling. - M3 Sims hub — lifecycle management; *stub:* manual init. ## 7. Invariants / laws -- **L1 (C4):** bounds are **statistical confidence intervals** — derived from the model's +- **L1 (C5):** bounds are **statistical confidence intervals** — derived from the model's distribution, not hand-picked. The confidence level (e.g. 0.95) is explicit in the output. -- **L2 (C4):** **multiple time scales run concurrently** — a tick-level volatility estimate and a - weekly price forecast coexist; neither blocks the other. -- **L3 (C3):** model parameters are **re-estimated on each calibration** from live data — no - stale parameters carried across regime changes. +- **L2 (C5):** **six time horizons run concurrently** — tick-level rough vol, hourly regime + detection, daily Heston surface, weekly Monte Carlo, annual mean-reversion, and 5-year macro + forecasts coexist; none blocks the others. +- **L3 (C4):** model parameters are **re-estimated on each calibration** from live data — no + stale parameters carried across regime changes. Regime transitions trigger immediate + re-estimation of conditional parameters. +- **L4 (C4):** the **Heston correlation $\rho$ between price and vol** is a fitted parameter, + never assumed — crypto assets exhibit leverage effects different from equities. +- **L5 (C4):** rough volatility Hurst exponent $H$ is **estimated from realized variance**, not + fixed — $H$ varies across assets and regimes (Gatheral et al. 2018). ## 8. Build steps 1. Implement geometric Brownian motion Monte Carlo (simplest price sim). 2. Add GARCH volatility estimation. -3. Wire M2 price data → Bayesian parameter re-estimation. -4. Implement the `BoundedPrediction` output with CIs. +3. Implement Heston SDE solver (Euler-Maruyama with full truncation for $\nu_t \geq 0$). +4. Add Merton jump-diffusion (Poisson jumps + GBM). +5. Implement rough volatility via fractional BM with rolling Hurst estimator. +6. Implement HMM regime detector (3-state Viterbi filter). +7. Add DCC-GARCH copula for cross-asset correlation. +8. Wire M2 price data → Bayesian parameter re-estimation. +9. Implement multi-horizon `BoundedPrediction` output with CIs. ## 9. Tests Monte Carlo: N sample paths produce a distribution with correct mean/variance. CI: 95% interval contains true value ≥ 95% of the time on historical backtest. GARCH: volatility clusters detected -in synthetic data. Calibration: new data shifts parameter estimates. +in synthetic data. Heston: implied vol smile reproduced for known parameters. Rough vol: Hurst +exponent recovered from synthetic fBM paths. HMM: regime transitions detected within 15–60s on +synthetic regime-switching data. Copula: tail dependence captured (BTC crash → alt crash +correlation spike). Jump-diffusion: fat tails reproduced. Calibration: new data shifts parameter +estimates. Multi-horizon: all six horizons produce concurrent outputs. ## 10. Open items -- Which distributions beyond GBM (heavy-tailed? Lévy?). -- Regime-switching model complexity (hidden Markov? threshold?). -- Computational budget (how many Monte Carlo paths per tick?). +- Heston calibration method (characteristic function inversion? particle filter?). +- Rough vol computational cost (fBM generation is O(N²) naively; FFT methods needed). +- HMM state count (3 sufficient? 4+ for crypto with "mania" regime?). +- Which copula family for tail dependence ($t$-copula? Clayton? Joe?). +- Computational budget per horizon (GPU for Monte Carlo paths?). diff --git a/core/docs/plans/M3b-sociological-sims.md b/core/docs/plans/M3b-sociological-sims.md index a23e6fc..79b6d23 100644 --- a/core/docs/plans/M3b-sociological-sims.md +++ b/core/docs/plans/M3b-sociological-sims.md @@ -1,38 +1,75 @@ # M3b — Sociological & population dynamics sims ## 1. Component -Sociological simulation: **evolutionary game theory, bounded rationality, sentiment cascades, and -population dynamics** among market participants. Pops here are **behavioral archetypes** — retail -herd followers, contrarian whales, MEV searchers, passive LPs — whose strategies evolve under -selection pressure. Grounded in evolutionary consensus game models [8,9] and bounded-rationality -coordination frameworks. +Sociological simulation: **evolutionary game theory, bounded rationality, opinion dynamics, +complex contagion, adaptive learning populations, and Mean-Field Game equilibria** among market +participants. Pops here are **behavioral archetypes** — retail herd followers, contrarian whales, +MEV searchers, passive LPs, pump-and-dump manipulators — whose strategies evolve under selection +pressure across **six concurrent time horizons**. Grounded in evolutionary consensus game models +[8,9], Hegselmann-Krause opinion dynamics (2002), complex contagion theory (Centola & Macy 2007), +MFG theory (Lasry & Lions 2007), and crypto manipulation ABMs. ## 2. Status / certainty -DESIGN-FIRST · ABSENT. Evolutionary game-theory foundations C4 (Cornell blockchain cooperation -literature [8]); pop behavioral models C1. +DESIGN-FIRST · ABSENT. Evolutionary game theory C4 (Cornell [8]). Hegselmann-Krause bounded +confidence C5 (established 2002). Complex contagion C4 (Centola & Macy 2007; crypto applications +C3). Bandit-replicator hybrid C3 (emerging). Mean-Field Games C4 (Lasry & Lions 2007; tensor-train +solvers C3). Crypto pump-and-dump ABM C3 (3-agent protocol validated on historical data). +Pop behavioral models C1. ## 3. Language & location TBD · `src/economy/sims/sociological/`. Agent-based modeling frameworks (Mesa/Python, NetLogo, -or custom). Needs efficient population iteration and strategy mutation. +or custom). Needs efficient population iteration, strategy mutation, PDE solvers for MFG +(HJB + Fokker-Planck), and bandit algorithms (UCB/Thompson). ## 4. Does / does-not - **Does:** simulate populations of behavioral archetypes competing in a market; apply - evolutionary dynamics (replicator equation, mutation, selection) to strategy distributions; - model sentiment cascades (fear/greed contagion across pop clusters); model bounded rationality - (pops satisfice, not optimize — they follow heuristics, not perfect strategies); produce - bounded predictions on market sentiment, herd behavior thresholds, and coordination breakdowns. + **replicator dynamics** $dx_i/dt = x_i(\pi_i(x) - \bar{\pi}(x))$ to strategy distributions; + model **opinion clustering** via Hegselmann-Krause bounded confidence: + $x_i(t+1) = x_i(t) + \mu(x_j(t) - x_i(t))$ for $|x_i - x_j| \leq d$ — agents only update + toward neighbors within confidence bound $d$, creating natural clustering and trend-reversal + thresholds; model **complex contagion** with heterogeneous thresholds: adoption probability + $P_i = f(n_i / k_i)$ where multiple exposures amplify adoption non-linearly (captures meme-coin + rallies and narrative-driven pumps); implement **bandit-replicator hybrid** where pops use + UCB or Thompson Sampling to estimate strategy payoffs: + $x_i'(t) = x_i(t)[\lambda_i(t) - \bar{\lambda}(t)]$ with $\lambda_i = \text{UCB}(\theta_i)$ + — bridges replicator dynamics with multi-armed bandit learning; solve **Mean-Field Game + equilibria** via coupled HJB + Fokker-Planck PDEs for large-population limits: + $-\partial_t u + H(x, \nabla u) = F(x, m)$ (HJB, individual optimization), + $\partial_t m - \nabla \cdot (m \nabla_p H) = 0$ (Fokker-Planck, population density) — Newton + iteration with tensor-train decomposition reduces $O(N^d)$ to $O(dNr^2)$ for high-dimensional + state spaces; simulate **crypto pump-and-dump protocol** with 3 pop types: Normal traders, + Market Analysts (MA, information-advantaged), Market Players (MP, manipulators) in a 4-phase + cycle (accumulation → promotion → distribution → collapse); model sentiment cascades + (fear/greed contagion across pop clusters); model bounded rationality (pops satisfice, not + optimize — heuristics, not perfect strategies); produce bounded predictions across all six + time horizons. - **Does-not:** model protocol mechanics (M3c–M3f); compute statistical forecasts (M3a); represent real individuals (pops are archetypes, not profiles). ## 5. Interface contract - Implements `query(PredictionQuery) -> BoundedPrediction` per M3 hub. -- **Output bounds:** population-fraction ranges and sentiment scales. - Example: `{ value: 7.3, lower_bound: 5.0, upper_bound: 9.1, confidence: 0.68, - time_horizon: "12h", sim_type: "sociological" }` — herd-panic index on a 0–10 scale. - Example: `{ value: 0.42, lower_bound: 0.31, upper_bound: 0.55, confidence: 0.72, +- **Output bounds:** population-fraction ranges, sentiment scales, MFG equilibrium stability. +- **Time-horizon mapping** (all run concurrently): + | Horizon | Primary models | Update cadence | + |---------|---------------|----------------| + | Hourly | Hegselmann-Krause opinion clusters, bandit-replicator | Every data tick | + | Daily | Complex contagion cascades, pump-and-dump phase detection | Hourly roll | + | Weekly | Replicator dynamics strategy evolution, MFG equilibrium | Daily roll | + | Monthly | Population archetype composition, narrative regime shifts | Weekly roll | + | Annual | Long-run evolutionary stable strategies (ESS) | Monthly roll | + | 5-year | MFG stationary equilibria, structural population shifts | Quarterly roll | +- Examples: + `{ value: 7.3, lower_bound: 5.0, upper_bound: 9.1, confidence: 0.68, + time_horizon: "12h", sim_type: "sociological" }` — herd-panic index (0–10). + `{ value: 0.42, lower_bound: 0.31, upper_bound: 0.55, confidence: 0.72, time_horizon: "1w", sim_type: "sociological" }` — fraction of pops in "contrarian" strategy. + `{ value: "promotion", lower_bound: null, upper_bound: null, confidence: 0.61, + time_horizon: "current", sim_type: "sociological" }` — pump-and-dump phase detection. + `{ value: 0.78, lower_bound: 0.65, upper_bound: 0.88, confidence: 0.70, + time_horizon: "30d", sim_type: "sociological" }` — MFG equilibrium stability index. - **Prediction types:** `sentiment_index`, `herd_threshold`, `strategy_distribution`, - `cascade_probability`, `coordination_stability`. + `cascade_probability`, `coordination_stability`, `opinion_cluster_count`, + `pump_dump_phase`, `mfg_equilibrium_stability`, `narrative_regime`. - Calibration: ingests `rss_news` (sentiment signal) and `price_tick` (realized behavior) from M2. ## 6. Dependencies & stubs @@ -40,28 +77,43 @@ or custom). Needs efficient population iteration and strategy mutation. - M3 Sims hub — lifecycle management; *stub:* manual init. ## 7. Invariants / laws -- **L1 (C4):** pops are **archetypes, not individuals** — no attempt to model or track real +- **L1 (C5):** pops are **archetypes, not individuals** — no attempt to model or track real market participants. The sim models emergent behavior from strategy populations. -- **L2 (C4):** strategies **evolve** — the population distribution shifts over time via +- **L2 (C5):** strategies **evolve** — the population distribution shifts over time via replicator dynamics. No fixed strategy ratios. -- **L3 (C3):** bounded rationality is the **default** — pops satisfice with heuristics, not +- **L3 (C4):** bounded rationality is the **default** — pops satisfice with heuristics, not optimize with perfect information. Rational-agent models are a special case, not the baseline. +- **L4 (C4):** **complex contagion requires multiple exposures** — adoption is non-linear in + neighbor count, not simple diffusion. Single-exposure models undercount threshold effects. +- **L5 (C4):** the MFG limit is **valid only for large populations** — below ~100 pops, use + discrete replicator dynamics; above, the continuum HJB+FP approximation applies. +- **L6 (C3):** pump-and-dump detection is **phase-based** — the 4-phase cycle (accumulate → + promote → distribute → collapse) has distinct statistical signatures in volume and price. ## 8. Build steps 1. Define pop archetypes and their heuristic strategies. 2. Implement replicator dynamics (strategy evolution over generations). -3. Implement sentiment contagion model (network-based cascade). -4. Wire M2 news/price data → calibration of pop parameters. -5. Implement `BoundedPrediction` output with population-fraction CIs. +3. Implement Hegselmann-Krause bounded confidence opinion model. +4. Implement complex contagion with heterogeneous thresholds. +5. Implement bandit-replicator hybrid (UCB payoff estimation + replicator selection). +6. Implement MFG solver (HJB + Fokker-Planck with Newton iteration). +7. Implement pump-and-dump 3-type ABM (Normal, MA, MP) with 4-phase protocol. +8. Wire M2 news/price data → calibration of pop parameters. +9. Implement multi-horizon `BoundedPrediction` output. ## 9. Tests Evolution: dominant strategy shifts when payoff landscape changes. Cascade: sentiment shock -propagates through pop network above threshold, not below. Bounded rationality: satisficing pop -underperforms optimizer in simple games but outperforms in noisy environments. Bounds: all -outputs include upper/lower. +propagates through pop network above threshold, not below. Bounded confidence: opinion clusters +form at predicted cluster count for given $d$. Complex contagion: multiple-exposure requirement +produces slower but more robust adoption than simple contagion. Bandit: explore-exploit tradeoff +produces adapting populations. MFG: equilibrium converges for large N; matches discrete sim for +small N. Pump-dump: 4-phase cycle detected on synthetic manipulation data. Bounds: all outputs +include upper/lower. ## 10. Open items - Pop archetype catalog (which behavioral types? how many?). - Network topology for sentiment contagion (small-world? scale-free?). - Calibration from real market data — how to infer pop distribution from observable price action. +- MFG tensor-train rank $r$ (accuracy vs. compute tradeoff). +- Hegselmann-Krause confidence bound $d$ — fixed or adaptive? - Cross-sim interaction: do sociological predictions feed into M3c (AMM) or M3d (MEV)? diff --git a/core/docs/plans/M3c-amm-liquidity-sims.md b/core/docs/plans/M3c-amm-liquidity-sims.md index 3be5158..422cc4b 100644 --- a/core/docs/plans/M3c-amm-liquidity-sims.md +++ b/core/docs/plans/M3c-amm-liquidity-sims.md @@ -31,6 +31,15 @@ invariant calculations (Solidity-equivalent precision). Python, Rust, or Julia. time_horizon: "7d", sim_type: "amm_liquidity" }` — projected impermanent loss for ETH/USDC pool. Example: `{ value: 0.082, lower_bound: 0.041, upper_bound: 0.127, confidence: 0.85, time_horizon: "30d", sim_type: "amm_liquidity" }` — net LP return (fees − IL). +- **Time-horizon mapping** (all run concurrently, ≥ 360:1 speed): + | Horizon | Primary models | Update cadence | + |---------|---------------|----------------| + | Tick–hourly | Slippage curves, invariant state, JIT liquidity | Every swap event | + | Daily | IL accumulation, fee income, LP profitability | Hourly roll | + | Weekly | Optimal LP range recalculation, pool composition | Daily roll | + | Monthly | LP strategy evolution (passive vs. active rebalance) | Weekly roll | + | Annual | Pool lifecycle, fee tier competitiveness | Monthly roll | + | 5-year | AMM design evolution, concentrated liquidity adoption | Quarterly roll | - **Prediction types:** `impermanent_loss`, `pool_return`, `optimal_range`, `slippage_estimate`, `lp_withdrawal_threshold`. - Calibration: ingests `dex_pool_state` and `price_tick` from M2. diff --git a/core/docs/plans/M3d-mev-adversarial-sims.md b/core/docs/plans/M3d-mev-adversarial-sims.md index 1e0e6bb..18ec3fa 100644 --- a/core/docs/plans/M3d-mev-adversarial-sims.md +++ b/core/docs/plans/M3d-mev-adversarial-sims.md @@ -1,69 +1,114 @@ # M3d — MEV & adversarial extraction sims ## 1. Component -Maximal Extractable Value simulation: models **transaction ordering as an optimization problem**, -**Priority Gas Auctions (PGA) as all-pay auctions**, and **block building as a multidimensional -knapsack problem**. Pops here are **searcher bots, block builders, and validators** competing for -extractable value. Grounded in ACM MEV game theory [3] and knapsack auction literature [4,5]. +Maximal Extractable Value and adversarial simulation: models **transaction ordering as an +optimization problem**, **Priority Gas Auctions (PGA) as all-pay auctions**, **block building as a +multidimensional knapsack problem**, **cross-chain adversarial arbitrage**, and **Dynamic +Stackelberg Mean-Field Games (DSMFG) for protocol-level adversarial policy**. Pops here are +**searcher bots, block builders, validators, cross-chain arbitrageurs, and adversarial +manipulators** competing for extractable value across **six concurrent time horizons**. + +Grounded in ACM MEV game theory [3], knapsack auction literature [4,5], Kolokoltsov adversarial +dynamics (non-linear Fokker-Planck with WENO shock capturing), and DSMFG bilevel optimization +(leader policy + follower MFG equilibrium). ## 2. Status / certainty -DESIGN-FIRST · ABSENT. PGA-as-all-pay-auction model C4 (ACM [3]); knapsack formulation C4 -(Cornell [4,5]); simulation parameterization C1. +DESIGN-FIRST · ABSENT. PGA-as-all-pay-auction C4 (ACM [3]); knapsack formulation C4 (Cornell +[4,5]); cross-chain arbitrage C4 (ACM SIGMETRICS 2025, 5.5x growth since Dencun); DSMFG bilevel +optimization C3 (emerging — SMFRL solvers); Kolokoltsov adversarial C3 (non-linear Fokker-Planck; +WENO discretization established but crypto application novel). Parameterization C1. ## 3. Language & location -TBD · `src/economy/sims/mev/`. Needs combinatorial optimization (for knapsack) and continuous-time -auction modeling. Python (PuLP/OR-Tools for optimization), Rust, or Julia. +TBD · `src/economy/sims/mev/`. Needs combinatorial optimization (PuLP/OR-Tools for knapsack), +continuous-time auction modeling, PDE solvers (WENO for shock-capturing in adversarial dynamics), +and bilevel optimization (DSMFG). Python, Rust, or Julia. ## 4. Does / does-not - **Does:** simulate Priority Gas Auctions where multiple searcher bots compete for the same arbitrage opportunity $V$ by bidding gas fees $g$ in a continuous-time all-pay auction; model block building as a multidimensional knapsack problem (scarce block space, heterogeneous transaction values/sizes); simulate endogenous selection cutoffs under paid-priority ordering; - predict MEV exposure for proposed trades; produce bounded predictions on extraction risk and - optimal gas strategies. -- **Does-not:** extract MEV itself (this is a simulator, not a searcher); model AMM mechanics - (M3c handles pool math); model social dynamics (M3b). + model **cross-chain adversarial arbitrage** with inventory vs. bridge execution trade-off: + $\pi_{\text{inv}} = (P_{\text{src}} - P_{\text{dst}} - \text{slippage} - \text{gas}) \times q$ + vs. $\pi_{\text{bridge}} = (P_{\text{src}} - P_{\text{dst}} - \text{fee} - + \text{depreciation}(\Delta t)) \times q$ where bridge latency $\Delta t \approx 242$s vs. + inventory $\Delta t \approx 9$s; solve **Dynamic Stackelberg MFG** for adversarial policy + design — bilevel optimization where a leader (protocol/regulator) sets policy and followers + (searchers) respond as an MFG equilibrium: the leader solves + $\min_\alpha J_L(\alpha, m^*(\alpha))$ subject to $m^*(\alpha)$ being the MFG Nash equilibrium + of followers under policy $\alpha$; model **Kolokoltsov adversarial dynamics** via non-linear + Fokker-Planck: $\partial_t m + \nabla \cdot (b(x,m)m) = \frac{1}{2}\nabla^2(\sigma^2 m)$ + with WENO shock-capturing for discontinuous adversarial strategies; predict MEV exposure for + proposed trades; produce bounded predictions on extraction risk across all six time horizons. +- **Does-not:** extract MEV itself (simulator, not a searcher); model AMM pool math (M3c); + model social dynamics (M3b); execute cross-chain bridges (Marketplace does). ## 5. Interface contract - Implements `query(PredictionQuery) -> BoundedPrediction` per M3 hub. -- **Output bounds:** extraction probability ranges and gas cost intervals. - Example: `{ value: 0.23, lower_bound: 0.11, upper_bound: 0.38, confidence: 0.80, - time_horizon: "next_block", sim_type: "mev_adversarial" }` — probability this trade gets - sandwiched. - Example: `{ value: 14.7, lower_bound: 8.2, upper_bound: 22.5, confidence: 0.75, - time_horizon: "next_block", sim_type: "mev_adversarial" }` — optimal gas bid (gwei) for - a given opportunity. +- **Output bounds:** extraction probability ranges, gas cost intervals, cross-chain profit + bounds, DSMFG equilibrium stability ranges. +- **Time-horizon mapping** (all run concurrently, ≥ 360:1 speed): + | Horizon | Primary models | Update cadence | + |---------|---------------|----------------| + | Tick–hourly | PGA auctions, sandwich detection, cross-chain arb | Every block | + | Daily | Knapsack builder strategies, MEV landscape | Hourly roll | + | Weekly | Searcher population dynamics, cross-chain flow patterns | Daily roll | + | Monthly | DSMFG policy equilibria, adversarial strategy evolution | Weekly roll | + | Annual | Kolokoltsov adversarial long-run dynamics | Monthly roll | + | 5-year | Structural MEV regime shifts, protocol-level policy effects | Quarterly roll | +- Examples: + `{ value: 0.23, lower_bound: 0.11, upper_bound: 0.38, confidence: 0.80, + time_horizon: "next_block", sim_type: "mev_adversarial" }` — sandwich probability. + `{ value: 14.7, lower_bound: 8.2, upper_bound: 22.5, confidence: 0.75, + time_horizon: "next_block", sim_type: "mev_adversarial" }` — optimal gas bid (gwei). + `{ value: 0.034, lower_bound: 0.018, upper_bound: 0.052, confidence: 0.82, + time_horizon: "1h", sim_type: "mev_adversarial" }` — cross-chain arb profit (ETH). - **Prediction types:** `sandwich_probability`, `frontrun_risk`, `optimal_gas_bid`, - `block_inclusion_probability`, `mev_exposure`. -- Calibration: ingests `on_chain_event` (mempool-like data) and `price_tick` from M2. + `block_inclusion_probability`, `mev_exposure`, `cross_chain_arb_profit`, + `adversarial_policy_stability`, `searcher_population_shift`. +- Calibration: ingests `on_chain_event` (mempool-like data), `price_tick`, and cross-chain + bridge state from M2. ## 6. Dependencies & stubs -- M2 Data Feeds — on-chain events and gas data; *stub:* canned mempool snapshots. +- M2 Data Feeds — on-chain events, gas data, cross-chain state; *stub:* canned snapshots. - M3 Sims hub — lifecycle management; *stub:* manual init. - M3c AMM sims — pool state for arbitrage opportunity detection; *stub:* fixed pool state. ## 7. Invariants / laws -- **L1 (C4):** PGA is modeled as an **all-pay auction** — all bidders pay their gas whether they +- **L1 (C5):** PGA is modeled as an **all-pay auction** — all bidders pay their gas whether they win or not. The sim must capture this cost structure (not winner-pays-only). -- **L2 (C4):** block building is a **knapsack problem, not a queue** — builders optimize for +- **L2 (C5):** block building is a **knapsack problem, not a queue** — builders optimize for total extracted value subject to gas limit constraints, not first-come-first-served. -- **L3 (C3):** MEV exposure predictions are **pre-trade** — traders query this sim *before* +- **L3 (C4):** MEV exposure predictions are **pre-trade** — traders query this sim *before* submitting to the Marketplace to understand their extraction risk. +- **L4 (C4):** cross-chain arb models **both execution paths** — inventory (fast, capital- + intensive) and bridge (slow, capital-light) — never assumes one dominates. +- **L5 (C3):** DSMFG solutions are **bilevel** — the leader's optimal policy depends on the + followers' MFG equilibrium, which itself depends on the leader's policy. Fixed-point iteration + or SMFRL solvers required. ## 8. Build steps 1. Implement the PGA all-pay auction model (N searchers, opportunity value V, gas bids). 2. Implement the block-building knapsack solver. 3. Add sandwich/frontrun detection heuristics. -4. Wire M2 on-chain data → calibration of searcher population and gas dynamics. -5. Wire pre-trade query interface for Traders. +4. Implement cross-chain arb model (inventory vs. bridge, latency, MEV exposure). +5. Implement Kolokoltsov non-linear Fokker-Planck with WENO discretization. +6. Implement DSMFG bilevel solver (leader policy + follower MFG equilibrium). +7. Wire M2 on-chain + cross-chain data → calibration. +8. Wire pre-trade query interface for Traders. ## 9. Tests All-pay: losing bidders still pay gas cost. Knapsack: builder selects optimal transaction set under gas limit. Sandwich: known sandwich-vulnerable trade flagged; non-vulnerable trade clear. -Bounds: all outputs bounded. Pre-trade: query does not submit any transaction. +Cross-chain: inventory path preferred when latency advantage exceeds capital cost. DSMFG: leader +policy converges to fixed point with follower equilibrium. Kolokoltsov: WENO captures shock +discontinuities in adversarial strategy distribution. Bounds: all outputs bounded. Pre-trade: +query does not submit any transaction. Speed: sim advances ≥ 360:1. ## 10. Open items - Mempool data access (public mempool? private order flow?). - Which MEV types to model initially (sandwich, backrun, liquidation, JIT?). - Multi-block MEV (cross-block extraction strategies). -- Integration with M3c (arbitrage opportunities arise from AMM pool state). +- DSMFG solver choice (SMFRL? fictitious play? direct bilevel optimization?). +- WENO order for Kolokoltsov (3rd? 5th? tradeoff with compute budget). +- Which L2s/bridges to model for cross-chain (Arbitrum? Optimism? Base?). diff --git a/core/docs/plans/M3e-tokenomics-macro-sims.md b/core/docs/plans/M3e-tokenomics-macro-sims.md index 01e7265..61bee9f 100644 --- a/core/docs/plans/M3e-tokenomics-macro-sims.md +++ b/core/docs/plans/M3e-tokenomics-macro-sims.md @@ -2,19 +2,26 @@ ## 1. Component Macro-level token economy simulation: models **token supply dynamics, monetary policy (halvings, -burns, inflation), and systemic stock-flow balances** using stochastic differential equations -(SDEs) and state-space models. Pops here are **aggregate behavioral cohorts** (miners/validators, -holders, speculators, protocol treasuries) whose collective behavior drives token-level dynamics. -Grounded in the Vienna University complex-systems token modeling [7] and ResearchGate engineering -token economy frameworks [6]. +burns, inflation), lending protocol dynamics, DeFi systemic risk, and stock-flow balances** using +stochastic differential equations (SDEs), state-space models, kinked interest rate curves, and +inter-protocol credit exposure networks. Pops here are **aggregate behavioral cohorts** +(miners/validators, holders, speculators, protocol treasuries, borrowers/lenders) whose collective +behavior drives token-level dynamics across **six concurrent time horizons** at ≥ 360:1 speed. + +Grounded in Vienna complex-systems token modeling [7], ResearchGate engineering token economy +frameworks [6], Aave/Compound kinked interest rate models (industry standard), and DeXposure +inter-protocol credit propagation (Matzakos et al. 2025). ## 2. Status / certainty DESIGN-FIRST · ABSENT. SDE state-space framework C4 (Vienna [7]); stock-flow modeling C4 -(ResearchGate [6]); specific token model parameters C1. +(ResearchGate [6]); kinked interest rate model C5 (Aave/Compound production standard); +DeXposure inter-protocol credit propagation C3 (emerging, 2025 — high DeFi specificity); +composable yield optimization C4 (Yearn v3, Beefy, production-validated). Specific parameters C1. ## 3. Language & location -TBD · `src/economy/sims/tokenomics/`. Needs SDE solvers (Euler-Maruyama, Milstein) and -state-space estimation. Julia (DifferentialEquations.jl), Python (scipy), or Octave. +TBD · `src/economy/sims/tokenomics/`. Needs SDE solvers (Euler-Maruyama, Milstein), +state-space estimation, and VAR (vector autoregression) for credit exposure impulse responses. +Julia (DifferentialEquations.jl), Python (scipy), or Octave. ## 4. Does / does-not - **Does:** simulate token state dynamics via the SDE framework: @@ -22,51 +29,90 @@ state-space estimation. Julia (DifferentialEquations.jl), Python (scipy), or Oct $u$ is the behavioral policy function, deterministic drift captures programmatic parameters (halvings, burns), and Brownian motion $\sigma dW_t$ captures stochastic behavioral shocks; model stock-flow balances (circulating supply, staked, locked, burned); simulate monetary - policy impacts (halving events, fee burns, treasury emissions); produce bounded predictions - on token supply trajectories, inflation rates, and velocity. + policy impacts (halving events, fee burns, treasury emissions); model **lending protocol + dynamics** via kinked interest rate curves: + $R = R_0 + R_{\text{slope1}} \times U$ if $U \leq U_{\text{opt}}$, + $R = R_0 + R_{\text{slope1}} \times U_{\text{opt}} + R_{\text{slope2}} \times + (U - U_{\text{opt}})$ if $U > U_{\text{opt}}$ where $U = \text{Borrowed}/(\text{Supplied} + + \text{Borrowed})$, $U_{\text{opt}} \approx 0.8$ — cascade liquidations when + $\text{collateral} \times \text{LTV} < \text{borrowed}$; model **DeFi systemic risk** via + DeXposure inter-protocol credit propagation: + $E_{ij}(t) = \sum_{\text{tokens}} [\text{TVL}_j(\text{token}) \times + \text{ownership}_i(\text{token})]$ with VAR impulse responses for shock contagion across + protocols sharing collateral; model **composable yield optimization**: + $\max \sum_i w_i(t) \cdot \text{APY}_i(t) - \lambda \sum_i w_i(t)^2 \sigma_i^2(t)$ + subject to $\sum_i w_i = 1$ — dynamic rebalancing across lending, LP, and staking strategies; + produce bounded predictions on token supply, protocol health, yield, and systemic risk. - **Does-not:** model individual transactions (M3c/M3d); model social sentiment (M3b); - model consensus mechanics (M3f). + model consensus mechanics (M3f); execute yield strategies (Traders/Marketplace do). ## 5. Interface contract - Implements `query(PredictionQuery) -> BoundedPrediction` per M3 hub. -- **Output bounds:** SDE confidence bands (derived from the stochastic component $\sigma dW_t$). - Example: `{ value: 2.1, lower_bound: 1.4, upper_bound: 3.2, confidence: 0.90, +- **Output bounds:** SDE confidence bands, utilization rate ranges, contagion impact intervals. +- **Time-horizon mapping** (all run concurrently, ≥ 360:1 speed): + | Horizon | Primary models | Update cadence | + |---------|---------------|----------------| + | Hourly | Lending rates, utilization, liquidation risk | Every block | + | Daily | Yield optimization, protocol TVL flows | Hourly roll | + | Weekly | SDE supply trajectory, stock-flow balances | Daily roll | + | Monthly | DeXposure credit contagion, systemic risk | Weekly roll | + | Annual | Halving/burn policy impacts, inflation trajectory | Monthly roll | + | 5-year | Token supply long-run equilibrium, protocol lifecycle | Quarterly roll | +- Examples: + `{ value: 2.1, lower_bound: 1.4, upper_bound: 3.2, confidence: 0.90, time_horizon: "90d", sim_type: "tokenomics_macro" }` — annualized inflation rate (%). - Example: `{ value: 0.67, lower_bound: 0.58, upper_bound: 0.74, confidence: 0.85, - time_horizon: "30d", sim_type: "tokenomics_macro" }` — staking ratio (fraction of supply). + `{ value: 0.67, lower_bound: 0.58, upper_bound: 0.74, confidence: 0.85, + time_horizon: "30d", sim_type: "tokenomics_macro" }` — staking ratio. + `{ value: 0.83, lower_bound: 0.78, upper_bound: 0.91, confidence: 0.88, + time_horizon: "1h", sim_type: "tokenomics_macro" }` — Aave ETH utilization rate. + `{ value: 0.12, lower_bound: 0.04, upper_bound: 0.25, confidence: 0.72, + time_horizon: "7d", sim_type: "tokenomics_macro" }` — systemic contagion risk index. - **Prediction types:** `supply_trajectory`, `inflation_rate`, `staking_ratio`, - `velocity_estimate`, `halving_impact`, `treasury_runway`. -- Calibration: ingests `on_chain_event` (supply metrics, staking data) from M2. + `velocity_estimate`, `halving_impact`, `treasury_runway`, `utilization_rate`, + `liquidation_cascade_risk`, `systemic_contagion_index`, `optimal_yield_allocation`. +- Calibration: ingests `on_chain_event` (supply metrics, staking data, lending protocol state, + TVL) from M2. ## 6. Dependencies & stubs -- M2 Data Feeds — on-chain supply/staking data; *stub:* canned supply snapshots. +- M2 Data Feeds — on-chain supply/staking/lending data; *stub:* canned supply snapshots. - M3 Sims hub — lifecycle management; *stub:* manual init. ## 7. Invariants / laws -- **L1 (C4):** the SDE framework is the **canonical representation** — all token dynamics are +- **L1 (C5):** the SDE framework is the **canonical representation** — all token dynamics are expressed as drift + diffusion. Deterministic policy (halvings, burns) lives in the drift $f$; behavioral uncertainty lives in the diffusion $\sigma dW_t$. -- **L2 (C4):** **stock-flow conservation** — tokens are never created or destroyed outside the - protocol's defined mechanisms. The sim must balance: circulating + staked + locked + burned = - total ever minted. -- **L3 (C3):** macro sims operate on **aggregate cohorts, not individuals** — the state vector +- **L2 (C5):** **stock-flow conservation** — tokens are never created or destroyed outside the + protocol's defined mechanisms. circulating + staked + locked + burned = total ever minted. +- **L3 (C5):** lending rate curves are **kinked at $U_{\text{opt}}$** — the steep slope above + optimal utilization is a design invariant of Aave/Compound, not a parameter to smooth. +- **L4 (C4):** macro sims operate on **aggregate cohorts, not individuals** — the state vector $X_t$ tracks population-level quantities (total staked, total circulating), not per-wallet. +- **L5 (C4):** DeXposure contagion is **directional** — protocol A's exposure to protocol B + is not symmetric. The exposure matrix $E_{ij}$ is not assumed symmetric. ## 8. Build steps 1. Implement Euler-Maruyama SDE solver for a simple token model (supply + staking). 2. Define the state vector $X_t$ and drift/diffusion functions for a reference token. 3. Add stock-flow accounting (verify conservation). -4. Wire M2 on-chain data → state estimation / calibration. -5. Add monetary policy events (halving, burn) as drift discontinuities. +4. Implement kinked lending rate model (Aave-style) with liquidation cascade simulation. +5. Implement DeXposure credit propagation network with VAR impulse responses. +6. Implement composable yield optimizer (risk-adjusted return maximization). +7. Wire M2 on-chain data → state estimation / calibration. +8. Add monetary policy events (halving, burn) as drift discontinuities. ## 9. Tests SDE: sample paths have correct mean (matches drift) and variance (matches diffusion). Stock-flow: conservation holds across all time steps. Halving: supply growth rate drops at halving event. -Calibration: state estimate converges to observed data. Bounds: SDE confidence bands correctly -cover realized paths on backtest. +Lending: rate curve exhibits kink at $U_{\text{opt}}$; liquidation cascades triggered when +collateral ratio breached. DeXposure: shock to protocol A propagates to protocol B through shared +collateral; isolated protocols unaffected. Yield: optimizer rebalances toward highest risk-adjusted +APY. Calibration: state estimate converges to observed data. Bounds: SDE confidence bands cover +realized paths on backtest. Speed: sim advances ≥ 360:1. ## 10. Open items - Which tokens to model initially (ETH? BTC? a specific alt?). - State vector dimensionality (how many state variables per token model?). - Behavioral policy function $u(X_t, t)$ — how to parameterize aggregate cohort behavior. - Multi-token interactions (correlated diffusions across tokens?). +- DeXposure graph granularity (how many protocols? top-10 by TVL?). +- Lending model extensions (Morpho AdaptiveCurveIRM? variable kink parameters?). diff --git a/core/docs/plans/M3f-consensus-staking-sims.md b/core/docs/plans/M3f-consensus-staking-sims.md index bb3e06a..6e61d61 100644 --- a/core/docs/plans/M3f-consensus-staking-sims.md +++ b/core/docs/plans/M3f-consensus-staking-sims.md @@ -9,7 +9,8 @@ theorems [10], and Monash dynamic PBFT modeling [11]. ## 2. Status / certainty DESIGN-FIRST · ABSENT. Evolutionary PoS game theory C4 (Cornell [8]); staking pool Nash -equilibrium proofs C4 (ACM [10]); Markov chain throughput models C4 (Monash [11]); +equilibrium proofs C4 (ACM [10]); Markov chain throughput models C4 (Monash [11]); MFG for +validator populations C4 (Lasry & Lions 2007; validator-specific application C3); simulation parameterization C1. ## 3. Language & location @@ -21,8 +22,12 @@ computation. Python, Julia, or R. under bounded rationality [8]; model staking pool delegation as a game with proven reward- parameter thresholds enforcing subgame-perfect Nash equilibria favoring honest validation over malicious slashing [10]; simulate **throughput stability under shifting validator states** via - Markov chains [11]; predict slashing risk, validator set stability, and staking yield; produce - bounded predictions on consensus health and staking returns. + Markov chains [11]; solve **Mean-Field Game equilibria for large validator populations** — + coupled HJB (individual validator optimization) + Fokker-Planck (population density): + $-\partial_t u + H(x, \nabla u) = F(x, m)$, $\partial_t m - \nabla \cdot (m \nabla_p H) = 0$ + — captures emergent staking coordination without enumerating every validator; predict slashing + risk, validator set stability, and staking yield across **six concurrent time horizons** at + ≥ 360:1 speed; produce bounded predictions on consensus health and staking returns. - **Does-not:** validate blocks (this is a simulator); model AMM pools (M3c); model token supply (M3e — but consumes staking ratio from M3e as input). @@ -34,8 +39,18 @@ computation. Python, Julia, or R. equilibrium. Example: `{ value: 4.2, lower_bound: 3.6, upper_bound: 5.1, confidence: 0.82, time_horizon: "30d", sim_type: "consensus_staking" }` — annualized staking yield (%). +- **Time-horizon mapping** (all run concurrently, ≥ 360:1 speed): + | Horizon | Primary models | Update cadence | + |---------|---------------|----------------| + | Hourly | Markov chain validator state transitions | Every epoch | + | Daily | Replicator dynamics strategy shifts, slashing events | Hourly roll | + | Weekly | Staking pool Nash equilibrium recalculation | Daily roll | + | Monthly | MFG equilibrium for validator population | Weekly roll | + | Annual | Evolutionary stable strategies, yield trajectory | Monthly roll | + | 5-year | Consensus mechanism structural evolution | Quarterly roll | - **Prediction types:** `validator_honesty_fraction`, `slashing_probability`, `staking_yield`, - `pool_delegation_equilibrium`, `throughput_stability`, `consensus_liveness`. + `pool_delegation_equilibrium`, `throughput_stability`, `consensus_liveness`, + `mfg_validator_equilibrium`. - Calibration: ingests `on_chain_event` (validator set changes, slashing events) from M2. ## 6. Dependencies & stubs @@ -56,8 +71,9 @@ computation. Python, Julia, or R. 1. Implement the evolutionary honesty game (replicator dynamics, bounded rationality). 2. Implement the Markov chain validator-state model. 3. Reproduce the staking pool Nash equilibrium reward threshold from [10]. -4. Wire M2 validator data → calibration of transition rates. -5. Wire M3e staking ratio input. +4. Implement MFG solver (HJB + Fokker-Planck) for large validator populations. +5. Wire M2 validator data → calibration of transition rates. +6. Wire M3e staking ratio input. ## 9. Tests Equilibrium: honesty fraction converges to Nash equilibrium under stable payoffs. Markov: diff --git a/core/docs/plans/M3g-market-microstructure-sims.md b/core/docs/plans/M3g-market-microstructure-sims.md index 18bc84a..4d82e17 100644 --- a/core/docs/plans/M3g-market-microstructure-sims.md +++ b/core/docs/plans/M3g-market-microstructure-sims.md @@ -2,69 +2,100 @@ ## 1. Component Market microstructure simulation: models **order flow, liquidity depth, slippage, spread dynamics, -and cross-exchange arbitrage** at the fastest time scales (tick-level to hourly). Pops here are -**market makers, takers, and arbitrageurs** interacting across multiple venues. The sim that -operates at the highest temporal resolution — where M3a provides statistical forecasts and M3c -models pool mechanics, M3g models the *plumbing* of how orders actually execute. +optimal execution, and cross-exchange arbitrage** at the fastest time scales. Pops here are +**market makers, takers, and arbitrageurs** interacting across multiple venues. Includes the +**Almgren-Chriss optimal execution framework** for minimizing market impact of large orders. +Operates at the highest temporal resolution — where M3a provides statistical forecasts and M3c +models pool mechanics, M3g models the *plumbing* of how orders actually execute, across **six +concurrent time horizons** at ≥ 360:1 speed. ## 2. Status / certainty -DESIGN-FIRST · ABSENT. Order-book microstructure theory C4 (established academic field); -DEX-specific microstructure C2 (emerging). Implementation C1. +DESIGN-FIRST · ABSENT. Order-book microstructure theory C4 (established). Almgren-Chriss +optimal execution C5 (industry standard since 2001; crypto adaptations validated 2023–2024, +Kurz CMC thesis). DEX-specific microstructure C2 (emerging). Implementation C1. ## 3. Language & location -TBD · `src/economy/sims/microstructure/`. Needs high-frequency data handling and event-driven -simulation. Rust, C++, or Python with optimized event loop. +TBD · `src/economy/sims/microstructure/`. Needs high-frequency data handling, event-driven +simulation, and Riccati equation solvers for optimal execution trajectories. Rust, C++, or +Python with optimized event loop. ## 4. Does / does-not - **Does:** simulate order flow across venues (DEXs and CEXs); model bid-ask spread dynamics as a function of inventory risk and adverse selection; simulate slippage curves for various order - sizes; model cross-exchange arbitrage opportunities and their decay; operate at **tick-level - resolution** (sub-second to minute); produce bounded predictions on execution quality, optimal - routing, and liquidity conditions. + sizes; solve **Almgren-Chriss optimal execution**: + $\min \int_0^T [\lambda \cdot x(t) \cdot \dot{x}(t) + \eta \cdot \dot{x}(t)^2] \, dt$ + where $\lambda$ = permanent impact, $\eta$ = temporary impact, $x(t)$ = remaining order — + splits large orders across time to minimize market impact + timing risk; impact parameters + $\lambda, \eta$ re-estimated from order-flow streams every 10–30s; execution trajectory solved + via Riccati equations in <100ms; model cross-exchange arbitrage opportunities and their decay; + operate at **tick-level resolution** (sub-second to minute); produce bounded predictions on + execution quality, optimal routing, and liquidity conditions. - **Does-not:** model protocol consensus (M3f); model macro token supply (M3e); model social behavior (M3b); execute trades (Marketplace does). ## 5. Interface contract - Implements `query(PredictionQuery) -> BoundedPrediction` per M3 hub. -- **Output bounds:** execution cost ranges and liquidity intervals. - Example: `{ value: 0.0034, lower_bound: 0.0018, upper_bound: 0.0052, confidence: 0.85, - time_horizon: "next_trade", sim_type: "market_microstructure" }` — expected slippage (%) for a - 10 ETH market sell. - Example: `{ value: 12400, lower_bound: 8200, upper_bound: 18600, confidence: 0.78, - time_horizon: "1h", sim_type: "market_microstructure" }` — available depth (USD) within 50bps - of mid. +- **Output bounds:** execution cost ranges, liquidity intervals, optimal trajectory envelopes. +- **Time-horizon mapping** (all run concurrently, ≥ 360:1 speed): + | Horizon | Primary models | Update cadence | + |---------|---------------|----------------| + | Tick–hourly | Almgren-Chriss execution, slippage, spread, arb decay | Every tick | + | Daily | Liquidity regime, venue depth profiles | Hourly roll | + | Weekly | Cross-venue flow patterns, impact parameter drift | Daily roll | + | Monthly | Structural liquidity shifts, venue market share | Weekly roll | + | Annual | Microstructure regime (DEX vs CEX share evolution) | Monthly roll | + | 5-year | Venue topology evolution, structural impact trends | Quarterly roll | +- Examples: + `{ value: 0.0034, lower_bound: 0.0018, upper_bound: 0.0052, confidence: 0.85, + time_horizon: "next_trade", sim_type: "market_microstructure" }` — slippage (%) for 10 ETH. + `{ value: 12400, lower_bound: 8200, upper_bound: 18600, confidence: 0.78, + time_horizon: "1h", sim_type: "market_microstructure" }` — depth (USD) within 50bps. + `{ value: [0.3, 0.3, 0.2, 0.1, 0.1], lower_bound: null, upper_bound: null, confidence: 0.80, + time_horizon: "30min", sim_type: "market_microstructure" }` — Almgren-Chriss optimal execution + schedule (fraction per 6-min bucket for 100 ETH sell). - **Prediction types:** `slippage_estimate`, `spread_forecast`, `depth_profile`, - `cross_venue_arb`, `optimal_execution_route`, `liquidity_score`. + `cross_venue_arb`, `optimal_execution_schedule`, `optimal_execution_route`, + `liquidity_score`, `impact_estimate`. - Calibration: ingests `price_tick`, `dex_pool_state`, and `execution_fill` from M2. ## 6. Dependencies & stubs - M2 Data Feeds — tick data and pool state; *stub:* canned order book snapshots. +- M3a Statistical — volatility estimates for Almgren-Chriss timing risk; *stub:* fixed vol. - M3c AMM sims — pool mechanics for DEX venues; *stub:* fixed pool state. - M3 Sims hub — lifecycle management; *stub:* manual init. ## 7. Invariants / laws -- **L1 (C4):** microstructure operates at the **highest temporal resolution** — predictions are +- **L1 (C5):** microstructure operates at the **highest temporal resolution** — predictions are valid for seconds to hours, not days. Stale microstructure data is worse than no data. -- **L2 (C4):** slippage is a **function of order size and current depth** — not a fixed +- **L2 (C5):** slippage is a **function of order size and current depth** — not a fixed percentage. The sim must model the non-linear relationship. -- **L3 (C3):** cross-venue arbitrage opportunities **decay** — the sim models the time-to-close +- **L3 (C5):** Almgren-Chriss impact parameters $\lambda, \eta$ are **estimated from live data**, + never hardcoded — crypto impact dynamics differ by asset, venue, and time-of-day. +- **L4 (C4):** cross-venue arbitrage opportunities **decay** — the sim models the time-to-close of an arb opportunity, not just its existence. +- **L5 (C4):** optimal execution trajectories are **re-solved on every significant state change** + (vol spike, depth drop, regime transition) — a stale trajectory is worse than naive execution. ## 8. Build steps 1. Implement a simplified order-book simulator (limit orders, market orders, cancels). 2. Add spread dynamics (inventory-based market maker model). 3. Add slippage curves (order size → execution cost). -4. Add cross-venue arb detection and decay modeling. -5. Wire M2 tick data → calibration. +4. Implement Almgren-Chriss optimal execution (Riccati solver, impact estimation). +5. Add cross-venue arb detection and decay modeling. +6. Wire M2 tick data → calibration of impact parameters. +7. Wire M3a vol estimates → Almgren-Chriss timing risk component. ## 9. Tests Slippage: larger orders produce greater slippage. Spread: spread widens under adverse selection. -Arb decay: detected arb opportunity closes over time. Depth: depth profile matches order book -state. Bounds: all outputs bounded. Resolution: predictions update at tick frequency. +Almgren-Chriss: optimal trajectory minimizes total cost vs. naive execution on backtest; impact +parameters update when market conditions change. Arb decay: detected arb opportunity closes over +time. Depth: depth profile matches order book state. Bounds: all outputs bounded. Resolution: +predictions update at tick frequency. Speed: sim advances ≥ 360:1. ## 10. Open items - CEX order book data access (API limitations, costs). - DEX-specific microstructure (AMM pools don't have order books — translate pool state to equivalent depth/spread). - Latency modeling (how fast can our traders actually reach an arb?). +- Almgren-Chriss extensions for crypto (volume-dependent variant? discrete block propagation?). - Which venues to model initially.