Engine
How synthCore turns a mess of venue streams into one consistent, inspectable view — and why it runs on almost nothing.
The pipeline
Every venue speaks its own dialect. An adapter per source translates incoming trades, quotes, and selected on-chain swaps into one normalized event: source, pair, time, price, quantity, instrument kind. Downstream, nothing knows or cares which exchange a message came from — it's all the same shape.
venue stream -> adapter -> normalized event -> candle engine -> SQLite + cold archive -> public JSON
Normalized events feed the candle engine and a crash-safe cold archive in parallel. The live view is served as static JSON snapshots; the raw archive and database never leave the box.
How a synthetic candle is built
A synthCore candle is not copied from one exchange. It is built from the normalized events observed across configured sources in a time bucket: open, high, low, close, VWAP, volume, trade count, contributing sources, and a rough quality score, all derived from what actually arrived.
Quiet markets look jumpy because the engine only knows about trades it saw. For readability, public snapshots may forward-fill silent minutes with flat bars. Those filled bars are display helpers; stored data keeps only real observed trades. The database stays pure; the gap-fill happens on read.
Feeds, and what stays distinct
The live target fuses CEX feeds — Binance, Bybit, OKX, Kraken, Coinbase — with selected Uniswap V3 pools. Related markets stay separate when their mechanics differ: ETH/BTC on an exchange and WETH/WBTC in an on-chain pool are not the same instrument and are never merged.
Feeds are expected to fail, go quiet, or drift. A missing source lowers confidence for its bucket; it never stops collection from the rest. The collector backfills gaps from venue REST on startup and repairs torn frames after a crash, so the archive heals itself rather than accumulating holes.
Why Rust, on a small box
The whole point is to do a lot, fast, on minimal hardware. Live multi-venue ingest, normalization, candle construction, a self-healing archive, and a paper-bot leaderboard all run as three systemd services under one unprivileged user on a 2-vCPU VM — and the machine is bored. Measured numbers, not estimates:
| piece | cpu | memory |
|---|---|---|
| Collector / candle engine | ~16% of one core, steady | ~20 MB |
| Leaderboard runner | ~1.4% | ~50 MB |
| Per-pair bot view | ~1.4% | ~36 MB |
| Whole stack | under 10% of the box | ~106 MB |
Load average sits at 0.04 across a 19-day uptime with 4.5 GiB of RAM free. Rust buys the headroom: no GC pauses on the ingest path, predictable memory, and a single static binary per service. That headroom is the feature — it leaves room to add pairs, venues, and timeframes without renting a bigger machine.
Where it comes from
synthCore is the infrastructure end of a much longer effort. The research behind it began years ago on TradingView — a disciplined, coherent family of indicators grown from a single thesis about market structure, refined release over release and published in part under my own name. That suite was ported into code faithfully — the same math, the same semantics — and then crossed the line from indicator to live system: the paper bots ranked across these pages are its running edge.
The specifics stay where they earn their keep. What's documented here is the shape, not the recipe.
Back to synthCore docs · next: Strategies.