investment-analyst: the simulated-ETF quantitative line that runs every day

What this line actually is

etf-rebalance v3.0.0 maintains RWHL, a simulated ETF of humanoid-robotics leaders (the page carries a built-in disclaimer; it must never be described externally as a real fund). It is the only quantitative line on this site that runs every day with results readers can see, and its executor in the digital-twin office is investment-analyst. The daily SOP is a fixed pipeline: qualitative overlay (human before machine) → incremental quote sync → redraw synthetic candles → backtest five strategies → compute today's rebalance → append to rebalance history → POST /api/etf/portfolio.

The whole thing is pure standard library, run from the repo root, with no pandas or numpy. That is not showing off: production has 3GB of RAM, the backtest rolls a 195-day window day by day, and thinner dependencies mean fewer things that can blow up.

First architectural principle: strategy and execution are strictly separate

Module	The only question it answers	What it must not do
scripts/etf/data_feed.py	Where candles and snapshots come from, where they are cached	Contains no weight logic
scripts/etf/strategy.py	"What are the target weights?"	Does not care about fillability or limit-up/down
scripts/etf/execution.py	Closing-auction fills, limit gates, blocked-order redistribution, costs	Does not alter the intent of target weights
scripts/etf/backtest.py	Rolling-window covariance, day-by-day solve, NAV synthesis	Never sees the future
scripts/etf/report.py	The rebalance report markdown	Pure function, now() banned: identical input must yield identical bytes

Adding a strategy means subclassing strategy.BaseStrategy, implementing target_weights() and registering it in STRATEGIES; it then joins the backtest comparison and the frontend charts automatically. Execution rules (closing auction, limit gates, costs) are uniform across strategies, and a strategy must never bypass the execution layer — bypassing means the backtest numbers stop being comparable.

The quantitative method (do not change the definitions)

The risk model uses 120 trading days of log returns, with the sample covariance analytically shrunk toward a scaled identity per Ledoit & Wolf (2004) at a data-driven intensity δ*. Weights solve the generalised ERC system w_i·(Σw)_i ∝ b_i (Maillard-Roncalli-Teïletche 2010; unique solution via the convex programme of Spinu 2013), where b is the importance score as risk-budget prior; equal b degenerates to classic risk parity. Single names are clamped to [2%, 12%].

The flagship size-tilt multiplies the risk-budget prior by a size factor mult_i = clip(1 + λ·log2(cap_i / CNY30bn), 0.5, 2.0) with λ=0.25. The size proxy is forward-adjusted close × current share count, and every rebalance date uses only information available up to that date. It carries two hard constraints: constituents with a size proxy ≤CNY20bn get budget zero (the ERC system is solved on the investable subset, falling back to the full basket when fewer than three names remain investable), and the CNY20–30bn band is capped at ≤3% per name (SIZE_BAND_CAP). Caps travel down with diag["caps"] per constituent and blocked-order redistribution must respect them; when the caps sum to less than one, the remainder stays in cash.

Five strategies are registered: equal-weight (the 1/n benchmark), risk-budget, size-tilt (flagship), min-variance and momentum-tilt (3-month cross-sectional momentum z-score multiplied into the budget, clipped to [0.25,1.75]). The benchmark is CSI 300 (000300) normalised to the same base date; per strategy we compute geometric excess return, annualised tracking error, information ratio and win rate versus the benchmark.

A-share microstructure lives in the execution layer

The backtest does not pretend fills are always available: every rebalance order fills at the rebalance date's closing auction, that day's return still accrues on the old weights, and new weights take effect the next trading day. Limit gates are per board — ±20% for ChiNext/STAR, ±10% for the main board. A name sealed at limit-down (close = low = limit price) cannot be sold and keeps its old weight; sealed at limit-up it cannot be bought. The gap left by blocked names is redistributed along the target direction to tradable constituents while respecting bounds, and any capacity shortfall is disclosed as residual cash exposure in the report. Costs are charged at 10bp one-way times turnover, deducted from NAV. Constituents whose target weight is zero (size-excluded) do not participate in redistribution and are never passively bought back.

Settlement cutoff: running before 15:05 is normal

Because the strategy assumes closing-auction fills, it consumes only settled sessions. data_feed.settled_cutoff() uses Asia/Shanghai 15:05: run the SOP earlier and today's bar is merely an intraday stub (close = instantaneous price, volume accumulated so far), which drop_unsettled() discards, so the rebalance falls back to the previous session's close. Hence fund.window_end is "the last settled session", not "today", and running intraday is not a bug. The constituent table's price/change_pct/mktcap_yi/pe_ttm are aligned to the settled close as well, so the page never shows intraday prices next to yesterday's weights. --allow-intraday is for research only and ⛔ must not be used for rebalancing.

Performance: three short-circuits, daily run <2s

Before the 2026-09-07 refactor the slowness came from one serial HTTP request per constituent, a full benchmark re-pull every time, and an unconditional backtest-plus-write. The current shape:

python3 scripts/sync_etf_quotes.py           # daily <2s (idempotent short-circuit), 10-20s when new data lands
python3 scripts/sync_etf_quotes.py --init --days 420   # first run / data repair

A single qt.gtimg.cn batch snapshot first proves whether new data exists (measured 0.4s). Then an input fingerprint (candles + benchmark + share count quantised to 0.01bn shares + code version) is compared against the last artefact in state.json; if it matches, the run prints [skip] and exits — no candle requests, no backtest, no rewrite; a repeated weekend run measured 0.5s. Only when data is stale does it fetch incrementally with only=stale, and the benchmark syncs from an anchor with OVERLAP_BARS overlap validation, self-healing into a full re-pull when drift exceeds bounds. A SourceBreaker retires a source after two consecutive connection-level failures; ⛔ once tripped, do not loop-probe the same source.

Two scars worth copying

Scar one: volume units differ by board. Turnover is uniformly in CNY, but daily-candle volume units change by board — main board and ChiNext report in lots (×100 → shares) while STAR 688/689 reports directly in shares. Both incidents landed on this line: the early Tencent branch did not convert, so one constituent's entire series was 100× too small; after hard-coding ×100, five STAR names came out 100× too large (one day's turnover was written as CNY722.7bn, more than its own float). The fix is to infer the factor from an independent turnover figure rather than memorise a board table: the Eastmoney primary source back-solves amount / (volume × typical price) and snaps to {1, 100}; the Tencent fallback gives no turnover on daily candles, so the SOP pulls the realtime snapshot first and calibrates per constituent with calibrate_volume_factors().

Both sentinels are needed: amount_unit_ok() requires the implied average price to fall inside the day's [low, high] band (catches asymmetric volume/turnover errors), and amount_drift() with OVERLAP_BARS=6 compares cached versus freshly pulled turnover on overlapping days, forcing a full re-pull when the ratio is out of bounds — the only probe that can catch "the whole series multiplied by a constant". The five STAR names were flagged dirty at ~100x by exactly that check and re-pulled.

Scar two: the publish path. Portfolio data originally shipped as /static/etf/portfolio.json, which the Tencent CDN cached at the edge with a cache key that ignores query strings (the ?v= bucket was useless), so the live site served stale data for up to 24h after each daily rebalance. Since 2026-08-31 it writes a single-row Postgres snapshot in etf_snapshots via POST /api/etf/portfolio (X-API-Key), and the frontend reads GET /api/etf/portfolio; /api/* is globally no-store, so a write is live with no cache-busting. That lesson became a site-wide rule: static paths carry media only, data always goes through the API.

Where the human sits: the qualitative overlay

The quantitative model consumes only score (a 0–10 importance prior) and price/volume data. Event-driven score changes are made by a person, and the rationale must appear in the report — unjustified score edits are forbidden: industry progress (mass-production milestones, design wins from Tesla Optimus / Figure / domestic OEMs, breakthroughs in reducers, ball screws or dexterous hands, industrial policy) → +1~+2 for the direct beneficiary; earnings beats +1~+2, misses / goodwill impairment / insider selling or pledging / regulatory investigation −1~−3, and two consecutive quarters falsifying the thesis means removal from the basket. Limit-up/down seals normally do not change scores (the execution layer defers automatically); only fundamental deterioration warrants −1~−2. This overlay is the single stage in the line that is never automated.

Quote-source etiquette

Public, unauthenticated endpoints: Eastmoney push2his as primary (forward-adjusted daily candles), Tencent ifzq.gtimg as automatic fallback, and the qt.gtimg.cn batch endpoint for realtime snapshots. Requests are spaced ≥0.4s apart, retried three times with linear backoff, and concurrent hammering is banned. Eastmoney blocks IPs for high-frequency probing (observed RemoteDisconnected, cooldown over ten minutes), so debugging must never loop across many symbols — a normal SOP makes about 20 requests and is fine, a dozen manual back-to-back probes gets you blocked. Being blocked is worse than slow: every constituent degrades to the Tencent fallback, turnover becomes a typical-price estimate, and the candle volume bars stop reflecting real turnover.