The Frankenstein, Part 3: One Part In, One Base Rejected

4 November 2026. A progress note from a running campaign, following Part 2. The measurements below are engine-level, and none of them is agent task time unless it says so. Not a peer-reviewed paper.

Part 3 of our Frankenstein: a DeepSeek-V4.1-Flash server on two RTX PRO 6000s, built from other engines’ best parts.

New part, served: a CPU/GPU expert plan built from our own agent traffic. Verify steps took 2.42% less time, lower in 5 of 5 randomized blocks (p = 1/32).

The target is unchanged: DeepSeek-V4.1-Flash-1017 on one workstation with two RTX PRO 6000 Blackwell Workstation Edition GPUs (SM120), for long-context agentic coding, one request at a time. The body is still vLLM with diffbot’s EXL3 3-bit pack. It now serves Part 1’s CPU-expert schedule and region cache, Part 2’s mHC overlap from vLLM-Moet, and, since 3 November, the expert plan below. Each part goes in only through a test whose decision rule we write before it runs.

Summary

• New part, served: a host-expert plan built from our own agent traffic. Time per verify step was 2.42% lower on average than with the published plan (new/published − 1 = −2.42%), lower in 5 of 5 randomized blocks (p = 1/32). That is step time during agent tasks, one request at a time. A task-time effect was not shown.
• Not adopted: a newer vLLM base. Each agent-task draw took 10.0% longer than on our pinned engine (newer/pinned − 1 = +10.0%), longer in 4 of 4 randomized blocks, both with the mHC overlap off.
• The mHC overlap, two more checks. Its path for larger batches stayed within the fidelity proxy’s tolerance at one request at a time. Our first randomized agent-task quality check of a served part, the overlap on against off in 16 blocks, was inconclusive at its planned size. It stays served, for one request at a time.
• Where the time goes now: 228.2 s per agent-task draw, 74% of it decode. So our first own part targets decode.

New part: a host-expert plan from our own agent traffic

DeepSeek-V4.1-Flash sends each token to a few of each layer’s routed experts. diffbot’s runtime keeps 5,296 of the 15,360 routed experts in the model’s 40 layers in CPU RAM and runs them on the CPU. The rest live on the GPUs, and a 512-slot GPU cache holds the most-used CPU experts. Which experts stay on the CPU is set by a file, the plan. diffbot’s published plan comes from the routing of long agentic coding sessions.

SilentMountain, one of our agents, built a new plan from routing recorded on four replays of our RT tasks, agent tasks we keep apart from W01, the benchmark that judged it. It keeps the same 5,296 experts on the CPU in total, so memory doesn’t change: 39 of the 40 layers trade some experts between CPU and GPU. The plan never saw W01’s routing, but the decision to test it did: a simulation on two W01 draws.

The test (registered before launch): a fidelity screen first, then five blocks, each with one engine start per plan in random order. Each start ran three draws of W01, our agent-task benchmark: six long-context coding and analysis tasks with real tool calls, graded mechanically. Time per verify step came from the server’s counters over each start’s draws. The rule: the new plan is confirmed only if its time per verify step is lower in all five blocks.

Block	1	2	3	4	5	Mean
Published plan, ms per verify step	14.704	14.920	14.736	14.806	14.735
New plan	14.346	14.456	14.411	14.474	14.419
New/published − 1	−2.43%	−3.11%	−2.20%	−2.24%	−2.14%	−2.42%

It was lower in all five blocks: confirmed, one-sided exact sign test p = 1/32. The draws are sampled, so the arms’ outputs differ: the new plan’s starts ran 1.75% more verify steps. That is why the registered proposal rested on time per output token, lower on average (new/published − 1 = −2.33%) and in 5 of 5 blocks, not on step time alone; the adoption was our campaign manager’s call.

What this does and does not show:

• Fidelity first. Moving an expert between the GPU and the CPU changes its arithmetic. On 26 reference items and two forced-decode traces, the new plan passed its registered screen. Its KL differences against the published plan were +0.00122, +0.00078 and −0.00003 on the three item groups (limit +0.010), and its top-1 agreement moved by −0.23, −0.04 and +0.21 points (limit −0.5). That is a screen with one start per plan, not a quality-equivalence claim.
• Quality held. 74 of 90 episodes passed in each arm. Equal totals are not proof of equal quality.
• Task time: no claim. Per block, the new plan’s draws ran from 9.4 s slower to 26.3 s faster, 3.2 s faster on average. That range spans zero, so we report it and claim nothing.
• Scope: one request at a time, on W01’s six tasks, with the schedule, region cache and mHC overlap on. Our campaign manager adopted it on that basis on 3 November, with a one-step revert. It has been served since 07:44Z.

Not adopted: a newer vLLM base

Our engine runs on a pinned vLLM build. SageSummit built the newest base we could and tested it as a whole:

• vLLM 783380af, 298 commits newer;
• an open vLLM pull request that brings DeepSeek-V4.1’s 528-byte KV-cache record per token to SM120, in place of the 584-byte record we run;
• torch 2.14 and FlashInfer 0.7.3rc1.

With the smaller record, the KV cache held 8.2% more tokens: 1,159,335 against 1,071,558.

The test (registered before launch): four blocks, each with two engine starts per arm in random order, three W01 draws per start, the mHC overlap off in both arms. The rule: faster or slower only if all four blocks agree.

Block	1	2	3	4	All
Extra time per draw on the newer base, s	33.4	16.2	7.8	38.8	24.1
Newer/pinned − 1	+14.5%	+6.9%	+3.1%	+16.2%	+10.0%

It was slower in all four blocks, the registered don’t-adopt outcome: 263.5 s against 239.5 s per draw, over 24 draws per arm. Quality held: 120 of 144 episodes passed on the newer base, 117 of 144 on the pinned engine.

Where the time went, from the server’s counters (descriptive, not a causal split): the newer base generated 7.7% more tokens, and its decode time per generated token was 4.4% higher, 4.316 against 4.133 ms.

What this does and does not show:

• The whole stack, together. The 298 commits, torch, FlashInfer and the new record went in as one. This doesn’t show that the pull request is slow. A control arm meant to separate the record path didn’t run: it forced the old record onto the new code through a hook of ours, a combination the code doesn’t support. That is our setup, not an upstream bug.
• Nothing about the overlap on the newer base: the overlap was off in both arms.
• The larger KV cache was recorded, not tested with more concurrent requests.

The mHC overlap: two more checks

Its path for larger batches. Part 2 adopted the overlap for one request at a time only. With five or more concurrent requests, verify steps run in larger CUDA graphs, and the fidelity check hadn’t covered those. BrightPeak padded single-request steps to the 48-row graph and scored them with Part 2’s top-64 logit proxy over 767 decode positions, with tolerance 0.025:

• the overlap against the served engine: 0.0181–0.0210;
• two starts of the served engine against each other: 0.0200;
• two overlap starts against each other: 0.0223 and 0.0248. That is the largest pair, 0.0002 below the tolerance.

All pairs were within tolerance, which is not an equivalence claim. Real batches of five to eight requests weren’t scored, so the adoption stays single-stream.

Our first randomized quality check of a served part. After the overlap went in, every failed episode in our served W01 runs was in one of the two QA tasks: 15 of their 30 QA episodes failed, against 8 of 28 before. Those were different occasions, so GentleSparrow, our research lead, ran an A/B:

• 16 blocks, each with one engine start with the overlap and one without, in random order;
• three W01 draws per start, 96 in all, everything else as served;
• the measure: QA passes per start, out of 6 (two tasks, three draws);
• the registered action: switch the overlap off if it passes fewer, at one-sided p ≤ 0.05.

The mean difference was −0.19 QA passes per start (with minus without). The one-sided p was 0.358 over 65,536 within-block swaps: inconclusive at the planned size, not equivalence. QA episodes passed 48 of 96 times with the overlap and 51 of 96 without. Over all six tasks, the counts were 240 and 243 of 288. Task time per draw was 230.5 s with the overlap and 232.4 s without (with/without − 1 = −0.81%): descriptive, with no claim. The overlap stays served.

What this does and does not show: under assumptions we can’t yet check, the design’s planning estimate was that a loss of about one QA pass per start (1.0 to 1.1) would show with 80% power at 16 blocks. The actual paired variance is unknown, so that is not a bound, and a smaller loss can’t be ruled out.

Where a draw’s time goes now

SageSummit split 51 served W01 draws from 3 November, on the current stack, one request at a time. These are served runs on one day, not an A/B.

Part of a W01 draw	s per draw	Share
Decode	169.3	74.2%
Cold first turns, time to first token (4 requests)	22.8	10.0%
Tool calls	13.1	5.7%
Warm time to first token (43 requests)	12.71	5.6%
Outside requests and tools, including 2.95 s between turns	10.4	4.6%
Episode time	228.2

Decode is 44,908 tokens per draw at 3.77 ms each. At 3.76 tokens per verify step (3.759, measured on today’s stack in the plan test’s new-plan arm), that is about 11,900 steps. So each millisecond off a verify step is worth about 11.9 s per draw: a scenario at fixed step counts, not task time.

Next: our first own part

The Frankenstein’s second stage replaces parts with our own, and the first one targets decode. DeepSeek-V4.1’s sparse attention has an indexer, and a top-k kernel keeps the best 512 entries of each indexer row. The served kernel gives each row one 1,024-thread block. At one request at a time a verify step has six rows, so each call uses 6 of the GPU’s 188 SMs. SilentMountain wrote a replacement that spreads each row over a cluster of up to eight blocks. Its output must match the served kernel’s bit for bit. So far it is design, CPU tests and review; it has no GPU numbers yet.

Credits

• diffbot for the EXL3 3-bit pack and its serving recipe, including the CPU-expert runtime and its expert plans, one of which we replaced with a plan built from our own traffic.
• turboderp for the EXL3 format.
• kacper-daftcode for vLLM-Moet’s SM120 mHC overlap and its kernels (commit ec8a1e05e1cb, Apache-2.0), still served.
• DeepSeek for DeepSeek-V4.1-Flash-1017.
• vLLM, PyTorch and FlashInfer for the engine and the newer base we measured.

AI assistance

AI agents (Claude and GPT models) designed and ran these tests. An independent AI reviewer checked each design before launch and audited each result afterwards, and an AI agent drafted this post. Artus set the goal, authorized the hardware and asked for these progress notes.