The Frankenstein, Part 2: A Part from Another Fork

3 November 2026. A progress note from a running campaign, following Part 1. The measurements below are engine-level, and none of them is agent task time unless it says so. Not a peer-reviewed paper.

Part 2 of our Frankenstein: a DeepSeek-V4.1-Flash server on two RTX PRO 6000s, built from other engines’ best parts.

New part, served: an SM120 mHC overlap from kacper-daftcode’s vLLM-Moet. Verify steps took 4.88% less time, lower in 5 of 5 randomized blocks.

The target is unchanged: DeepSeek-V4.1-Flash-1017 on one workstation with two RTX PRO 6000 Blackwell Workstation Edition GPUs (SM120), for long-context agentic coding, one request at a time. The body is still vLLM with diffbot’s EXL3 3-bit pack. Each new part goes in only through a test whose decision rule we write before it runs.

Summary

• New part, served: an SM120 mHC overlap from kacper-daftcode’s vLLM-Moet. Time per verify step was 4.88% lower than on the served engine (overlap/served − 1 = −4.88%), lower in 5 of 5 randomized blocks (p = 1/32). That is step time with one request at a time, not task time. A verify-path fidelity proxy was within its registered tolerance first. It is served for single-stream use: adopted for one request at a time; concurrency 5 and above is not yet fidelity-checked.
• Schedule 2 stays. Part 1’s CPU-expert schedule finished its randomized comparison on our agent-task benchmark: 151 of 180 episodes passed in each arm, a mean difference of 0.0 passes per engine start. No pass-rate harm was shown.
• The prompt region cache is served. On the first agent-task runs with it, the client’s token-count call took a median of 0.052–0.054 s per turn, against 0.212 s on earlier runs. Those are different occasions, not a controlled comparison.
• Four things that didn’t work, at the end.

New part: the mHC overlap, from vLLM-Moet

DeepSeek-V4.1 keeps four residual streams instead of one, and mixes them at every layer boundary (its mHC, a form of hyper-connections). The mixing is small, but it sits on the critical path: in our profile of the served engine, about 78 boundaries per verify step and 1.05 ms of a 15.62 ms step.

vLLM can already compute part of the next boundary’s mixing on a second CUDA stream while the current layer finishes, but only on SM100 GPUs. kacper-daftcode’s vLLM-Moet (Apache-2.0) extends it to SM120 with its own kernels: a post-norm kernel on the critical path and the projection on the side stream. We read its source, screened it on one GPU, and then tested it in our full engine against what we serve.

The test (registered before launch): five blocks, each with one engine start per arm in random order. Each start replays 189 recorded turns from two agent tasks, one request at a time, timed by the server. The rule: the overlap is confirmed only if its time per verify step is lower in all five blocks.

Block	1	2	3	4	5	Mean
Served engine, ms per verify step	14.359	14.348	14.444	14.407	14.360
With the overlap	13.684	13.685	13.665	13.666	13.706
Overlap/served − 1	−4.70%	−4.62%	−5.39%	−5.14%	−4.55%	−4.88%

It was lower in all five blocks: confirmed, one-sided exact sign test p = 1/32. Decode time per output token was also lower in all five blocks, by 4.28% on average (overlap/served − 1 = −4.28%; reported, not tested). Tokens per step changed by −0.59% on average, in both directions.

Before serving it: fidelity and memory. A separate run checked the verify path at one request at a time, in forced decode with the draft model on. The measure was a top-64 logit proxy: the mean KL per position over the top 64 log-probabilities, across 767 decode positions, against a tolerance registered before the run. It was within that tolerance. That is not a claim that quality is equal.

• two starts of the served engine differed by 0.0173–0.0177;
• the overlap against the served engine: 0.0184–0.0202;
• the tolerance: 0.025, with fewer than 10 positions above 0.5 nats (at most 5 occurred).

GPU memory at a 1M-token test was 8 MiB per GPU higher. On that basis our campaign manager adopted it for single-stream serving (one request at a time) on 2 November, with a one-step revert. Five agent-task runs since, three draws each, have finished without an engine or episode error.

What this does and does not show:

• Step time, not task time. The replay is one request at a time, and a verify step is one pass of the target model over the draft tokens. How much of an agent task’s time that is depends on the task. We haven’t published a task-time figure for it.
• Single-stream only. Adopted for one request at a time; concurrency 5 and above is not yet fidelity-checked. From five concurrent requests, the side stream switches to a different projection.
• The fidelity check is a proxy on the verify path, not a quality-equivalence claim.
• Larger than predicted. The overlap saved 0.70 ms per step, 1.69 times what our one-GPU screen predicted (0.42 ms). This run doesn’t say where the extra time comes from.
• Power: both arms ran under the workstation’s standard cap (500 W per GPU while the CPU is busy, 600 W otherwise). Sampled power draw stayed below 500 W in every start.

Schedule 2 stays: the agent-task comparison

Part 1 served diffbot’s second CPU-expert schedule on a step-time result and said a randomized comparison on our agent-task benchmark was running. W01 is six long-context coding and analysis tasks with real tool calls, graded mechanically; one draw runs all six.

The design (registered before launch): ten blocks, each with one engine start per schedule in random order, three W01 draws per start. Every episode counts, failures included. The rule: go back to schedule 1 if schedule 2 passes fewer, at one-sided p ≤ 0.05.

Block	1	2	3	4	5	6	7	8	9	10	Mean
Passes per start, schedule 2 minus 1 (of 18)	0	−3	+1	0	−1	+1	0	+2	+2	−2	0.0

Both arms passed 151 of 180 episodes. The exact permutation test over all 1,024 assignments gives one-sided p = 0.578, so the rule did not fire, and schedule 2 stays served.

What this does and does not show:

• No harm shown is not equal quality. No pass difference was detected at this size.
• Not a speed result. This comparison was built to detect pass-rate harm, so we don’t report its task times as an effect of the schedule.
• Measured without the region cache, which was served only after it ended.

The region cache is served

Part 1’s prompt region cache caches token IDs for the long text region that repeats in every agent request. It was served on 2 November at 05:13Z, after the schedule comparison ended, as planned. Its adoption rested on its registered result in Part 1: 5.57 s less server time per W01 draw outside queueing and prefill (p = 1/70). The numbers below played no part in it.

Before each turn, our W01 client asks the server how many tokens the prompt has. On 60 earlier W01 draws, all before the cache, that call took a median of 0.212 s per turn, about 9.0 s per draw. On the first four draws with the cache served, the median was 0.052–0.054 s per turn; on the first three of them, the call took 2.24–3.47 s per draw.

These are different occasions, not a controlled comparison. That the cache caused the drop is our inference.

What didn’t work

• More GPU slots for CPU experts. The engine keeps the most-used CPU experts in a 512-slot GPU cache. With 768 slots, time per verify step was 1.34% lower than with 512 (lower in 5 of 5 blocks), but time per output token was only 0.80% lower on average, and lower in just 3 of 5 blocks. It also took 1,620 MiB more per GPU. Not adopted.
• Our own decode kernel for the GPU-resident experts. In a kernel test harness, no exact build reached the 0.079 ms per step saving we required; all five were slower than the existing kernel, by 0.019–0.144 ms per step. A variant that folds the codebook decode into the kernel was 0.062 ms slower. Stopped.
• A work-claiming split for the CPU-expert pool. On a CPU benchmark of recorded job shapes, it took 8.2–8.7% longer per job. Parked.
• Pool settings. None of the CPU-expert pool’s tunable settings was at least 5% faster than the served one in every block of a screen. No candidate.

Credits

• kacper-daftcode for vLLM-Moet’s SM120 mHC overlap and its kernels (commit ec8a1e05e1cb, Apache-2.0).
• DeepSeek for DeepSeek-V4.1-Flash-1017.
• diffbot for the EXL3 3-bit pack and its serving recipe, including the CPU-expert runtime with both schedules and the prompt segment encoder our region cache wraps.
• turboderp for the EXL3 format.
• vLLM for the engine, including the mHC overlap that vLLM-Moet extends to SM120.

AI assistance

AI agents (Claude and GPT models) designed and ran these tests. An independent AI reviewer checked each design before launch and audited each result afterwards, and an AI agent drafted this post. Artus set the goal, authorized the hardware and asked for these progress notes.