← 回到 infra 主页

DeepSeek Harness · Upstream · Issue #10 草案

进程才是并行与隔离的单位:同一 DSH 进程内所有会话与所有 subagent 共用一条 JS 主线程,主线程贴到 98.2% of one core,而整个进程只用掉 2.03 个核

先说结论:用户报「同一台机器上,2 个 dsh 网页各 2 个 session 明显比 1 个 dsh 网页 4 个 session 流畅」,在确认拓扑(每户一个独立 Node 进程)之后,根因不在网络、不在浏览器、也不在某一条慢请求,而在进程模型本身——Host 进程才是并行与隔离的单位,会话不是。所有 subagent 都由 dsh-subagent-in-process-driver 在父进程内创建(本随包只注册了两个 in-process 提供者,没有 child_process 系 subagent 驱动),因此一个 Host 里会话数 × 每会话 subagent 数个 agent 全排在同一条 JS 主线程上。最硬的一条证据不依赖任何速率估计:8 秒窗口内,K=1 完成 2 次 RPC、K=4 完成 4 次、K=8 在窗口内完成 0 次(另外 8 次是排空后才完成的)——提供的并发涨了 8 倍,窗口内吞吐没有涨,涨的全是排队。同刻那条不读任何会话状态的定开销探针样本数掉 51%、p99 变差 3.6×,主线程贴到 98.2% of one core,而进程总共只用掉 2.03 个核(宿主 256 核、load1 9.7)。切成 2 个 Host 就多拿到一条 JS 线程。本页给出零依赖复现探针、代码锚点(指到行)、已排除的原因、未定论项、建议修法与可直接粘贴的英文 issue 正文。

一句话判据(30 秒复现):对一个正在用的 Host 压 K 路并发 session/list,同时用 GET /api/ping(不读任何会话状态)做尾延迟探针——判词出现 main_pct_load ≥ 80 而 host_load1 ≪ host_cores,且 rpc_ok_kN 不随 K 增长,即为天花板:MAIN_THREAD_CEILING … main_pct_load=98.2 process_pct_load=202.6 host_cores=256 host_load1=9.72 rpc_ok_k1=2 rpc_ok_kN=0 ok_after_drain_kN=8 window_s=8 result=SATURATED。探针:node ops/dsh/upstream-issue-single-process-main-thread/main-thread-ceiling-probe.mjs --port 4002 --pid <host-pid>。
优先级:中高。①这不是性能调优问题而是容量模型问题:一台 Host 的并发 agent 需求一旦超过 1 核,多出来的部分只会变成排队,而 DSH 既没有拒绝也没有排队语义,用户只看到「卡」;②当前在监控上完全不可见——本仓与 DSH 都没有任何持续导出或被采集的 per-Host CPU / RSS / event-loop lag / GC 指标(本仓 dsh-perf-probe.py 只统计会话内直方图,从不读 /proc;本仓 DSH 工具链(ops/dsh + ops/monitoring/dsh)里唯一读 /proc 的 per-Host CPU 读数就是本页探针,它现读现打印、不落任何时序库),只能靠用户报;③与本专栏 #7/#8 同源:那条实测 9.7–11.9 秒的 session/list 正跑在这条主线程上,会话越多、subagent 越多,常驻负债越重。
本页经过一轮对抗性核验并因此改过数(2026-09-14)。核验发现探针自身有一个分子分母不同窗的缺陷(把排空后的完成数除以窗口长度),首版据此写出的「8× 并发换来 2.63× 吞吐」已撤回;同时撤掉「其余 255 核闲置」(实际进程用了 2.03 核)、把「线程数恒为 12」收窄为「不变核 11 条 + 随用随起的 WorkerThread」,并把「4 个会话一起等」从实测列移到推断列。现版本改以窗口内完成次数立论,不再依赖速率点估计。

一句话结论与三条事实

三条都是本次实测(2026-09-14 15:20,生产实例),命令与原始输出都在本页;源代码锚点按随包发布的 0.1.5-rc.1 lib/ 核对。

① 所有 subagent 都在父进程内跑,而本随包没有第二种选择

dsh-subagent-in-process-driver/lib/index.js:161 的 startInProcessRun() 直接调 parent.ctx.agents.create()(:180),把子代理交给同文件的 drivePublishedRun()(:195),由它 await child.whenIdle()(:211);dsh-subagent-spawn-in-process/lib/index.js:35 与 dsh-subagent-fork-in-process/lib/index.js:50 都只是它的薄壳,而这两个 in-process 提供者就是本随包唯一注册过的两个。dsh-subagent*/lib/**/*.js 里 child_process 命中 0——spawn/fork 有若干命中,但都是 provider 名与文档词,不是进程创建。所以「会话数 × 每会话 subagent 数」个 agent 共享同一条 JS 主线程与同一个 V8 堆。(注意口径:这里的「后端」指 subagent 子级后端。随包确有 8 个与 ACP / SDK / Codex / Claude Code 同名的包——dsh-acp / dsh-acp-app / dsh-sdk-app / dsh-sdk-jsonrpc-server / dsh-sdk-minimal / dsh-sdk-protocol / dsh-hooks-codex / dsh-hooks-claude-code——但它们都不是子级后端,没有一个注册 subagent 提供者;上游真正的 ACP / SDK / Codex / Claude Code 子级后端不在这个离线包里。本页只对随包 0.1.5-rc.1 下结论。)

② 主线程是天花板:它贴到 98.2%,而进程只用掉 2.03 个核

K=8 时 MainThread = 98.2% of one core,同刻整个进程 202.6%(= 2.03 个核)——多出来的部分是 4 条 V8Worker(GC/编译)与 4 条 libuv-worker(I/O),不是第二条 JS 执行线程。宿主 256 核、load1 = 9.7(约 4% 利用率)。线程数不是固定的:不变核是 11 条(MainThread + DelayedTaskSche + 4×V8Worker + SignalInspector + 4×libuv-worker),WorkerThread 随 workflow / run_code 起停,同一时刻三个实例量到 11 / 11 / 13;没有任何一条随会话数或 subagent 数增长。

③ 加并发只排队,不提吞吐——用窗口内完成次数立论

三个档各跑一个约 8 秒窗口,窗口内完成的 RPC 次数是 2 / 4 / 0(K=1 / 4 / 8);K=8 那 8 次全部在窗口之后才完成(ok_after_drain_kN=8)。按闭合循环(完成一次才发下一次)反推,单次 RPC 墙钟时间从 K=1 的约 4 秒升到 K=4 的 4–8 秒、K=8 的 > 8 秒——并发涨 8 倍,吞吐不涨,延迟全部吸收。同刻那条不读任何会话状态的定开销探针,K=1 → K=8 样本数 15268 → 7464(−51%)、p99 0.92 → 3.30 ms(3.6×)、p99.9 7.17 → 20.02 ms(2.8×),K=4 还测到一次 831.9 ms 的停顿。

先把「谁慢」说清楚。本页说的不是「DSH 单请求慢」,而是同一进程内并发度上去以后,无关请求也一起变慢。单请求在负载下仍然很快:定开销探针 p50 全程只有 0.43–0.90 ms,8 秒能打 7 千到 1.5 万次。缺陷在于这条线程没有第二次机会——它是进程内唯一的一条。

实测锚点(含原始输出)

环境:dsh 0.1.5-rc.1(随包 lib/),生产实例 dsh-user-d(pid 1017149,127.0.0.1:4002);宿主 256 逻辑核、load1 9.7;本户盘上 2996 个会话目录;Node v24.16.0。读数采集于 2026-09-14 15:20(+08:00),数字是活账,提交 issue 前请以当刻读数为准。

1. 剂量-响应主实验(本页的核心证据)

K = 同一进程内并发 session/list 数;探针 = GET /api/ping(定开销、不读会话状态);CPU 取自 /proc/<pid>/task/<tid>/stat,main% 只算 JS 主线程。rpc_ok 只数落在窗口内的完成(分子分母同窗),ok_after_drain 另计。

# DSH main-thread ceiling probe
# target        : http://127.0.0.1:4002   (pid 1017149)
# host cores    : 256   load1 9.71
# load          : K concurrent session/list RPCs
# probe         : GET /api/ping (fixed cost, reads no session state)
#
K	rpc_ok	rpc_per_s@window	canary_n	p50	p99	p99.9	max	>20ms	main%	other%	proc%
0	0	0.00@8.1s	14547	0.44	1.13	9.37	92.9	3	69.3	5	74.3
1	2	0.25@8.1s	15268	0.43	0.92	7.17	272	6	87.9	50	137.9
4	4	0.50@8.0s	9754	0.62	1.62	9.75	831.9	7	98.4	87.3	185.7
8	0	0.00@8.0s	7464	0.9	3.3	20.02	93	8	98.2	104.4	202.6
#
MAIN_THREAD_CEILING main_pct_idle=69.3 main_pct_load=98.2 process_pct_load=202.6 host_cores=256 host_load1=9.72 rpc_per_s_k1=0.25 rpc_per_s_kN=0 window_s=8 rpc_ok_k1=2 rpc_ok_kN=0 ok_after_drain_kN=8 speedup=n/a speedup_note=n/a_because_in_window_count_is_0 canary_p999_idle_ms=9.37 canary_p999_load_ms=20.02 canary_max_load_ms=93 result=SATURATED

读法:主线程贴到 98.2%,而进程只用了 2.03 个核、机器上还有 254 个核没被它用上;吞吐不看速率列,看 rpc_ok:2 → 4 → 0,K=8 在窗口内一次都没完成。速率列(rpc_per_s@window)在有档位为 0 时不再是可比的比率,判词里的 speedup 因此显式为 n/a。

K=0 那一档不是空载。它已经包含探针自身约 1800 次/秒 canary 的成本(main% 69.3),而真正空闲时(12 秒窗,无探针)MainThread 另测只有 4.7%(进程 11.3%)——69.3% 是 4.7% 的约 15 倍,所以那一档基本是探针自己在跑:8.1 秒窗内 14547 次 ping,即 69.3% × 8.1 s ÷ 14547 ≈ 0.39 ms 主线程 CPU/次,低于该档 p50 0.44 ms。所以本页的单调论证一律走 K=1 → K=4 → K=8,不拿 K=0 当基线。同理,main% 的绝对值里含仪表自身开销——但天花板结论不依赖它:无论怎么混,这条线程都没超过一个核。

2. 线程账本:不变核 11 条,WorkerThread 随用随起

同一时刻三个实例,以及 K=8 负载下按线程拆开的 CPU:

# 同时刻三条账本(同一台机器,三个不同实例)
instance  pid        threads  composition
user-j       432706     11       MainThread + DelayedTaskSche + 4xV8Worker + SignalInspector + 4xlibuv-worker
user-d      1017149    11       同上(本页主实例)
user-c       3527712    13       同上 + 2xWorkerThread

# 14 秒窗口、K=8 负载下按线程拆开的 CPU(本页主实例,14:4x 那一轮,当时 threads=12)
pid=1017149 window=14s threads=12
MAINTHREAD  94.9% of one core
OTHER       116.6% of one core
PROCESS     211.4% of one core
tid	thread	cpu_s	%one_core
1017149	MainThread	13.28	94.9%
1017161	V8Worker	2.87	20.5%
1017160	V8Worker	2.81	20.1%
1017162	V8Worker	2.8	20%
1017159	V8Worker	2.75	19.6%
1017177	libuv-worker	1.56	11.1%
1017176	libuv-worker	1.27	9.1%
1017174	libuv-worker	1.26	9%
1017175	libuv-worker	0.99	7.1%
126564	WorkerThread	0.01	0.1%
1017158	DelayedTaskSche	0	0%
1017163	SignalInspector	0	0%

OTHER 全部来自 4 条 V8Worker(GC/编译)与 4 条 libuv-worker(I/O)。三实例同时刻分别是 11 / 11 / 13 条:差的正是 WorkerThread(本页主实例在 14:4x 那一轮有 1 条、15:2x 已退出;user-c 当时有 2 条)。结论要按不变核说:11 条是底座,`WorkerThread` 是随 workflow / run_code 起停的临时线程,两者都不随会话数或 subagent 数增长。页面 ⑧ 号锚点列出的三处 new Worker() 就是这些临时线程的来源。

3. 尾延迟对照:五档 20 秒探针

顺序执行(不是同一时刻的对照),每档约 6.5–7.3 万样本;进程 CPU 那一列是另测的 10 秒窗口,只用来说明「这台 Host 当时在不在用」:

# 五档 20 秒 keep-alive 探针,顺序执行(不是同一时刻的对照),每档 6.5-7.3 万样本
instance  probe_n   p50    p99    p99.9   max      >20ms
user-j       71650     0.24   0.46   1.48    16.93    0
user-k     71328     0.24   0.43   1.32    15.79    0
user-a       68889     0.24   0.57   3.72    106.85   10
user-c       72884     0.23   0.48   2.78    99.46    5
user-d      65873     0.23   0.80   6.97    256.98   15

# 同一批实例在 15:2x 另测 10 秒的进程 CPU(与上面探针窗口不同刻,只用于判断
# 「这台 Host 当时在不在用」,不构成受控对照)
user-j 0.0% / user-k 0.2% / user-a 3.4% / user-d 16.7% / user-c 40.1%  of one core

进程 CPU 约 0% 的两台(user-j / user-k):>20ms 样本 0 个,max 16.9 / 15.8 ms。进程 CPU 可测的三台(user-a / user-c / user-d):max 106.9 / 99.5 / 257.0 ms,>20ms 样本 10 / 5 / 15 个。这是「在用就抖、闲着不抖」的旁证,不是受控对照:五档是顺序跑的、CPU 列与探针窗不同刻、也没有做同刻负载注入。

4. 旁证:这条线程上的常驻工作有多贵

观测值
session/list 单次(本户 2996 个会话目录)2.01 / 1.94 / 2.13 秒,响应 1,364,103 字节(连打三次;另一次同接口读到 1,376,189 字节,两次都是实测;该行原始输出未随页保存,同本表空闲行口径)
进程常驻内存 RSS本次负载期约 1.32 GB(同日更早一轮 12 个 agent 在飞时约 1.17 GB),缓慢增长(该行原始输出未随页保存,同本表空闲行口径)
进程累计 CPU(采样峰值)110.5% of one core(256 核机器上,采样窗口法;该行原始输出未随页保存,同本表空闲行口径)
对照:真正空闲 12 秒窗(无探针;本行原始输出未随页保存)MainThread 4.7% / 进程 11.3%——与 K=0 档的 69.3% 不矛盾:那一档主要是探针自身的流量(8.1 秒窗内 14547 次 ping,69.3% × 8.1 s ÷ 14547 ≈ 0.39 ms 主线程 CPU/次)。探针的 canary 循环在每一档都无条件跑(ops/dsh/upstream-issue-single-process-main-thread/main-thread-ceiling-probe.mjs:165),K=0 时它是探针唯一注入的负载

影响链:为什么「1 Host × 4 会话」≠「2 Host × 2 会话」

这是用户原始问题的直接回答。左列凡标「实测」的有本页原始输出支撑;标「推断」的没有——本页没有跑 2 进程对照实验,按本仓约定如实标注。

维度1 个 Host 装 4 个会话2 个 Host 各装 2 个会话
可用 JS 执行线程1 条(实测:主线程 98.2% 时仍只有 1 条)2 条,分布在 2 个进程、2 个 V8 堆(推断)
加并发之后实测:窗口内完成 2 → 4 → 0,吞吐不涨,延迟吸收全部增量推断:两份队列各自独立
一次停顿的波及面实测:存在 831.9 ms 级停顿;推断它让当时该 Host 上活着的每个会话一起等(本页没有做该窗口的会话数普查)推断:每次只波及该进程内的会话
GC / 堆压力1 个 V8 堆(本页未测其大小:页面唯一的记忆量是进程常驻内存 RSS 1.17–1.32 GB,含非堆部分,见「仍未定论」①;该 RSS 读数同样无页内原始输出,只有 §4 旁证表那一处记录);每个会话的整份事件日志常驻内存(代码锚点可查)推断:两个小堆,互不牵连
共享的 I/O 线程池1 个 4 线程 libuv 池 + 1 条 fsync 队列,每个 step / 工具调用都要过(代码锚点可查)推断:两个池、两条队列
每事件的主线程成本约 O(会话数 × 跟随流数):每个 follow 都注册全局 session/event 监听器,别的会话的事件也要先叫醒它(⑩ 号锚点)推断:两份 O(S×F) 各自减半
进程级共享状态共享:同一份 rootEncodingCheck 粘性缓存(#8 的 ③)、同一个 storage-domain 写入链推断:各自一份,一个中毒不拖累另一个
可观测性无持续导出或被采集的 per-Host 指标,只能体感(本仓与 DSH 都没有这类指标;本页探针的 per-Host CPU 是现读 /proc 现打印,不进任何时序库)同上
与 #7/#8 是同一条链。#8 量出 session/list 的成本是 O(盘上全部会话),其口径是单个工作区 [repo-slug] 在 09-13 的 2935 个会话目录,其中 2824 个(96%)是 subagent 会话。本页的 2996 是次日、全部工作区的口径——两者不是同一份语料(差额 = 另外几个工作区的目录 + 一天的增长)。把两个口径对齐后结论不变:用 subagent 本身就在给这条主线程增加常驻负债,而这笔负债由同一个 Host 里的每个会话共同承担,且这条线程只有一条。

代码级证据(指到行)

行号按随包发布的 lib/(0.1.5-rc.1)核对;每条都由独立核验 agent 回源逐字对过。

#事实锚点
①subagent 在父进程内创建并驱动,不是子进程dsh-subagent-in-process-driver/lib/index.js:161 startInProcessRun();:180 parent.ctx.agents.create();:211 await child.whenIdle()
②两个 subagent 入口都只是 ① 的薄壳,本随包只注册了这两个 in-process 提供者dsh-subagent-spawn-in-process/lib/index.js:35;dsh-subagent-fork-in-process/lib/index.js:50;ctx.subagents.registerProvider() 全树只有 2 处(:42 / :58),都在 in-process 包内;child_process 在 dsh-subagent*/lib/**/*.js 命中 0(spawn/fork 的命中是 provider 名与文档词)
③模型流是逐 chunk 同步扇出、没有合并:进程内所有会话与子代理的 chunk 率相加就是这条线程的负载dsh-agent-loop/lib/index.js:1040-1043 for await (const chunk of stream) { … live.push(chunk); } → :1032 this.dispatch.emit("agent/assistant-stream", { frame });(全树唯一 emit 点,逐 chunk 一次,无 await/队列/定时器夹在中间)
④每个 step、每个顶层工具调用都有一次被 await 的持久化屏障dsh-session-checkpoint-policy/lib/index.js:28 是全树同形 await ctx.sessions.flush(session) 的 3 处之一(另两处 dsh-schedule/lib/index.js:694、dsh-schedule/lib/types/persistence.js:21);本包独有的是把它挂在每 step / 每顶层工具调用的路径上;:64(llm/stream,转交 afterCheckpoint)、:66-68(tools/execute,实参 exec.agent.session)、:72-73(agent/pre-step,实参 agent.session)三处都走这条 awaited 路径
⑤那次 flush 落到盘上是 open + stat + write + fsync + close,且默认 zstd 压缩;全部跑在进程共享的 libuv 线程池(默认 4 条)上dsh-session-persistence-jsonl/lib/index.js:3046-3073 appendLines()::3049 open、:3056-3062 写入段(:3057 stat、:3059 writeFile、:3060 handle.sync())、:3054 是 closeAppendHandle() 里的 handle.close()(正常路径在 :3071 的 finally 调用,即写在关之前;回滚路径另在 :3063);:2176 DEFAULT_COMPRESSION = "zstd";:1287 promisify(zstdCompress);活进程 /proc/<pid> 只有 4 条 libuv-worker,environ 无 UV_THREADPOOL_SIZE
⑥每个活动会话把整份事件日志永久留在内存里,外加一份快照副本;每个事件两次深拷贝(data 与 surface metadata 各一次)+ 一次全量 deep-freezedsh-session/lib/index.js:989 log = [](无截断);:1109 this.eventsSnapshot ??= Object.freeze([...this.log]);:1176 snapshotJsonValue(data)、:1178 snapshotJsonValue(surfaceMetadata);:1182-1188 deepFreeze({ … })
⑦投影缓存写入在每个 storage domain 上串成一条链,且是 pretty-print JSON + atomic write + fsyncdsh-storage-domain/lib/index.js:119-120、:221-224(enqueue() 单链);dsh-session-projection-cache/lib/index.js:346-352 → dsh-storage-json/lib/index.js serializeRecord(pretty-print::127-132,:131 }, null, 2));writeAtomic(atomic write + fsync::25-41,写同目录临时文件 → :31 await handle.sync(); → :35 await rename(tmp, path); → :36 await fsyncDirectory(dirname(path));,实现在 :44-52)
⑧上游已经会把活挪出主线程:workflow 编排、run_code、以及持久化校验各有 new Worker();bash/pwsh 工具走子进程dsh-workflow-worker-thread/lib/index.js:311;dsh-code-runtime-worker-thread/lib/index.js:737;dsh-session-persistence-jsonl/lib/index.js:1491(仅校验用)。它们是本页线程账本里那些 WorkerThread 的来源。bash/pwsh 工具走子进程:dsh-tool-bash/lib/index.js:429 / dsh-tool-pwsh/lib/index.js:402 ctx.shell.run(ctx.shell.resolve(…))(ctx.shell 即 dsh-shell/lib/index.js:86 super(ctx, "shell"); 注册的执行器)→ dsh-bash-local/lib/index.js:237 / dsh-pwsh-local/lib/index.js:327 this.ctx.subprocess.spawn(…) → dsh-subprocess-local/lib/index.js:7 import { execFile, spawn, spawnSync } from "node:child_process";、:990 spawn(spec) {(dsh-base/cordis.patch.yml:199-200 挂 dsh-subprocess-local,:215 / :221 挂继承上述两个 local 执行器的 dsh-bash-sandbox / dsh-pwsh-sandbox)
⑨本部署已关闭(勿当成本次的原因):同步 SQLite 检索索引确实存在且是同步的,但配置成 openAt: never / :memory:dsh-session-query-sqlite/lib/index.js:50-51;关闭点在 dsh-web-app/cordis.patch.yml:27-30、dsh-base/cordis.patch.yml:129-133
⑩超线性的来源:每个 session/follow 都注册一个 { global: true } 的 session/event 监听器,进程内任何会话的事件都会先把它叫醒,之后才按 id 过滤——每事件派发成本约 O(跟随流数),进程总量约 O(会话数 × 跟随流数)dsh-api-session-controller/lib/index.js:1419-1426 this.ctx.on("session/event", (session, event) => { … if (session.id !== target) return; … }, { global: true })(:1419 注册、:1420 才按 id 过滤、:1426 才是 { global: true });:1427-1435(session/created 同样是每个 follow 一份无条件的 { global: true });:1436-1444(agent/assistant-stream 同形::1437 一样先派发后过滤、:1444 才是 { global: true };注意它仅在 request.assistantStream === true 时注册);「global 绕过作用域过滤」的直接证据:cordis/lib/index.js:263 filter((hook) => hook.global || !filter || filter.call(thisArg, hook.ctx))
⑪派发本身每次都要新建一个数组与一批闭包;广播还给每个 socket 各做一次完整 JSON.stringify,且不做订阅过滤cordis/lib/index.js:263 (this._hooks[name] || []).filter((hook) => hook.global || !filter || filter.call(thisArg, hook.ctx)).map((hook) => hook.callback.bind(thisArg));dsh-api-gateway/lib/index.js:346-349 send(message) { text = JSON.stringify(message); … }(「每个 socket」= :230 new RemoteStreamMuxConnection(websocket, this.open, this.failure).run() 每 upgrade 一个连接 + :358 this.socket.send(text, (error) => { 写自己那条 socket;「不做订阅过滤」= :621-629 broadcastRemoteEvent(frame) { … for (const client of this.remoteEventClients.values()) client.queue.push(wire); };每条连接经 :582 if (endpoint === "$events") return this.openRemoteEvents(payload, signal); → :605 yield* client.queue.iterate(lifetime); → :325 for await (const value of source) await this.send({ 各过一次 :349 的序列化)

期望 vs 实际

期望实际
并发会话数增加时,Host 能用上更多核(或至少明确拒绝 / 排队)主线程贴 98.2%,进程总共只用 2.03 个核;既不限流也不排队,只表现为变慢
一个会话里的 subagent 扇出,不应拖慢其它会话的定开销请求样本数掉 51%、p99 变差 3.6×、单次停顿最高 831.9 ms——同进程所有会话一起承受
提高并发应带来吞吐窗口内完成次数 2 → 4 → 0:K=8 时 8 个并发在整个窗口内一次都没完成,增量全进队列
容量模型可观测:能回答「这台 Host 还能不能再接一个会话」无持续导出或被采集的 per-Host CPU / RSS / event-loop lag / GC 指标(一次性现读不算:本页探针的 98.2% / 2.03 核就是现读 /proc/<pid>/task/<tid>/stat 算出来的);只能靠用户报「卡」

已排除的原因(都实测过)

写这一节是为了让「不是这些」也有据可查,避免下一轮重复劳动。

仍未定论(禁止当结论)

按本仓约定标「未定论」:下面几条形状相符或按结构推断,但本次没有取得能定案的证据。

① MainThread 的时间里 GC 占多少?#7 已把这个问题留成悬案。本次在负载下量到 4 条 V8Worker 合计约 80% of one core,与「并发标记在干活」相符,但这不能把主线程那 98.2% 归因给 GC——缺的是堆统计/GC 计时实验(例:--trace-gc 或 perf_hooks 的 GC observer)。未定论。

② 2 进程对照没有跑。「切成 2 个 Host 更快」来自用户体感 + 结构推断(多一条 JS 线程、波及面减半、两个 libuv 池)。本页只实测了一个进程内部。要做成定论,需要同机同负载跑 1×4 与 2×2 的对照。影响链右列因此逐行标了「推断」。未定论。

③ 浏览器侧贡献了多少?用户的两组配置同时变了标签页数:2 个网页 = 2 个渲染进程 / 2 条浏览器主线程,1 个网页 4 个会话 = 1 条。本次没有测浏览器侧,本页只主张服务端这一半;体感差异应是两者之和。未定论。

④ 速率列只作上界。rpc_per_s@window 的分子已改成同窗计数,但当某档计数为 0 时它不再是可比比率(判词里 speedup=n/a)。要做速率结论需要多轮重复并报中位数与离散度,本次每档只有一个窗口。未定论。

复现:零依赖探针与判据行

探针只用 node:http 与 /proc,不依赖任何第三方包;只读接口,不写状态。

# 本机某一户:读 $DSH_HOME/web-url.txt 的启动令牌自动换浏览器 Cookie
DSH_HOME=/home/user-d/.dsh node ops/dsh/upstream-issue-single-process-main-thread/main-thread-ceiling-probe.mjs \
  --port 4002 --pid "$(pgrep -u user-d -f 'node_modules/.bin/dsh' | head -1)"

# 看曲线:K = 同进程并发 session/list 数(0 = 带仪表本底,8 = 压满)
DSH_HOME=/home/user-d/.dsh node ops/dsh/upstream-issue-single-process-main-thread/main-thread-ceiling-probe.mjs \
  --port 4002 --pid 1017149 --levels 0,1,4,8 --seconds 8

# 判据行:main_pct_load ≥ 80 且 host_load1 ≪ host_cores
#         ⇒ result=SATURATED(退出码 0)
#         (rpc_ok_kN / speedup 只作旁证,不进门槛——main-thread-ceiling-probe.mjs:202-204:「not used as a gate」)
result退出码含义
SATURATED0复现成功:JS 主线程 ≥ 80% of one core,而宿主仍有空核(load1 < cores/2)
NO_CEILING1没测到天花板——宿主本身也忙、或该实例已把工作挪出主线程
握手 / 鉴权失败2读不到 web-url.txt、或令牌换不到 Cookie,判据无效
别在别人正在用的实例上把 --levels 开很大。session/list 是实测 ~2 秒 / 1.36 MB 的全量磁盘扫描(#7/#8),本页 K=8 那一轮就已经让同进程的定开销请求样本数掉 51%——这正是本 issue 要说明的现象,对同事实例上的人来说就是「变卡了」。

建议修法(分级)

级别修法为什么是这条 / 怎么做
P0补 per-Host 可观测性暴露进程 CPU、RSS、event-loop lag 与 GC 时间(perf_hooks.monitorEventLoopDelay 即可)。本仓与 DSH 现在都没有持续导出或被采集的这些指标(本页探针只现读 /proc 打一次表,无落库、无告警),所以这条天花板只能靠用户体感发现;先能看见,才谈容量规划。
P0给并发 agent 加限流 / 背压per-Host 的并发 agent 上限 + 明确的「排队中」状态。现在的行为是既不拒绝也不排队,只是大家一起变慢——用户无法区分「在跑」和「跑不动」。
P1把每个 follow 的 { global: true } 监听器换成按 id 订阅⑩ 号锚点:这是超线性的主要来源(每事件派发约 O(跟随流数),进程总量约 O(会话数 × 跟随流数));⑪ 号锚点的每事件分配同样随跟随流数放大。改掉它是消除一项,而不是缩小一个常数。
P1提供 out-of-process(或 worker thread)subagent 驱动,并让 profile 可选现在本随包只有 in-process 两种(② 号锚点)。上游在 ⑧ 号锚点那三处已经会用 new Worker(),把这套用到普通 subagent 路径,就能让「会话内扇出」不再吃掉其它会话依赖的那条线程。
P1把持久化屏障与压缩从交互路径上解耦④⑤ 号锚点:每个 step、每个顶层工具调用都 await flush(),而那次 flush 做的是 open+stat+write+fsync+close 加 zstd,全部挤在共享的 4 条 libuv 线程上。两个进程 = 两套 fsync 队列。
P2文档化「进程 = 并行单位」,并提供官方分片方式把容量模型写进文档,并给出把一个用户的会话分散到多个 Host 进程的受支持做法(本部署目前是每户一个 dsh-<user>.service,横向切分靠人工)。

可提交的 issue #10 正文(英文)

下面是可直接粘贴的正文;行号按随包 lib/(0.1.5-rc.1)核对。

Title: Every session and every subagent in a DSH Host shares one JavaScript
       main thread - the process, not the session, is the unit of parallelism

Summary
-------
A DSH Host process serves every session and runs every subagent on a single
JavaScript main thread. dsh-subagent-in-process-driver creates children with
parent.ctx.agents.create() inside the same process, and the shipped tree
contains no child_process-based subagent driver. Consequently:

  * one Host cannot use more than about one core of JavaScript, no matter how
    many cores the machine has;
  * adding concurrency to one Host does not add throughput, it only adds queueing;
  * latency for UNRELATED requests served by the same Host degrades as the
    number of concurrent operations grows.

The practical consequence, which is why this is reported: running N sessions in
one Host is NOT equivalent to running N/2 sessions in each of two Hosts, even
at identical total load. Splitting into two Hosts yields a second JS thread.

Environment
-----------
  DSH        0.1.5-rc.1 (shipped lib/, offline bundle @deepseek-ai/dsh)
  Host       Linux x86_64, 256 logical cores, load1 9.7 (about 4% utilised)
  Instance   dsh-user-d - dsh web --profile infra --host 127.0.0.1 --port 4002 (pid 1017149)
  Corpus     2996 session directories on disk for this user
  Node       v24.16.0

Steps to reproduce
------------------
  DSH_HOME=/home/USER/.dsh node main-thread-ceiling-probe.mjs \
      --port 4002 --pid HOST_PID --levels 0,1,4,8 --seconds 8

  load  = K concurrent `session/list` RPCs (a real, unbounded on-disk scan)
  probe = a tight keep-alive loop of `GET /api/ping`, a fixed-cost request that
          reads no session state, so its latency is time the main thread was
          not free for anyone
  cpu   = per-thread CPU from /proc/PID/task/TID/stat

Actual behaviour
----------------
  K  rpc_ok  rpc/s@window  canary_n   p50   p99  p99.9    max  >20ms  main%  other%  proc%
  0       0   0.00@8.1s      14547  0.44  1.13   9.37   92.9      3   69.3     5.0   74.3
  1       2   0.25@8.1s      15268  0.43  0.92   7.17  272.0      6   87.9    50.0  137.9
  4       4   0.50@8.0s       9754  0.62  1.62   9.75  831.9      7   98.4    87.3  185.7
  8       0   0.00@8.0s       7464  0.90  3.30  20.02   93.0      8   98.2   104.4  202.6

  MAIN_THREAD_CEILING main_pct_idle=69.3 main_pct_load=98.2
    process_pct_load=202.6 host_cores=256 host_load1=9.72
    rpc_ok_k1=2 rpc_ok_kN=0 ok_after_drain_kN=8 window_s=8 result=SATURATED

  * The JS MainThread reaches 98.2% of one core, while the WHOLE PROCESS uses
    202.6% - about 2.03 cores - of a 256-core machine at ~4% load. The excess
    over one core is four V8Worker threads (GC/compiler) plus four libuv I/O
    workers, not a second JS execution thread. Note the probe's own canary is
    itself part of this load: with the canary running but no RPC load (K=0) the
    thread already sits at 69.3%, against 4.7% main-thread (11.3% process) measured
    separately on the truly idle instance. Either way the JS thread never exceeds
    one core.
  * Thread count is not fixed. The invariant core is ELEVEN threads
    (MainThread + DelayedTaskScheduler + 4 V8Worker + SignalInspector +
    4 libuv-worker); transient WorkerThreads appear on top while workflows or
    run_code are active (11 / 11 / 13 observed on three instances at the same
    moment). Nothing scales with session or subagent count.
  * QUEUEING, measured without relying on any rate estimate: in an 8 s window,
    K=1 completed 2 RPCs, K=4 completed 4, and K=8 completed ZERO inside the
    window (8 more only after the drain). Per-RPC wall latency therefore grows
    from about 4 s (K=1) to 4-8 s (K=4) to more than 8 s (K=8), while the
    offered concurrency rises 8x. Throughput is flat; latency absorbs
    everything. (An earlier revision of this report quoted a "2.63x speedup"
    from a probe bug that divided a post-drain count by a pre-drain window;
    the probe is fixed and that number is withdrawn.)
  * A probe that touches no session state loses half its throughput as K rises,
    K=1 -> K=8: 15268 -> 7464 samples per window (-51%), p99 0.92 -> 3.30 ms
    (3.6x), p99.9 7.17 -> 20.02 ms (2.8x). One 831.9 ms stall was recorded at
    K=4. K=0 is NOT an idle baseline: the canary loop runs at every level, so
    with no session/list load it is the only load the probe injects. The 69.3%
    is essentially the cost of serving those pings - 14547 in 8.1 s (1796/s),
    about 0.39 ms of main-thread CPU each, under the K=0 row's own 0.44 ms
    p50. This instance's own background load is the 4.7% main-thread figure
    measured separately with no probe running. That is why the monotone arm used
    above is K=1 -> K=4 -> K=8.
  * Control, weaker than a controlled experiment and labelled as such: five
    20 s canary runs against five instances, executed sequentially. The two
    instances whose process CPU was ~0% showed max 15.8 / 16.9 ms and ZERO
    samples above 20 ms; the instances carrying measurable CPU showed max
    99.5 / 106.9 / 257.0 ms with 5 / 10 / 15 such samples.

Expected behaviour
------------------
  Either (a) a Host should be able to use more than one core when several
  sessions and subagents are active, or (b) DSH should document and expose the
  process as the unit of parallelism, with a supported way to shard sessions
  across Hosts, a per-Host concurrency limit / backpressure, and metrics that
  make the ceiling visible.

Code references (0.1.5-rc.1, shipped lib/)
-----------------------------------------
  dsh-subagent-in-process-driver/lib/index.js:161  startInProcessRun()
  dsh-subagent-in-process-driver/lib/index.js:180  parent.ctx.agents.create()
  dsh-subagent-in-process-driver/lib/index.js:211  await child.whenIdle()
  dsh-subagent-spawn-in-process/lib/index.js:35    -> startInProcessRun
  dsh-subagent-fork-in-process/lib/index.js:50     -> startInProcessRun
  (the two in-process providers are the only ones registered in this bundle;
   no child_process-based subagent driver is shipped here)

  dsh-agent-loop/lib/index.js:1040-1043            for await (chunk) ... live.push
  dsh-agent-loop/lib/index.js:1032                 dispatch.emit("agent/assistant-stream")
                                                   - one frame per chunk, no coalescing
  dsh-session-checkpoint-policy/lib/index.js:28    await ctx.sessions.flush(session)
  dsh-session-checkpoint-policy/lib/index.js:64-73 flush awaited on llm/stream,
                                                   tools/execute and agent/pre-step
  dsh-session-persistence-jsonl/lib/index.js:3046-3073 appendLines = open :3049,
                                                   stat :3057, write :3059,
                                                   fsync :3060, close :3054
                                                   (close runs at :3071 in the finally, after the write)
  dsh-session-persistence-jsonl/lib/index.js:2176  DEFAULT_COMPRESSION = "zstd"
  dsh-session/lib/index.js:989,1109,1176,1182      whole event log retained in memory,
                                                   deep copy x2 (data + surface metadata) + freeze per event

  dsh-api-session-controller/lib/index.js:1419-1426 { global: true } listener on
                                                   session/event per follow stream
                                                   (:1420 filters by id only after
                                                    delivery; :1427-1435 session/created
                                                    is a second per-follow one;
                                                    :1436-1444 agent/assistant-stream,
                                                    same shape, same late id filter)
  cordis/lib/index.js:263                          filter(...).map(hook =>
                                                   hook.callback.bind(thisArg))
  dsh-api-gateway/lib/index.js:346-349             JSON.stringify per socket
  dsh-workflow-worker-thread/lib/index.js:311      new Worker(...) - one of the
                                                   off-thread paths that already exist
  dsh-code-runtime-worker-thread/lib/index.js:737  new Worker(...) - run_code
  dsh-session-persistence-jsonl/lib/index.js:1491  new Worker(...) - verification only

Ruled out (all measured)
------------------------
  * Host saturation - 256 cores at load1 9.7 (about 4%). The 202.6% process CPU
    is one saturated JS thread plus GC and I/O helper threads.
  * Network / gateway - loopback direct, the per-user gateway is bypassed; p50
    stays at 0.43-0.90 ms throughout.
  * A frozen event loop - the canary still answers 7464-15268 times per 8 s
    window across the K sweep (15268 at K=1, 7464 at K=8), and issue #9
    separately measured the 2 s heartbeat timer not stalling (2000.0 ms idle;
    2085 ms under 16 concurrent session/list calls, 2026-09-13 on the user-a
    instance). The thread is SATURATED AND QUEUEING, not frozen.
  * One slow request holding the loop - the K=1 level is a closed loop with at
    most one session/list in flight, and there the canary p99 over the 8.1 s
    window was only 0.92 ms (p99.9 7.17 ms), against about 2.0 s for one
    session/list: three orders of magnitude apart, so the scan yields; the
    cost is aggregate CPU, not one blocking call. Caveat: 0.92 ms is a
    whole-window percentile, NOT a conditional percentile over a single
    in-flight scan - the probe has no such mode, so "the tail stays low while
    a scan runs" is an inference, not a measurement.
  * A process-wide mutex - the per-turn path waits on no process-wide lock.
    Persistence does hold a PER-SESSION kernel write lease
    (dsh-session-persistence-jsonl/lib/index.js:652, SessionWriteLease; flock on
    POSIX at :11/:690, a named kernel semaphore at :558 on Windows), but a
    handle takes it once, on its first durable write (:248,
    this.lease ??= await this.storage.acquireWriteLease(this.header)), and it
    belongs to that one session, so it cannot queue turns across sessions.
    Persistence is a true append with one write chain per session and a
    200 ms batching window.
  * Synchronous SQLite - disabled in this deployment (openAt: never); it is also
    not on the per-turn path. Latent risk, not this cause.
  * Per-token durable writes - format v3 coalesces token deltas into
    text-chunks records; the durable write happens once per assistant/message.

Not yet proven
--------------
  * How much MainThread time is GC versus request work. Issue #7 left the same
    question open. Under load the four V8Worker threads total roughly 80% of
    one core, consistent with active concurrent marking, but that does not
    attribute the MainThread time.
  * The two-process configuration. This report measures ONE process; the 2x2
    comparison comes from the reporter's experience plus structural inference.
    No two-process control run was performed.
  * The browser side. The reporter's two configurations also differ in the
    number of browser tabs, hence renderer processes and event loops.

Suggested fixes
---------------
  1. Publish per-Host metrics: process CPU, RSS, event-loop lag, GC time.
     Nothing in DSH or in this deployment exposes them today, so the ceiling
     stays invisible until a user complains.
  2. Add an out-of-process or worker-thread subagent driver and let the profile
     choose it, so a fan-out in one session cannot consume the thread every
     other session depends on.
  3. Bound the fan-out: a per-Host concurrency limit / backpressure for
     concurrent agents, with an explicit "queued" state instead of silent
     degradation.
  4. Replace the per-follow { global: true } listener with an id-keyed
     subscription, so a session's events do not wake every other session's
     followers. This removes a superlinear term rather than shrinking a constant.
  5. Move the fsync + zstd persistence barrier off the interactive path, or make
     the durability level configurable.
  6. Document that the process is the unit of parallelism, and provide a
     supported way to shard sessions across Hosts.

最后修改:2026-10-01 19:51:07 +08:00