DeepSeek Harness · Upstream · Issue #10 草案
98.2% of one core,而整个进程只用掉 2.03 个核先说结论:用户报「同一台机器上,2 个 dsh 网页各 2 个 session 明显比 1 个 dsh 网页 4 个 session 流畅」,在确认拓扑(每户一个独立 Node 进程)之后,根因不在网络、不在浏览器、也不在某一条慢请求,而在进程模型本身——Host 进程才是并行与隔离的单位,会话不是。所有 subagent 都由 dsh-subagent-in-process-driver 在父进程内创建(本随包只注册了两个 in-process 提供者,没有 child_process 系 subagent 驱动),因此一个 Host 里会话数 × 每会话 subagent 数个 agent 全排在同一条 JS 主线程上。最硬的一条证据不依赖任何速率估计:8 秒窗口内,K=1 完成 2 次 RPC、K=4 完成 4 次、K=8 在窗口内完成 0 次(另外 8 次是排空后才完成的)——提供的并发涨了 8 倍,窗口内吞吐没有涨,涨的全是排队。同刻那条不读任何会话状态的定开销探针样本数掉 51%、p99 变差 3.6×,主线程贴到 98.2% of one core,而进程总共只用掉 2.03 个核(宿主 256 核、load1 9.7)。切成 2 个 Host 就多拿到一条 JS 线程。本页给出零依赖复现探针、代码锚点(指到行)、已排除的原因、未定论项、建议修法与可直接粘贴的英文 issue 正文。
K 路并发 session/list,同时用 GET /api/ping(不读任何会话状态)做尾延迟探针——判词出现 main_pct_load ≥ 80 而 host_load1 ≪ host_cores,且 rpc_ok_kN 不随 K 增长,即为天花板:MAIN_THREAD_CEILING … main_pct_load=98.2 process_pct_load=202.6 host_cores=256 host_load1=9.72 rpc_ok_k1=2 rpc_ok_kN=0 ok_after_drain_kN=8 window_s=8 result=SATURATED。探针:node ops/dsh/upstream-issue-single-process-main-thread/main-thread-ceiling-probe.mjs --port 4002 --pid <host-pid>。dsh-perf-probe.py 只统计会话内直方图,从不读 /proc;本仓 DSH 工具链(ops/dsh + ops/monitoring/dsh)里唯一读 /proc 的 per-Host CPU 读数就是本页探针,它现读现打印、不落任何时序库),只能靠用户报;③与本专栏 #7/#8 同源:那条实测 9.7–11.9 秒的 session/list 正跑在这条主线程上,会话越多、subagent 越多,常驻负债越重。三条都是本次实测(2026-09-14 15:20,生产实例),命令与原始输出都在本页;源代码锚点按随包发布的 0.1.5-rc.1 lib/ 核对。
dsh-subagent-in-process-driver/lib/index.js:161 的 startInProcessRun() 直接调 parent.ctx.agents.create()(:180),把子代理交给同文件的 drivePublishedRun()(:195),由它 await child.whenIdle()(:211);dsh-subagent-spawn-in-process/lib/index.js:35 与 dsh-subagent-fork-in-process/lib/index.js:50 都只是它的薄壳,而这两个 in-process 提供者就是本随包唯一注册过的两个。dsh-subagent*/lib/**/*.js 里 child_process 命中 0——spawn/fork 有若干命中,但都是 provider 名与文档词,不是进程创建。所以「会话数 × 每会话 subagent 数」个 agent 共享同一条 JS 主线程与同一个 V8 堆。(注意口径:这里的「后端」指 subagent 子级后端。随包确有 8 个与 ACP / SDK / Codex / Claude Code 同名的包——dsh-acp / dsh-acp-app / dsh-sdk-app / dsh-sdk-jsonrpc-server / dsh-sdk-minimal / dsh-sdk-protocol / dsh-hooks-codex / dsh-hooks-claude-code——但它们都不是子级后端,没有一个注册 subagent 提供者;上游真正的 ACP / SDK / Codex / Claude Code 子级后端不在这个离线包里。本页只对随包 0.1.5-rc.1 下结论。)
K=8 时 MainThread = 98.2% of one core,同刻整个进程 202.6%(= 2.03 个核)——多出来的部分是 4 条 V8Worker(GC/编译)与 4 条 libuv-worker(I/O),不是第二条 JS 执行线程。宿主 256 核、load1 = 9.7(约 4% 利用率)。线程数不是固定的:不变核是 11 条(MainThread + DelayedTaskSche + 4×V8Worker + SignalInspector + 4×libuv-worker),WorkerThread 随 workflow / run_code 起停,同一时刻三个实例量到 11 / 11 / 13;没有任何一条随会话数或 subagent 数增长。
三个档各跑一个约 8 秒窗口,窗口内完成的 RPC 次数是 2 / 4 / 0(K=1 / 4 / 8);K=8 那 8 次全部在窗口之后才完成(ok_after_drain_kN=8)。按闭合循环(完成一次才发下一次)反推,单次 RPC 墙钟时间从 K=1 的约 4 秒升到 K=4 的 4–8 秒、K=8 的 > 8 秒——并发涨 8 倍,吞吐不涨,延迟全部吸收。同刻那条不读任何会话状态的定开销探针,K=1 → K=8 样本数 15268 → 7464(−51%)、p99 0.92 → 3.30 ms(3.6×)、p99.9 7.17 → 20.02 ms(2.8×),K=4 还测到一次 831.9 ms 的停顿。
环境:dsh 0.1.5-rc.1(随包 lib/),生产实例 dsh-user-d(pid 1017149,127.0.0.1:4002);宿主 256 逻辑核、load1 9.7;本户盘上 2996 个会话目录;Node v24.16.0。读数采集于 2026-09-14 15:20(+08:00),数字是活账,提交 issue 前请以当刻读数为准。
K = 同一进程内并发 session/list 数;探针 = GET /api/ping(定开销、不读会话状态);CPU 取自 /proc/<pid>/task/<tid>/stat,main% 只算 JS 主线程。rpc_ok 只数落在窗口内的完成(分子分母同窗),ok_after_drain 另计。
# DSH main-thread ceiling probe
# target : http://127.0.0.1:4002 (pid 1017149)
# host cores : 256 load1 9.71
# load : K concurrent session/list RPCs
# probe : GET /api/ping (fixed cost, reads no session state)
#
K rpc_ok rpc_per_s@window canary_n p50 p99 p99.9 max >20ms main% other% proc%
0 0 0.00@8.1s 14547 0.44 1.13 9.37 92.9 3 69.3 5 74.3
1 2 0.25@8.1s 15268 0.43 0.92 7.17 272 6 87.9 50 137.9
4 4 0.50@8.0s 9754 0.62 1.62 9.75 831.9 7 98.4 87.3 185.7
8 0 0.00@8.0s 7464 0.9 3.3 20.02 93 8 98.2 104.4 202.6
#
MAIN_THREAD_CEILING main_pct_idle=69.3 main_pct_load=98.2 process_pct_load=202.6 host_cores=256 host_load1=9.72 rpc_per_s_k1=0.25 rpc_per_s_kN=0 window_s=8 rpc_ok_k1=2 rpc_ok_kN=0 ok_after_drain_kN=8 speedup=n/a speedup_note=n/a_because_in_window_count_is_0 canary_p999_idle_ms=9.37 canary_p999_load_ms=20.02 canary_max_load_ms=93 result=SATURATED
读法:主线程贴到 98.2%,而进程只用了 2.03 个核、机器上还有 254 个核没被它用上;吞吐不看速率列,看 rpc_ok:2 → 4 → 0,K=8 在窗口内一次都没完成。速率列(rpc_per_s@window)在有档位为 0 时不再是可比的比率,判词里的 speedup 因此显式为 n/a。
main% 69.3),而真正空闲时(12 秒窗,无探针)MainThread 另测只有 4.7%(进程 11.3%)——69.3% 是 4.7% 的约 15 倍,所以那一档基本是探针自己在跑:8.1 秒窗内 14547 次 ping,即 69.3% × 8.1 s ÷ 14547 ≈ 0.39 ms 主线程 CPU/次,低于该档 p50 0.44 ms。所以本页的单调论证一律走 K=1 → K=4 → K=8,不拿 K=0 当基线。同理,main% 的绝对值里含仪表自身开销——但天花板结论不依赖它:无论怎么混,这条线程都没超过一个核。同一时刻三个实例,以及 K=8 负载下按线程拆开的 CPU:
# 同时刻三条账本(同一台机器,三个不同实例)
instance pid threads composition
user-j 432706 11 MainThread + DelayedTaskSche + 4xV8Worker + SignalInspector + 4xlibuv-worker
user-d 1017149 11 同上(本页主实例)
user-c 3527712 13 同上 + 2xWorkerThread
# 14 秒窗口、K=8 负载下按线程拆开的 CPU(本页主实例,14:4x 那一轮,当时 threads=12)
pid=1017149 window=14s threads=12
MAINTHREAD 94.9% of one core
OTHER 116.6% of one core
PROCESS 211.4% of one core
tid thread cpu_s %one_core
1017149 MainThread 13.28 94.9%
1017161 V8Worker 2.87 20.5%
1017160 V8Worker 2.81 20.1%
1017162 V8Worker 2.8 20%
1017159 V8Worker 2.75 19.6%
1017177 libuv-worker 1.56 11.1%
1017176 libuv-worker 1.27 9.1%
1017174 libuv-worker 1.26 9%
1017175 libuv-worker 0.99 7.1%
126564 WorkerThread 0.01 0.1%
1017158 DelayedTaskSche 0 0%
1017163 SignalInspector 0 0%
OTHER 全部来自 4 条 V8Worker(GC/编译)与 4 条 libuv-worker(I/O)。三实例同时刻分别是 11 / 11 / 13 条:差的正是 WorkerThread(本页主实例在 14:4x 那一轮有 1 条、15:2x 已退出;user-c 当时有 2 条)。结论要按不变核说:11 条是底座,`WorkerThread` 是随 workflow / run_code 起停的临时线程,两者都不随会话数或 subagent 数增长。页面 ⑧ 号锚点列出的三处 new Worker() 就是这些临时线程的来源。
顺序执行(不是同一时刻的对照),每档约 6.5–7.3 万样本;进程 CPU 那一列是另测的 10 秒窗口,只用来说明「这台 Host 当时在不在用」:
# 五档 20 秒 keep-alive 探针,顺序执行(不是同一时刻的对照),每档 6.5-7.3 万样本
instance probe_n p50 p99 p99.9 max >20ms
user-j 71650 0.24 0.46 1.48 16.93 0
user-k 71328 0.24 0.43 1.32 15.79 0
user-a 68889 0.24 0.57 3.72 106.85 10
user-c 72884 0.23 0.48 2.78 99.46 5
user-d 65873 0.23 0.80 6.97 256.98 15
# 同一批实例在 15:2x 另测 10 秒的进程 CPU(与上面探针窗口不同刻,只用于判断
# 「这台 Host 当时在不在用」,不构成受控对照)
user-j 0.0% / user-k 0.2% / user-a 3.4% / user-d 16.7% / user-c 40.1% of one core
进程 CPU 约 0% 的两台(user-j / user-k):>20ms 样本 0 个,max 16.9 / 15.8 ms。进程 CPU 可测的三台(user-a / user-c / user-d):max 106.9 / 99.5 / 257.0 ms,>20ms 样本 10 / 5 / 15 个。这是「在用就抖、闲着不抖」的旁证,不是受控对照:五档是顺序跑的、CPU 列与探针窗不同刻、也没有做同刻负载注入。
| 观测 | 值 |
|---|---|
session/list 单次(本户 2996 个会话目录) | 2.01 / 1.94 / 2.13 秒,响应 1,364,103 字节(连打三次;另一次同接口读到 1,376,189 字节,两次都是实测;该行原始输出未随页保存,同本表空闲行口径) |
| 进程常驻内存 RSS | 本次负载期约 1.32 GB(同日更早一轮 12 个 agent 在飞时约 1.17 GB),缓慢增长(该行原始输出未随页保存,同本表空闲行口径) |
| 进程累计 CPU(采样峰值) | 110.5% of one core(256 核机器上,采样窗口法;该行原始输出未随页保存,同本表空闲行口径) |
| 对照:真正空闲 12 秒窗(无探针;本行原始输出未随页保存) | MainThread 4.7% / 进程 11.3%——与 K=0 档的 69.3% 不矛盾:那一档主要是探针自身的流量(8.1 秒窗内 14547 次 ping,69.3% × 8.1 s ÷ 14547 ≈ 0.39 ms 主线程 CPU/次)。探针的 canary 循环在每一档都无条件跑(ops/dsh/upstream-issue-single-process-main-thread/main-thread-ceiling-probe.mjs:165),K=0 时它是探针唯一注入的负载 |
这是用户原始问题的直接回答。左列凡标「实测」的有本页原始输出支撑;标「推断」的没有——本页没有跑 2 进程对照实验,按本仓约定如实标注。
| 维度 | 1 个 Host 装 4 个会话 | 2 个 Host 各装 2 个会话 |
|---|---|---|
| 可用 JS 执行线程 | 1 条(实测:主线程 98.2% 时仍只有 1 条) | 2 条,分布在 2 个进程、2 个 V8 堆(推断) |
| 加并发之后 | 实测:窗口内完成 2 → 4 → 0,吞吐不涨,延迟吸收全部增量 | 推断:两份队列各自独立 |
| 一次停顿的波及面 | 实测:存在 831.9 ms 级停顿;推断它让当时该 Host 上活着的每个会话一起等(本页没有做该窗口的会话数普查) | 推断:每次只波及该进程内的会话 |
| GC / 堆压力 | 1 个 V8 堆(本页未测其大小:页面唯一的记忆量是进程常驻内存 RSS 1.17–1.32 GB,含非堆部分,见「仍未定论」①;该 RSS 读数同样无页内原始输出,只有 §4 旁证表那一处记录);每个会话的整份事件日志常驻内存(代码锚点可查) | 推断:两个小堆,互不牵连 |
| 共享的 I/O 线程池 | 1 个 4 线程 libuv 池 + 1 条 fsync 队列,每个 step / 工具调用都要过(代码锚点可查) | 推断:两个池、两条队列 |
| 每事件的主线程成本 | 约 O(会话数 × 跟随流数):每个 follow 都注册全局 session/event 监听器,别的会话的事件也要先叫醒它(⑩ 号锚点) | 推断:两份 O(S×F) 各自减半 |
| 进程级共享状态 | 共享:同一份 rootEncodingCheck 粘性缓存(#8 的 ③)、同一个 storage-domain 写入链 | 推断:各自一份,一个中毒不拖累另一个 |
| 可观测性 | 无持续导出或被采集的 per-Host 指标,只能体感(本仓与 DSH 都没有这类指标;本页探针的 per-Host CPU 是现读 /proc 现打印,不进任何时序库) | 同上 |
行号按随包发布的 lib/(0.1.5-rc.1)核对;每条都由独立核验 agent 回源逐字对过。
| # | 事实 | 锚点 |
|---|---|---|
| ① | subagent 在父进程内创建并驱动,不是子进程 | dsh-subagent-in-process-driver/lib/index.js:161 startInProcessRun();:180 parent.ctx.agents.create();:211 await child.whenIdle() |
| ② | 两个 subagent 入口都只是 ① 的薄壳,本随包只注册了这两个 in-process 提供者 | dsh-subagent-spawn-in-process/lib/index.js:35;dsh-subagent-fork-in-process/lib/index.js:50;ctx.subagents.registerProvider() 全树只有 2 处(:42 / :58),都在 in-process 包内;child_process 在 dsh-subagent*/lib/**/*.js 命中 0(spawn/fork 的命中是 provider 名与文档词) |
| ③ | 模型流是逐 chunk 同步扇出、没有合并:进程内所有会话与子代理的 chunk 率相加就是这条线程的负载 | dsh-agent-loop/lib/index.js:1040-1043 for await (const chunk of stream) { … live.push(chunk); } → :1032 this.dispatch.emit("agent/assistant-stream", { frame });(全树唯一 emit 点,逐 chunk 一次,无 await/队列/定时器夹在中间) |
| ④ | 每个 step、每个顶层工具调用都有一次被 await 的持久化屏障 | dsh-session-checkpoint-policy/lib/index.js:28 是全树同形 await ctx.sessions.flush(session) 的 3 处之一(另两处 dsh-schedule/lib/index.js:694、dsh-schedule/lib/types/persistence.js:21);本包独有的是把它挂在每 step / 每顶层工具调用的路径上;:64(llm/stream,转交 afterCheckpoint)、:66-68(tools/execute,实参 exec.agent.session)、:72-73(agent/pre-step,实参 agent.session)三处都走这条 awaited 路径 |
| ⑤ | 那次 flush 落到盘上是 open + stat + write + fsync + close,且默认 zstd 压缩;全部跑在进程共享的 libuv 线程池(默认 4 条)上 | dsh-session-persistence-jsonl/lib/index.js:3046-3073 appendLines()::3049 open、:3056-3062 写入段(:3057 stat、:3059 writeFile、:3060 handle.sync())、:3054 是 closeAppendHandle() 里的 handle.close()(正常路径在 :3071 的 finally 调用,即写在关之前;回滚路径另在 :3063);:2176 DEFAULT_COMPRESSION = "zstd";:1287 promisify(zstdCompress);活进程 /proc/<pid> 只有 4 条 libuv-worker,environ 无 UV_THREADPOOL_SIZE |
| ⑥ | 每个活动会话把整份事件日志永久留在内存里,外加一份快照副本;每个事件两次深拷贝(data 与 surface metadata 各一次)+ 一次全量 deep-freeze | dsh-session/lib/index.js:989 log = [](无截断);:1109 this.eventsSnapshot ??= Object.freeze([...this.log]);:1176 snapshotJsonValue(data)、:1178 snapshotJsonValue(surfaceMetadata);:1182-1188 deepFreeze({ … }) |
| ⑦ | 投影缓存写入在每个 storage domain 上串成一条链,且是 pretty-print JSON + atomic write + fsync | dsh-storage-domain/lib/index.js:119-120、:221-224(enqueue() 单链);dsh-session-projection-cache/lib/index.js:346-352 → dsh-storage-json/lib/index.js serializeRecord(pretty-print::127-132,:131 }, null, 2));writeAtomic(atomic write + fsync::25-41,写同目录临时文件 → :31 await handle.sync(); → :35 await rename(tmp, path); → :36 await fsyncDirectory(dirname(path));,实现在 :44-52) |
| ⑧ | 上游已经会把活挪出主线程:workflow 编排、run_code、以及持久化校验各有 new Worker();bash/pwsh 工具走子进程 | dsh-workflow-worker-thread/lib/index.js:311;dsh-code-runtime-worker-thread/lib/index.js:737;dsh-session-persistence-jsonl/lib/index.js:1491(仅校验用)。它们是本页线程账本里那些 WorkerThread 的来源。bash/pwsh 工具走子进程:dsh-tool-bash/lib/index.js:429 / dsh-tool-pwsh/lib/index.js:402 ctx.shell.run(ctx.shell.resolve(…))(ctx.shell 即 dsh-shell/lib/index.js:86 super(ctx, "shell"); 注册的执行器)→ dsh-bash-local/lib/index.js:237 / dsh-pwsh-local/lib/index.js:327 this.ctx.subprocess.spawn(…) → dsh-subprocess-local/lib/index.js:7 import { execFile, spawn, spawnSync } from "node:child_process";、:990 spawn(spec) {(dsh-base/cordis.patch.yml:199-200 挂 dsh-subprocess-local,:215 / :221 挂继承上述两个 local 执行器的 dsh-bash-sandbox / dsh-pwsh-sandbox) |
| ⑨ | 本部署已关闭(勿当成本次的原因):同步 SQLite 检索索引确实存在且是同步的,但配置成 openAt: never / :memory: | dsh-session-query-sqlite/lib/index.js:50-51;关闭点在 dsh-web-app/cordis.patch.yml:27-30、dsh-base/cordis.patch.yml:129-133 |
| ⑩ | 超线性的来源:每个 session/follow 都注册一个 { global: true } 的 session/event 监听器,进程内任何会话的事件都会先把它叫醒,之后才按 id 过滤——每事件派发成本约 O(跟随流数),进程总量约 O(会话数 × 跟随流数) | dsh-api-session-controller/lib/index.js:1419-1426 this.ctx.on("session/event", (session, event) => { … if (session.id !== target) return; … }, { global: true })(:1419 注册、:1420 才按 id 过滤、:1426 才是 { global: true });:1427-1435(session/created 同样是每个 follow 一份无条件的 { global: true });:1436-1444(agent/assistant-stream 同形::1437 一样先派发后过滤、:1444 才是 { global: true };注意它仅在 request.assistantStream === true 时注册);「global 绕过作用域过滤」的直接证据:cordis/lib/index.js:263 filter((hook) => hook.global || !filter || filter.call(thisArg, hook.ctx)) |
| ⑪ | 派发本身每次都要新建一个数组与一批闭包;广播还给每个 socket 各做一次完整 JSON.stringify,且不做订阅过滤 | cordis/lib/index.js:263 (this._hooks[name] || []).filter((hook) => hook.global || !filter || filter.call(thisArg, hook.ctx)).map((hook) => hook.callback.bind(thisArg));dsh-api-gateway/lib/index.js:346-349 send(message) { text = JSON.stringify(message); … }(「每个 socket」= :230 new RemoteStreamMuxConnection(websocket, this.open, this.failure).run() 每 upgrade 一个连接 + :358 this.socket.send(text, (error) => { 写自己那条 socket;「不做订阅过滤」= :621-629 broadcastRemoteEvent(frame) { … for (const client of this.remoteEventClients.values()) client.queue.push(wire); };每条连接经 :582 if (endpoint === "$events") return this.openRemoteEvents(payload, signal); → :605 yield* client.queue.iterate(lifetime); → :325 for await (const value of source) await this.send({ 各过一次 :349 的序列化) |
| 期望 | 实际 |
|---|---|
| 并发会话数增加时,Host 能用上更多核(或至少明确拒绝 / 排队) | 主线程贴 98.2%,进程总共只用 2.03 个核;既不限流也不排队,只表现为变慢 |
| 一个会话里的 subagent 扇出,不应拖慢其它会话的定开销请求 | 样本数掉 51%、p99 变差 3.6×、单次停顿最高 831.9 ms——同进程所有会话一起承受 |
| 提高并发应带来吞吐 | 窗口内完成次数 2 → 4 → 0:K=8 时 8 个并发在整个窗口内一次都没完成,增量全进队列 |
| 容量模型可观测:能回答「这台 Host 还能不能再接一个会话」 | 无持续导出或被采集的 per-Host CPU / RSS / event-loop lag / GC 指标(一次性现读不算:本页探针的 98.2% / 2.03 核就是现读 /proc/<pid>/task/<tid>/stat 算出来的);只能靠用户报「卡」 |
写这一节是为了让「不是这些」也有据可查,避免下一轮重复劳动。
load1 9.7(约 4%);同刻进程仅 2.03 个核。那一档的其它 CPU 是 GC 与 I/O 辅助线程,不是第二条 JS 执行线程。127.0.0.1,绕过每户的 gateway.mjs;p50 全程 0.43–0.90 ms。15268 / 9754 / 7464 次往返(K=1 / 4 / 8);#9 另在 2026-09-13 于 user-a 实例(127.0.0.1:4006)独立测到 2 秒心跳计时器未停摆(idle 2000.0 ms;16 路 session/list 并发下 2085 ms)。这条线程是饱和排队,不是冻住——两者体感相似但修法不同。K=1 档就是「任意时刻最多一条 session/list 在飞」的闭合循环,该档 8.1 秒窗口内探针 p99 只有 0.92 ms、p99.9 只有 7.17 ms(§实测锚点 1 原始输出的 K=1 行),而单次 session/list 本身约 2.0 秒(2.01 / 1.94 / 2.13 秒,§4 旁证表)——两者相差约三个数量级,说明扫描会让出。代价是累计 CPU,不是单次阻塞。口径:0.92 ms 是整个 8.1 秒窗口的 p99(窗口内只完成了 2 次 session/list),不是「单次扫描进行中的条件分位」——本页探针没有这种模式,因此「扫描期间尾延迟仍低」是推断,不是直测。dsh-session-persistence-jsonl/lib/index.js:652 SessionWriteLease,POSIX 走 flock(:11、:690),Windows 走命名信号量 :558),但每个 handle 只在首次落盘时取一次(:248 this.lease ??= await this.storage.acquireWriteLease(this.header)),只属于这一个会话、不跨会话排队。持久化是真追加(不重写、不压缩整档),每个会话一条自己的写入链,还有 200 ms 合批窗口。所以这不是「锁」问题。openAt: never),它也不在每轮路径上——只在显式检索时按需对账。text-chunks 记录,落盘发生在一次 assistant/message 里,不是每个 token 各写一次。run_code 起停的 WorkerThread,不变核恒为 11 条。按本仓约定标「未定论」:下面几条形状相符或按结构推断,但本次没有取得能定案的证据。
① MainThread 的时间里 GC 占多少?#7 已把这个问题留成悬案。本次在负载下量到 4 条 V8Worker 合计约 80% of one core,与「并发标记在干活」相符,但这不能把主线程那 98.2% 归因给 GC——缺的是堆统计/GC 计时实验(例:--trace-gc 或 perf_hooks 的 GC observer)。未定论。
② 2 进程对照没有跑。「切成 2 个 Host 更快」来自用户体感 + 结构推断(多一条 JS 线程、波及面减半、两个 libuv 池)。本页只实测了一个进程内部。要做成定论,需要同机同负载跑 1×4 与 2×2 的对照。影响链右列因此逐行标了「推断」。未定论。
③ 浏览器侧贡献了多少?用户的两组配置同时变了标签页数:2 个网页 = 2 个渲染进程 / 2 条浏览器主线程,1 个网页 4 个会话 = 1 条。本次没有测浏览器侧,本页只主张服务端这一半;体感差异应是两者之和。未定论。
④ 速率列只作上界。rpc_per_s@window 的分子已改成同窗计数,但当某档计数为 0 时它不再是可比比率(判词里 speedup=n/a)。要做速率结论需要多轮重复并报中位数与离散度,本次每档只有一个窗口。未定论。
探针只用 node:http 与 /proc,不依赖任何第三方包;只读接口,不写状态。
# 本机某一户:读 $DSH_HOME/web-url.txt 的启动令牌自动换浏览器 Cookie
DSH_HOME=/home/user-d/.dsh node ops/dsh/upstream-issue-single-process-main-thread/main-thread-ceiling-probe.mjs \
--port 4002 --pid "$(pgrep -u user-d -f 'node_modules/.bin/dsh' | head -1)"
# 看曲线:K = 同进程并发 session/list 数(0 = 带仪表本底,8 = 压满)
DSH_HOME=/home/user-d/.dsh node ops/dsh/upstream-issue-single-process-main-thread/main-thread-ceiling-probe.mjs \
--port 4002 --pid 1017149 --levels 0,1,4,8 --seconds 8
# 判据行:main_pct_load ≥ 80 且 host_load1 ≪ host_cores
# ⇒ result=SATURATED(退出码 0)
# (rpc_ok_kN / speedup 只作旁证,不进门槛——main-thread-ceiling-probe.mjs:202-204:「not used as a gate」)
result | 退出码 | 含义 |
|---|---|---|
SATURATED | 0 | 复现成功:JS 主线程 ≥ 80% of one core,而宿主仍有空核(load1 < cores/2) |
NO_CEILING | 1 | 没测到天花板——宿主本身也忙、或该实例已把工作挪出主线程 |
| 握手 / 鉴权失败 | 2 | 读不到 web-url.txt、或令牌换不到 Cookie,判据无效 |
--levels 开很大。session/list 是实测 ~2 秒 / 1.36 MB 的全量磁盘扫描(#7/#8),本页 K=8 那一轮就已经让同进程的定开销请求样本数掉 51%——这正是本 issue 要说明的现象,对同事实例上的人来说就是「变卡了」。| 级别 | 修法 | 为什么是这条 / 怎么做 |
|---|---|---|
| P0 | 补 per-Host 可观测性 | 暴露进程 CPU、RSS、event-loop lag 与 GC 时间(perf_hooks.monitorEventLoopDelay 即可)。本仓与 DSH 现在都没有持续导出或被采集的这些指标(本页探针只现读 /proc 打一次表,无落库、无告警),所以这条天花板只能靠用户体感发现;先能看见,才谈容量规划。 |
| P0 | 给并发 agent 加限流 / 背压 | per-Host 的并发 agent 上限 + 明确的「排队中」状态。现在的行为是既不拒绝也不排队,只是大家一起变慢——用户无法区分「在跑」和「跑不动」。 |
| P1 | 把每个 follow 的 { global: true } 监听器换成按 id 订阅 | ⑩ 号锚点:这是超线性的主要来源(每事件派发约 O(跟随流数),进程总量约 O(会话数 × 跟随流数));⑪ 号锚点的每事件分配同样随跟随流数放大。改掉它是消除一项,而不是缩小一个常数。 |
| P1 | 提供 out-of-process(或 worker thread)subagent 驱动,并让 profile 可选 | 现在本随包只有 in-process 两种(② 号锚点)。上游在 ⑧ 号锚点那三处已经会用 new Worker(),把这套用到普通 subagent 路径,就能让「会话内扇出」不再吃掉其它会话依赖的那条线程。 |
| P1 | 把持久化屏障与压缩从交互路径上解耦 | ④⑤ 号锚点:每个 step、每个顶层工具调用都 await flush(),而那次 flush 做的是 open+stat+write+fsync+close 加 zstd,全部挤在共享的 4 条 libuv 线程上。两个进程 = 两套 fsync 队列。 |
| P2 | 文档化「进程 = 并行单位」,并提供官方分片方式 | 把容量模型写进文档,并给出把一个用户的会话分散到多个 Host 进程的受支持做法(本部署目前是每户一个 dsh-<user>.service,横向切分靠人工)。 |
下面是可直接粘贴的正文;行号按随包 lib/(0.1.5-rc.1)核对。
Title: Every session and every subagent in a DSH Host shares one JavaScript
main thread - the process, not the session, is the unit of parallelism
Summary
-------
A DSH Host process serves every session and runs every subagent on a single
JavaScript main thread. dsh-subagent-in-process-driver creates children with
parent.ctx.agents.create() inside the same process, and the shipped tree
contains no child_process-based subagent driver. Consequently:
* one Host cannot use more than about one core of JavaScript, no matter how
many cores the machine has;
* adding concurrency to one Host does not add throughput, it only adds queueing;
* latency for UNRELATED requests served by the same Host degrades as the
number of concurrent operations grows.
The practical consequence, which is why this is reported: running N sessions in
one Host is NOT equivalent to running N/2 sessions in each of two Hosts, even
at identical total load. Splitting into two Hosts yields a second JS thread.
Environment
-----------
DSH 0.1.5-rc.1 (shipped lib/, offline bundle @deepseek-ai/dsh)
Host Linux x86_64, 256 logical cores, load1 9.7 (about 4% utilised)
Instance dsh-user-d - dsh web --profile infra --host 127.0.0.1 --port 4002 (pid 1017149)
Corpus 2996 session directories on disk for this user
Node v24.16.0
Steps to reproduce
------------------
DSH_HOME=/home/USER/.dsh node main-thread-ceiling-probe.mjs \
--port 4002 --pid HOST_PID --levels 0,1,4,8 --seconds 8
load = K concurrent `session/list` RPCs (a real, unbounded on-disk scan)
probe = a tight keep-alive loop of `GET /api/ping`, a fixed-cost request that
reads no session state, so its latency is time the main thread was
not free for anyone
cpu = per-thread CPU from /proc/PID/task/TID/stat
Actual behaviour
----------------
K rpc_ok rpc/s@window canary_n p50 p99 p99.9 max >20ms main% other% proc%
0 0 0.00@8.1s 14547 0.44 1.13 9.37 92.9 3 69.3 5.0 74.3
1 2 0.25@8.1s 15268 0.43 0.92 7.17 272.0 6 87.9 50.0 137.9
4 4 0.50@8.0s 9754 0.62 1.62 9.75 831.9 7 98.4 87.3 185.7
8 0 0.00@8.0s 7464 0.90 3.30 20.02 93.0 8 98.2 104.4 202.6
MAIN_THREAD_CEILING main_pct_idle=69.3 main_pct_load=98.2
process_pct_load=202.6 host_cores=256 host_load1=9.72
rpc_ok_k1=2 rpc_ok_kN=0 ok_after_drain_kN=8 window_s=8 result=SATURATED
* The JS MainThread reaches 98.2% of one core, while the WHOLE PROCESS uses
202.6% - about 2.03 cores - of a 256-core machine at ~4% load. The excess
over one core is four V8Worker threads (GC/compiler) plus four libuv I/O
workers, not a second JS execution thread. Note the probe's own canary is
itself part of this load: with the canary running but no RPC load (K=0) the
thread already sits at 69.3%, against 4.7% main-thread (11.3% process) measured
separately on the truly idle instance. Either way the JS thread never exceeds
one core.
* Thread count is not fixed. The invariant core is ELEVEN threads
(MainThread + DelayedTaskScheduler + 4 V8Worker + SignalInspector +
4 libuv-worker); transient WorkerThreads appear on top while workflows or
run_code are active (11 / 11 / 13 observed on three instances at the same
moment). Nothing scales with session or subagent count.
* QUEUEING, measured without relying on any rate estimate: in an 8 s window,
K=1 completed 2 RPCs, K=4 completed 4, and K=8 completed ZERO inside the
window (8 more only after the drain). Per-RPC wall latency therefore grows
from about 4 s (K=1) to 4-8 s (K=4) to more than 8 s (K=8), while the
offered concurrency rises 8x. Throughput is flat; latency absorbs
everything. (An earlier revision of this report quoted a "2.63x speedup"
from a probe bug that divided a post-drain count by a pre-drain window;
the probe is fixed and that number is withdrawn.)
* A probe that touches no session state loses half its throughput as K rises,
K=1 -> K=8: 15268 -> 7464 samples per window (-51%), p99 0.92 -> 3.30 ms
(3.6x), p99.9 7.17 -> 20.02 ms (2.8x). One 831.9 ms stall was recorded at
K=4. K=0 is NOT an idle baseline: the canary loop runs at every level, so
with no session/list load it is the only load the probe injects. The 69.3%
is essentially the cost of serving those pings - 14547 in 8.1 s (1796/s),
about 0.39 ms of main-thread CPU each, under the K=0 row's own 0.44 ms
p50. This instance's own background load is the 4.7% main-thread figure
measured separately with no probe running. That is why the monotone arm used
above is K=1 -> K=4 -> K=8.
* Control, weaker than a controlled experiment and labelled as such: five
20 s canary runs against five instances, executed sequentially. The two
instances whose process CPU was ~0% showed max 15.8 / 16.9 ms and ZERO
samples above 20 ms; the instances carrying measurable CPU showed max
99.5 / 106.9 / 257.0 ms with 5 / 10 / 15 such samples.
Expected behaviour
------------------
Either (a) a Host should be able to use more than one core when several
sessions and subagents are active, or (b) DSH should document and expose the
process as the unit of parallelism, with a supported way to shard sessions
across Hosts, a per-Host concurrency limit / backpressure, and metrics that
make the ceiling visible.
Code references (0.1.5-rc.1, shipped lib/)
-----------------------------------------
dsh-subagent-in-process-driver/lib/index.js:161 startInProcessRun()
dsh-subagent-in-process-driver/lib/index.js:180 parent.ctx.agents.create()
dsh-subagent-in-process-driver/lib/index.js:211 await child.whenIdle()
dsh-subagent-spawn-in-process/lib/index.js:35 -> startInProcessRun
dsh-subagent-fork-in-process/lib/index.js:50 -> startInProcessRun
(the two in-process providers are the only ones registered in this bundle;
no child_process-based subagent driver is shipped here)
dsh-agent-loop/lib/index.js:1040-1043 for await (chunk) ... live.push
dsh-agent-loop/lib/index.js:1032 dispatch.emit("agent/assistant-stream")
- one frame per chunk, no coalescing
dsh-session-checkpoint-policy/lib/index.js:28 await ctx.sessions.flush(session)
dsh-session-checkpoint-policy/lib/index.js:64-73 flush awaited on llm/stream,
tools/execute and agent/pre-step
dsh-session-persistence-jsonl/lib/index.js:3046-3073 appendLines = open :3049,
stat :3057, write :3059,
fsync :3060, close :3054
(close runs at :3071 in the finally, after the write)
dsh-session-persistence-jsonl/lib/index.js:2176 DEFAULT_COMPRESSION = "zstd"
dsh-session/lib/index.js:989,1109,1176,1182 whole event log retained in memory,
deep copy x2 (data + surface metadata) + freeze per event
dsh-api-session-controller/lib/index.js:1419-1426 { global: true } listener on
session/event per follow stream
(:1420 filters by id only after
delivery; :1427-1435 session/created
is a second per-follow one;
:1436-1444 agent/assistant-stream,
same shape, same late id filter)
cordis/lib/index.js:263 filter(...).map(hook =>
hook.callback.bind(thisArg))
dsh-api-gateway/lib/index.js:346-349 JSON.stringify per socket
dsh-workflow-worker-thread/lib/index.js:311 new Worker(...) - one of the
off-thread paths that already exist
dsh-code-runtime-worker-thread/lib/index.js:737 new Worker(...) - run_code
dsh-session-persistence-jsonl/lib/index.js:1491 new Worker(...) - verification only
Ruled out (all measured)
------------------------
* Host saturation - 256 cores at load1 9.7 (about 4%). The 202.6% process CPU
is one saturated JS thread plus GC and I/O helper threads.
* Network / gateway - loopback direct, the per-user gateway is bypassed; p50
stays at 0.43-0.90 ms throughout.
* A frozen event loop - the canary still answers 7464-15268 times per 8 s
window across the K sweep (15268 at K=1, 7464 at K=8), and issue #9
separately measured the 2 s heartbeat timer not stalling (2000.0 ms idle;
2085 ms under 16 concurrent session/list calls, 2026-09-13 on the user-a
instance). The thread is SATURATED AND QUEUEING, not frozen.
* One slow request holding the loop - the K=1 level is a closed loop with at
most one session/list in flight, and there the canary p99 over the 8.1 s
window was only 0.92 ms (p99.9 7.17 ms), against about 2.0 s for one
session/list: three orders of magnitude apart, so the scan yields; the
cost is aggregate CPU, not one blocking call. Caveat: 0.92 ms is a
whole-window percentile, NOT a conditional percentile over a single
in-flight scan - the probe has no such mode, so "the tail stays low while
a scan runs" is an inference, not a measurement.
* A process-wide mutex - the per-turn path waits on no process-wide lock.
Persistence does hold a PER-SESSION kernel write lease
(dsh-session-persistence-jsonl/lib/index.js:652, SessionWriteLease; flock on
POSIX at :11/:690, a named kernel semaphore at :558 on Windows), but a
handle takes it once, on its first durable write (:248,
this.lease ??= await this.storage.acquireWriteLease(this.header)), and it
belongs to that one session, so it cannot queue turns across sessions.
Persistence is a true append with one write chain per session and a
200 ms batching window.
* Synchronous SQLite - disabled in this deployment (openAt: never); it is also
not on the per-turn path. Latent risk, not this cause.
* Per-token durable writes - format v3 coalesces token deltas into
text-chunks records; the durable write happens once per assistant/message.
Not yet proven
--------------
* How much MainThread time is GC versus request work. Issue #7 left the same
question open. Under load the four V8Worker threads total roughly 80% of
one core, consistent with active concurrent marking, but that does not
attribute the MainThread time.
* The two-process configuration. This report measures ONE process; the 2x2
comparison comes from the reporter's experience plus structural inference.
No two-process control run was performed.
* The browser side. The reporter's two configurations also differ in the
number of browser tabs, hence renderer processes and event loops.
Suggested fixes
---------------
1. Publish per-Host metrics: process CPU, RSS, event-loop lag, GC time.
Nothing in DSH or in this deployment exposes them today, so the ceiling
stays invisible until a user complains.
2. Add an out-of-process or worker-thread subagent driver and let the profile
choose it, so a fan-out in one session cannot consume the thread every
other session depends on.
3. Bound the fan-out: a per-Host concurrency limit / backpressure for
concurrent agents, with an explicit "queued" state instead of silent
degradation.
4. Replace the per-follow { global: true } listener with an id-keyed
subscription, so a session's events do not wake every other session's
followers. This removes a superlinear term rather than shrinking a constant.
5. Move the fsync + zstd persistence barrier off the interactive path, or make
the durability level configurable.
6. Document that the process is the unit of parallelism, and provide a
supported way to shard sessions across Hosts.
最后修改:2026-10-01 19:51:07 +08:00