This technical whitepaper proposes Layered Cluster-Compute Architecture, or LCCA, a cost-first and verification-aware lifecycle for reducing repetitive large language model inference.
The central claim is not that caching, routing, retrieval, or tool use are new. Rather, LCCA organizes these familiar components into a governed lifecycle for routine AI work: detect repeated workloads, cluster them, route them, verify them, compile them into reusable cluster-kernels, register them with ownership and rollback, monitor them for drift, and retire or rebuild them when necessary.
A cluster-kernel is defined as a versioned, callable, testable, and auditable routine compiled from repeated verified traces. LCCA is designed for high-frequency, stable, bounded, and verifiable workloads such as metadata generation, document engineering, support routing, standard code scaffolding, repository descriptions, schema transformations, low-risk summarization, and recurring tool workflows.
The primary evaluation metric is cost per verified useful output. Secondary metrics include Wh per verified output, TOTEN per verified output, kernel hit rate, frontier escalation rate, verification pass rate, fallback success rate, latency, and pilot risk magnitude. TOTEN is defined as Total Token-Equivalent Number and accounts for routing prompts, verifier prompts, retrieval context, tool instructions, fallback calls, PromptOps activity, night-cycle distillation prompts, and other token-equivalent overhead.
The architecture is sidecar-first rather than replacement-first. It is intended to be tested through offline replay, shadow mode, read-only assist, and limited canary rollout before any production traffic is affected. It also includes PromptOps cards, allowing AI assistants to help engineers perform workload analysis, kernel candidate mining, cost estimation, verifier design, dashboard configuration, and night-cycle distillation under human supervision.
Version v0.4 introduced production-edge safeguards for extreme boundary states, including asymmetric low-cost verifier policy, verifier tax accounting, gateway latency budget, fast-path short-circuit routing, fractional canary probing for kernel drift, in-flight request coalescing for burst new loads, probabilistic jittered degradation for hot-kernel eviction, stale-while-revalidate, stale-if-error, and capacity-aware frontier leakage control.
Version v0.5 adds logic-closure and flow-accounting patches. It clamps probabilistic leakage into the valid probability range, separates predicted route quantities from posterior verification results, treats gateway cost as traffic-mix dependent, separates initial dispatch fractions from final mutually exclusive service-route fractions, accounts for probe calls as side costs, and adds nonlinear pilot-risk modeling options for production-grade risk assessment.
LCCA does not claim to replace foundation models, frontier reasoning, retrieval systems, caches, routers, or tool-calling frameworks. It proposes a practical engineering lifecycle for reducing repetitive LLM inference while reserving frontier models for genuinely novel, uncertain, or high-value tasks.
The guiding motto is: compile the routine; reserve the frontier.
本技术白皮书提出了一种名为“分层集群计算架构”（Layered Cluster-Compute Architecture，简称 LCCA）的方案，旨在通过一种优先考虑成本并具备验证意识的生命周期管理机制，减少大型语言模型（LLM）的重复推理工作。
其核心主张并非宣称缓存、路由、检索或工具使用等技术是全新的，而是强调 LCCA 将这些既有组件整合进一个受控的生命周期流程中，以处理常规 AI 任务：识别重复的工作负载，对其进行聚类、路由和验证，将其编译为可复用的“集群内核”（cluster-kernels），注册所有权与回滚机制，监控性能漂移，并在必要时将其停用或重建。
“集群内核”被定义为一种经由重复且已验证的执行轨迹（traces）编译而成的程序单元，具备版本控制、可调用、可测试及可审计的特性。LCCA 专为高频、稳定、边界明确且可验证的工作负载而设计，例如元数据生成、文档工程、支持请求路由、标准代码脚手架构建、代码库描述生成、模式（schema）转换、低风险摘要生成以及周期性工具工作流等。
主要的评估指标是“每次经验证的有效输出所产生的成本”。次要指标包括：每次经验证输出的能耗（Wh）、每次经验证输出的 TOTEN 值、内核命中率、前沿模型升级率（frontier escalation rate）、验证通过率、回退成功率、延迟以及试点风险量级。TOTEN 定义为“总 Token 等效数”（Total Token-Equivalent Number），涵盖了路由提示词（prompts）、验证器提示词、检索上下文、工具指令、回退调用、PromptOps 活动、夜间蒸馏提示词以及其他 Token 等效开销。
该架构采用“Sidecar（边车）优先”而非“替换优先”的设计理念。在影响任何生产流量之前，它会先通过离线回放、影子模式、只读辅助及有限的金丝雀发布（canary rollout）进行测试。此外，该架构还包含 PromptOps 卡片功能，允许 AI 助手在人工监督下协助工程师进行工作负载分析、内核候选挖掘、成本估算、验证器设计、仪表板配置以及夜间蒸馏任务。
v0.4 版本引入了针对极端边界状态的生产边缘防护机制，包括：非对称低成本验证器策略、验证器成本核算、网关延迟预算、快速路径短路路由、针对内核漂移的分段金丝雀探测、针对突发新负载的请求合并（in-flight request coalescing）、针对热点内核淘汰的概率性抖动降级策略、Stale-while-revalidate（过期时重新验证）与 Stale-if-error（出错时使用过期数据）机制，以及感知容量的前沿模型溢出控制。
v0.5 版本增加了逻辑闭包（logic-closure）与流程核算（flow-accounting）相关的补丁。它将概率性泄漏（probabilistic leakage）限制在有效概率范围内，将预测的路由数量与后验验证结果分离开来，将网关成本视为取决于流量组合（traffic-mix）的变量，区分初始调度比例与最终互斥的服务路由比例，将探测调用（probe calls）计为额外成本，并增加了用于生产级风险评估的非线性试点风险建模选项。
LCCA 并不旨在取代基础模型、前沿推理、检索系统、缓存、路由器或工具调用框架。它提出了一种实用的工程生命周期方案，旨在减少重复性的大语言模型（LLM）推理，同时将前沿模型留给真正新颖、具有不确定性或高价值的任务。
其指导原则是：常规流程“编译化”，前沿能力“预留化”。
发表于 2026-7-4 15:28
收藏
分享