Answer first
On 25 October 2026 we do not know whether the planned changes will beat the current DSHREM architecture. That is the honest position, and this page is written to make it checkable rather than rhetorical. It pins the starting point, names what is still unknown, states which changes are being tested, and records in advance what result would prove each change useless.
Two rules govern the record. A number appears only with a source and a date. An unmeasured value is written as unknown, never as zero. Where a statement is a forecast it is labelled a forecast and paired with the observation that would refute it.
What is measured on 25 October 2026
Seven self-service service accounts are registered, and the public domain opened on 18 October 2026. This counts accounts, not people, and excludes marked and provisioned QA accounts. The editor extension is at version 0.10.9 with a successful CI run. These are adoption and release counts.
The dated runtime verification receipt of 24 October records 61 passing runtime regression checks, 32 knowledge checks, 4 authenticated browser finish checks, 6 of 6 public HTTPS flows and 821 editor checks. The same receipt records 7 settled paid coordinator calls on the low-cost route at an observed provider cost of 0.0026731264 USD with nothing pending. The first publicly receipted paid calls on 18 October were two attempts totalling 0.00016842 USD; one draft was rejected by a pre-declared whitelist and the accepted correction was made by hand.
What these numbers describe is a service that works, is tested and is accounted for. They do not describe how fast it works, how much a finished coding task costs, or whether one editor host is better than another. Confusing the two is the error this record exists to prevent.
What is not measured
Not measured at this date: cost per successful coding task, p50 and p95 wall-clock latency, input and output tokens per task, provider-reported cache hit rate, peak resident memory per task, CPU seconds per task, error rate by task class, paid retries per accepted task, and any relative advantage of one editor host over another.
The machine-readable list is published as baseline JSON together with this article, under the same correction rule. A missing value there means exactly that: nobody has measured it yet, and no claim is being made from silence.
How each metric is defined
A successful task is one whose acceptance check was declared before the run, passes on the produced artifact, and leaves a recorded receipt. A task that "looks good" without a declared check counts as not accepted. Cost per successful task divides all provider and service cost of the task, including failed attempts, by the number of accepted tasks in the set. This definition is chosen so that an approach cannot look cheap by failing often.
Latency is wall-clock time from the first request to the final artifact, including retries and waiting for a model, measured at the client so that provider-side time cannot disappear. Cache hit rate is the provider-reported share of cached input tokens; if a provider does not report it, the rate is unknown and savings are not claimed. Peak memory and CPU seconds are read from the host control group while one task runs alone, because shared measurement on a busy host produces decorative digits.
Retries are paid attempts beyond the first one for the same task and are counted, not hidden. Error rate is split by task class, because a single aggregate hides the difference between a failed rename and a failed multi-file refactor.
The changes under test
The programme under test covers six technical changes. First, semantic code intelligence through a language server, reusing an existing equivalent before adding a new dependency. Second, a compact repository map that replaces whole-file context with a structural summary. Third, structural editing of code instead of free-text patching. Fourth, prompt caching together with context compaction. Fifth, loading tools on demand instead of offering a long catalogue to every request. Sixth, an audit of routing and escalation, including the cost of repeated requests and the rule that sends work to a more expensive model.
Three further items are research, not commitments. A reproducible benchmark on thirty real tasks. A comparison with lighter agent runtimes, including the editor-host question. And a study of automatic workflow optimisation, which is read before it is installed. Local model inference is not part of this round: without an accelerator the full cost, including hardware, energy and operator time, would have to be proved lower before it is adopted, and no such proof exists yet.
The method is fixed in advance. Find an existing analogue in our own code before adding a library. Test candidates, keep the better one, and introduce it behind a feature flag. Each change lives on its own branch with tests and a commit. No production deployment, no merge into the main line and no paid operation happens without separate permission.
Hypotheses and refutation thresholds
H1, code intelligence: navigation and symbol lookup reduce input context on multi-file work. Keep it if input tokens per accepted task fall by at least 15 percent with no quality loss; drop it below 5 percent. H2, repository map: a compact structural summary replaces whole-file context on repository-level tasks. Keep it at a 20 percent reduction or more; drop it below 8 percent. H3, structural editing: fewer failed patch attempts, concentrated in edits that touch several places. Keep it at 20 percent fewer failures; drop it below 5 percent.
H4, caching and compaction: cost falls only where the prefix is really reused. Keep it where the provider reports cached tokens and the task cost falls by 10 percent or more; where the provider reports no cached tokens, the measured effect is zero and the claim is withdrawn. H5, on-demand tools: fewer input tokens with no rise in wrong-tool selection. Keep it at 10 percent fewer tokens; drop it if wrong-tool selection rises. H6, routing audit: every escalation to a more expensive model carries a logged reason, and repeated identical work does not pay twice. Keep it at 20 percent lower cost on repeated tasks; drop it if escalation cannot be attributed.
A hypothesis that fails does not get a second, softer threshold. It is recorded as refuted with its measurement, and the work stops there.
Method: thirty tasks, two arms
The benchmark is thirty real tasks from this project, chosen and frozen before any candidate is measured: ten small edits, ten multi-file changes, five failing-test repairs, five knowledge or documentation tasks with a checkable result. Each task names its repository revision, its acceptance check and a reference outcome, so a later reader can replay it without trusting the author.
Each task runs twice: once on the current architecture and once with the candidate flag enabled, with the same model identifiers and the same task ceiling. The order alternates, so a warm cache or an unlucky hour cannot consistently favour one arm. Cost is read from the metering ledger by task identifier, not from a model describing its own bill.
A comparison is published only when both arms completed the same task set on the same day with the same model identifiers. If that condition fails, the result is labelled not comparable instead of being presented with a percentage.
Controls and confounders
Five confounders are recorded for every run: provider price changes, cache warmth, subscription quota against metered API, machine load on the shared host, and silent model version drift. A sixth is the observer. The author of the change also measures it, which is a real bias even when the intent is honest.
The mitigation is procedural rather than rhetorical: thresholds are fixed before the run, raw artifacts are kept, negative results are published, and any run where a confounder could not be excluded is marked as such in the report.
Decision dates and checkpoints
1 November 2026: the thirty-task set is frozen and the harness produces its first baseline run. 8 November 2026: checkpoint on code intelligence and the repository map. 24 November 2026: decision on structural editing, caching and on-demand tools, one verdict per hypothesis. 24 December 2026: the before-and-after report, including the editor-host comparison and a list of what was rejected. 23 January 2027: programme review that either continues the surviving items or closes the programme with an autopsy.
These are decision dates, not delivery promises. A date can pass with the honest verdict "not measured, because the harness was not ready"; it cannot pass with an invented number.
Acceptance checklist
The programme is acceptable when each item is either pass or explicitly not run with a reason. The baseline dataset responds with a stable digest and every measurement in it carries a source and a date. Every hypothesis has a threshold fixed before measurement and a stated kill rule. All thirty tasks are replayable from pinned revisions. Every paid call is attributable to a task identifier in the ledger. No paid work starts without a declared ceiling. The final report names its commit revisions and publishes what did not work.
Kill criteria
The programme stops in four situations. If thirty tasks show no measured reduction in cost per successful task, the growth path is closed and only zero-cost structural items remain. If a candidate causes a quality regression that is not repaired, its flag is reverted rather than compensated by a cheaper model. If a spend is uncertain or a reservation is unreconciled, paid work freezes until a named operator reconciles it; unknown is not zero. If a published claim loses its artifact, the claim is withdrawn in public on this page.
Forecasts and what would break them
Forecast, code intelligence and repository map: input tokens on repository-level tasks fall by 15 to 35 percent, with no effect on single-file edits. Refuted if the measured reduction is below 5 percent, or if acceptance quality falls. Forecast, structural editing: failed patch attempts fall by 20 to 40 percent, concentrated in edits touching several places; refuted if failures are simply redistributed to a later phase.
Forecast, caching: cost falls by 5 to 20 percent only where prefix reuse is real, and the effect is exactly zero where the provider reports no cached tokens. Forecast, on-demand tools: input tokens fall by 5 to 15 percent, with a risk of wrong-tool selection that would erase the gain. The programme target of minus 20 to minus 30 percent cost per successful task is a goal under test, not a prediction that will be met.
Two forecasts are negative and stated as such. These changes are not expected to improve p95 latency, because latency is bound by the provider rather than by our client. Local inference is not expected to win on a host without an accelerator once hardware, energy and operator time are counted. A negative forecast that holds is a useful result, not a failure.
Editor comparison: VSCodium with MCP against the VS Code extension
The common question is whether a build of the coding workspace inside VSCodium with a local MCP server is more effective than the existing VS Code extension working through the hosted gateway. As posed, the question cannot be answered, because it changes two things at once: the editor host and the transport. A difference in cost could come from either.
The design is therefore a two-by-two comparison on the same thirty tasks: VSCodium with the local server, VSCodium with the hosted gateway, VS Code with the local server, VS Code with the hosted gateway. Same model identifiers, same task ceiling, alternating order. Measured: cost per successful task, p50 and p95, setup time to first accepted task, peak memory of the editor process, and failure modes rather than only failure counts.
The recorded prediction is narrow: model cost should be the same, because the model is the same, while differences should appear in setup time, memory and failure modes. That prediction is written here so that it can be refuted by measurement rather than defended afterwards.
Threats to validity
The sample is small: thirty tasks, one operator, a young service with seven registered accounts. The tasks were chosen by the same person who builds the improvement, which makes them easier than the real distribution. Providers change prices and model versions without asking. A shared host adds noise. A result measured in November may not hold in February.
These threats are not removed by confidence. They are bounded by pre-registered thresholds, published artifacts, negative results and a fixed replay set, and they are repeated in every report so that a reader can discount the numbers by the right amount.
What is published and what is not
Published: the method, the metric definitions, aggregate counts, verdicts, dead ends and the revisions behind them. Not published, ever: credentials, the money path, client data and personal contact data, and any number that has no artifact behind it. A figure that cannot be checked is not published with a hedge; it is left out until it can be checked.
The reason is practical. A published percentage enters other people's decisions. If it cannot be traced back to an observation, it does harm long after the author has moved on.
How this record is revised
The baseline is never rewritten. Later results are added as dated revisions below the original text, and a correction is written next to the claim it corrects. The verification date at the top of the page means that the sources were checked on that day; it does not mean that anything improved.
The machine-readable companion is the baseline dataset published beside this page. It carries the same numbers, the same unknowns and the same correction rule, so an automated reader and a human reader cannot be told different things.