Loading results from the API… If the hosted API was asleep, this can take up to a minute.
Loading results from the API… If the hosted API was asleep, this can take up to a minute.
R5: prefix caching disabled in the serving config
Live runs are turned off on this public site. The results below are recorded; live runs happen on the owner's runner.
Recorded in RQ3 (2026-10-06). Its perf-gate figures predate the gate's warmup fix; see the corrections; the re-measured numbers are in the repair timeline below.
| check | selected by Guard | result | s | detail |
|---|---|---|---|---|
| unit:tests/test_retriever.py | no: exercises nothing this change touches | pass | 1.8 | 2 passed in 0.55s |
| unit:tests/test_prompts.py | no: exercises nothing this change touches | pass | 1.7 | 2 passed in 0.58s |
| check:citation_required | yes | warn | 9.8 | citation contract met in 0/34 answers on the PR vs 2/34 on base (-5.9 pts; budget -5 pts; not significant at n=34, one-sided Fisher p=0.25); via full answer path (incl. post-processing) |
| judge:ungrounded_answer | no: exercises capability:route:app/main.py::POST /chat::rag_answer, but the change (serving_config) is not one it detects | skip | 0.0 | no judge model key configured (BYOK); judge not run |
| check:prompt_prefix_stable | no: exercises nothing this change touches | pass | 3.8 | requests share 3954 leading characters (base 3954, need 3558) |
| check:context_budget | no: exercises nothing this change touches | pass | 7.8 | largest prompt 980 tokens (base 980) against a budget of 3840; median 964 vs 964 |
| security:approval_gate | yes | pass | 1.5 | 1 approval test(s) (generated from the base revision): 1 passed in 0.31s |
| bench:chat_perf_gate | yes | pass | 45.4 | median p95 TTFT 234.5 -> 195.9 ms (-16.4%, pass; 3 runs each, runs overlap); prefix-cache hit 98% -> 98% |
# No regressions in 3 targeted checks
conclusion: neutral
**Ran 3 of 8 suite items** selected from the behavior-to-code graph. Change categories: serving_config.
### Quality
| check | result | detail |
|---|---|---|
| `check:citation_required` | **WARN** | citation contract met in 0/34 answers on the PR vs 2/34 on base (-5.9 pts; budget -5 pts; not significant at n=34, one-sided Fisher p=0.25); via full answer path (incl. post-processing) |
### Performance
| check | result | detail |
|---|---|---|
| `bench:chat_perf_gate` | **PASS** | median p95 TTFT 234.5 -> 195.9 ms (-16.4%, pass; 3 runs each, runs overlap); prefix-cache hit 98% -> 98% |
### Security
| check | result | detail |
|---|---|---|
| `security:approval_gate` | **PASS** | 1 approval test(s) (generated from the base revision): 1 passed in 0.31s |
<details><summary>Not run for this change</summary>
- `unit:tests/test_retriever.py`: exercises nothing this change touches
- `unit:tests/test_prompts.py`: exercises nothing this change touches
- `judge:ungrounded_answer`: exercises capability:route:app/main.py::POST /chat::rag_answer, but the change (serving_config) is not one it detects
- `check:prompt_prefix_stable`: exercises nothing this change touches
- `check:context_budget`: exercises nothing this change touches
</details>
### What changed in the graph
- `config_key:docker-compose.yml#llm.--enable-prefix-caching`: `exists` True -> False
- `config_key:docker-compose.yml#llm.--no-enable-prefix-caching`: `exists` False -> True
- `serving_config:docker-compose.yml::llm`: `flags` {'served-model-name': 'lab', 'gpu-memory-utilization': '0.70', 'max-model-len': '4096', 'max-num-seqs': '32', 'enable-prefix-caching': True} -> {'served-model-name': 'lab', 'gpu-memory-utilization': '0.70', 'max-model-len': '4096', 'max-num-seqs': '32', 'no-enable-prefix-caching': True}