# AM-RP-001: latency-boundary arithmetic reference

Author: Anant Mishra. Prepared 2 November 2026. Version 0.1.0.

**Status: synthetic timing arithmetic and software tests only. No model, vLLM server, DGX machine, admission queue, load generator or application is executed.** This is reproducibility preparation, not a benchmark, queue simulation, field observation, representative demand trace, or evidence of service performance.

## Reproduce

Download `latency_boundaries.py`, `test_latency_boundaries.py`, `synthetic-latency-boundaries.json`, and this guide into the same directory. Use Python 3.10 or later; there are no third-party dependencies:

```sh
PYTHONDONTWRITEBYTECODE=1 python3 latency_boundaries.py
PYTHONDONTWRITEBYTECODE=1 python3 test_latency_boundaries.py -v
```

The CLI accepts labelled synthetic fixtures and prints `synthetic_timing_arithmetic_only`. Test assertions verify authored arithmetic and input rules. They do not collect performance evidence. The existing request-accounting, quality, permission and hardware-plan files remain unchanged.

## Clock and record contract

All offsets are non-negative finite numbers in **milliseconds on one monotonic clock**, relative to a chosen test origin. They are not UTC timestamps. `scheduled_ms` is the load generator's intended arrival; it is not automatically a user's button-click time. This reference cannot detect mislabeled units or prove that clocks were actually shared; instrumentation must establish those facts before analyzing real records.

Supply a separate `schedule` and `observations` list, plus `observation_cutoff_ms`. Their unique request-ID sets must agree exactly. Missing observations, unplanned records and repeated IDs are rejected. This prevents a missing scheduled request from silently disappearing from the report, provided the input schedule itself is complete and independently preserved.

Schedule records contain exactly `request_id` and `scheduled_ms`. Observation records contain exactly `request_id`, `status`, `dispatch_ms`, `first_output_ms`, and `end_ms`. First output means a qualifying stream event under a declared instrumentation rule; it does not establish token count, useful content or correctness. Allowed states are:

| State | Required evidence in this small reference |
| --- | --- |
| `completed` | Dispatch, first output and actual completion; these events must occur in order. |
| `timeout` | Dispatch and client observation-end event; first output may exist. The remote result is not inferred. |
| `rejected` | Dispatch and explicit rejection event; no qualifying output under this contract. |
| `not_dispatched` | No dispatch or output; `end_ms` records a generator decision/observation before or at cutoff. |
| `in_flight` | Dispatch occurred, but completion is unresolved; observation ends exactly at cutoff. First output may exist. |

For non-completed states, `end_ms` is an observation boundary, never an invented completion. A request still running at the cutoff must not be relabelled as timed out without a real client timeout event. Client cancellation, explicit server errors, retries, warm-ups and revised observations need separately defined records or a versioned extension; do not force them into misleading categories.

## Intervals and sample populations

For each dispatched record:

- Generator waiting = dispatch minus scheduled arrival. This can include waiting for a client concurrency slot or generator delay. It is not server queue time.
- Client-observed duration = observation end minus dispatch. For a timeout, this is the observed waiting period, not completion latency.
- Client TTFT = first qualifying output minus dispatch, if that output was observed.
- Schedule-relative TTFT = first qualifying output minus scheduled arrival, if observed.
- Client E2E and schedule-relative E2E are calculated **only for completed records**.

Non-dispatched intervals remain null. The report preserves a partial timeout's first-output observation but includes **only completed requests** in every aggregate latency group. Every group names its sample count; counts for all five states appear alongside it. Do not interpret these conditional percentiles as an all-arrival service percentile.

This reference chooses nearest-rank percentiles: sort `n` samples and select the one-based rank `ceil(p × n)`. It uses no interpolation. Empty populations have null mean/p50/p95 and `n=0`. Means are rounded to six decimal places for JSON presentation; retain higher-resolution raw instrumentation independently. p50 under this convention can differ from an interpolated median. Never average run-level percentiles into a purported pooled percentile.

The fixture has six scheduled records: three completed, one client timeout, one rejection and one not dispatched. There are zero in-flight records in this invented snapshot. L01 is scheduled at 0, dispatched at 1,800, first observed at 2,400, and completed at 3,000 ms. Dispatch-relative TTFT is 600 ms; schedule-relative TTFT is 2,400 ms. The conditional groups have only three samples, so nearest-rank p95 selects their maximum. These numbers illustrate boundaries; they do not estimate a tail distribution or demonstrate capacity.

## Primary-source mapping and limits

Reviewed 2 November 2026:

- vLLM's current benchmark guide distinguishes client queue time and schedule-relative E2E under finite arrival rates: https://docs.vllm.ai/en/latest/benchmarking/cli/
- Grafana k6 explains open versus closed arrivals: https://grafana.com/docs/k6/latest/using-k6/scenarios/concepts/open-vs-closed/
- vLLM production metrics describe server waiting and generation phases: https://docs.vllm.ai/en/latest/usage/metrics/

These are references for boundary definitions, not validation of this fixture. This code is not a parser for any vLLM result schema and does not configure current CLI flags. Verify installed help, client/runtime versions, saved fields, event meanings, observation cutoff, actual token counts and collection overhead before a real experiment. Server timing needs independently captured evidence; subtracting aggregate percentiles cannot reconstruct request-level components.

Next dependencies remain authorized hardware access, frozen environment/model revisions, a justified schedule, representative permitted tasks, actual traces and failures, sufficient repetitions and accountable quality review. The queue Design study and prior permission scenarios are unexecuted. AI assistance was used in preparation; no empirical paper, peer review, identifier or Scholar indexing is claimed.