Local LLM Inference on Compact AI Systems: Research Protocol

Operating-agreement preparation — 7 November 2026

A proposed operating agreement adds five unassigned accountability slots and six planned decision exercises covering scope, evaluation authorization, change, suspension, resumption and retirement. The guide and blank record appoint nobody and establish no adopted policy. Actual people, authority evidence, decisions, timestamps, objectives and approvals remain unfilled. Offline validation checks structure and blank evidence only; zero decision exercises executed.

• Proposed authority boundaries, preparation steps and evidence requirements
• Unassigned roles and unfilled decision-exercise record

The operating-agreement Design study describes the fictional service. Real participants, accepted mandates, justified objectives and attributable exercise records remain required. Earlier quality, permission, accounting, timing and recovery materials are preserved.

Recovery rehearsal preparation — 3 November 2026

A proposed isolated rehearsal adds five scenarios for compatible release restoration, missing artifacts, index mismatch, permission freshness and application readiness. None has been executed. Actual evidence fields remain blank.

• Recovery rehearsal checklist and evidence boundaries
• Blank recovery evidence record

Research question

Under which recorded workload and operating conditions can a compact local AI system deliver useful language-model performance? The proposed study examines the relationship between task quality, response latency, concurrent demand, memory use, and operating cost.

Planned experimental design

The investigation will select a reproducible model checkpoint with a suitable licence, record its exact revision, and freeze the serving environment. Hardware, operating system, driver, runtime, model format, quantisation, and serving options will be recorded for every configuration.

Workloads will vary input length, output length, and concurrent demand. Acceptance tasks and scoring rules will be defined before comparing configurations. Repeated runs will report latency distributions and failures alongside throughput and quality. Energy measurements will be included only if a suitable measurement boundary and instrument are available.

Measurement boundaries

Client-observed time to first token, generation latency, and end-to-end completion time will be distinguished. The vLLM documentation provides definitions for its benchmark measurements; any alternative tool will need an explicit mapping before results are compared. [1]

The evaluation will document warm-up, caching, request arrival patterns, timeouts, and unsuccessful requests. A single-user result will not be extrapolated into an unmeasured multi-user capacity claim.

Operational considerations

Data access, retained prompts, recovery, update controls, and ownership will be reviewed separately from model speed. NIST AI 600-1 is a risk-management reference for that review, not evidence that a particular system meets a certification or legal requirement. [2]

Release requirements

A later report must include actual measurements, reproducible commands, environment records, permitted evaluation inputs, limitations, and a contribution and AI-assistance statement. It will require human verification and technical review. No DOI, journal acceptance, peer review, or Google Scholar indexing is claimed for this protocol.

Preparation pack — 28 October 2026

The first preparation pack contains a proposed workload matrix, eight synthetic quality tasks, an unfilled environment manifest, and a scoring guide. Its file structure and consistency have been checked locally. It is not a completed benchmark, representative production dataset, frozen hardware configuration, or scholarly results release.

• Preparation guide and scoring rubric
• Proposed evaluation plan
• Unfilled environment manifest
• Eight synthetic quality cases

Hardware access, a verified model and environment, actual run records, representative permitted tasks, service objectives, and accountable quality review remain dependencies. The evaluation design study explains the plan and its limits.

Permission evaluation preparation — 30 October 2026

A supplemental plan develops the access-control boundary introduced by quality cases AM-RP-001-Q05 and Q06. It provides twelve synthetic scenarios covering allowed and denied retrieval, tenant isolation, missing identity or permissions, cache reuse, revocation, source links, stale access metadata, and authorization outages. Local validation checks the preparation files only. No scenario has been executed against an assistant.

• Permission evaluation guide and evidence boundaries
• Synthetic matrix and twelve planned scenarios
• Blank scenario evidence template

The permission-aware retrieval study explains the proposed flow. A test application, controlled identities, permission transitions, actual traces, and accountable review remain required. This addition reports no model-quality, hardware, security-effectiveness, or field result.

Request-accounting reference — 1 November 2026

A small offline reference now separates logical inference requests from physical attempts. Its fixed fictional fixture includes success responses, explicit error responses, unresolved attempts, and multiple success responses. A success response does not erase an unresolved sibling or establish answer quality. Synthetic software tests check the counting rules and invalid-input handling; they do not contact a model or an assistant.

• Accounting contract, limitations, and reproduction commands
• Offline Python accounting reference
• Fixed synthetic attempt fixture and expected counts
• Executable synthetic accounting tests

This is preparation for accurate failure reporting, not evidence that an agent recovers safely. Actual model runs, full attempt capture, declared observation cutoffs, quality review, and any business-effect reconciliation remain separate work. The agent failure essay explains why unknown outcomes must stay visible.

Latency-boundary reference — 2 November 2026

An offline timing reference now requires observations to match an independently preserved fictional arrival schedule. It distinguishes generator waiting, dispatch-relative latency and schedule-relative latency. Completed-only summaries name their sample counts and nearest-rank percentile convention, while timeouts, rejections, non-dispatched and in-flight work remain visible as counts. Software checks validate arithmetic and input rules only; no model, queue scheduler or load generator is executed.

• Timing contract, populations, reproduction commands and limits
• Offline Python latency-boundary reference
• Six fictional scheduled requests and timing observations
• Executable synthetic timing tests

The queue-test Design study describes a proposed later experiment. Hardware access, installed-version verification, justified arrivals, actual traces, repeated measurements and accountable quality review remain dependencies. The original capacity matrix and all earlier preparation files are preserved.