Agent Runtime (@nsys-cortex/runtime)

The agent runtime transforms Nsys Cortex from a sandbox orchestration platform into an autonomous AI agent. It provides the LLM integration loop, response streaming, context window management, loop detection, and session persistence.

Architecture

User Message
     │
     ▼
┌───────────────────────┐
│   CortexAgentLoop     │
│                       │
│  1. Load history      │
│  2. Build prompt      │
│  3. Stream to LLM     │──────▶ LLM Provider (Anthropic / OpenAI / Gemini / Grok / Mistral / Ollama / vLLM / LM Studio / SGLang)
│  4. Collect response  │◀──────
│  5. Execute tools     │──────▶ Tool Registry (27 tools)
│  6. Inject results    │◀──────
│  7. Repeat or stop    │
└───────────────────────┘
     │
     ▼
SSE Stream / JSON Response

LLM Providers

Nine providers are included: Anthropic, OpenAI, Gemini, Grok, Mistral, Ollama (local CLI runtime), vLLM and SGLang (self-hosted GPU servers), and LM Studio (desktop GUI runtime). The provider is selected via NSYS_CORTEX_LLM_PROVIDER (default: anthropic). For the local providers, see Ollama vs vLLM vs LM Studio — which to pick before deciding (SGLang is a vLLM alternative — covered in its own section below).

Anthropic Claude

import { AnthropicProvider } from '@nsys-cortex/runtime';

const provider = new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY });

Supported models (Oct 2026): claude-opus-5-5 (the current Opus and the default — $4 / $20 per MTok), claude-fable-5-1 (Anthropic's most capable model, Fable-tier pricing), claude-sonnet-5-5, claude-haiku-5-5, claude-opus-5, claude-sonnet-5, claude-fable-5, and the 4.x line (claude-opus-4-8, claude-opus-4-7, claude-opus-4-6, claude-sonnet-4-6, claude-sonnet-4-5, claude-haiku-4-5-20251001). Any future Anthropic model also works — the provider passes the model string through verbatim, picks the request shape by generation (anthropicRequestShape(): no temperature on 4.6+, adaptive thinking with an effort level, output_config.format for structured outputs where forced tool use is rejected), and reads the budget from MODEL_CONTEXT_BUDGETS (falls back to DEFAULT_CONTEXT_BUDGET for unknown names). On Opus 5.5, Fable and the other 5.x models thinking is always on (adaptive; the provider never sends disabled), and a safety classifier may decline a request with stop_reason: refusal — the provider surfaces that as an error event naming the category instead of a blank reply.

Anthropic-specific features wired through LLMProviderOptions:

• Prompt caching — cache_control: { type: 'ephemeral' } is applied to the system prompt, the last tool definition, and the 2nd-to-last user message (see Prompt Caching).
• Extended thinking — set thinking: { type: 'enabled', budgetTokens } to have the model emit reasoning blocks before its response (on 4.6+ the budget becomes output_config.effort under adaptive thinking; on the pre-4.6 models it is sent as budget_tokens). The provider's stream loop translates the SDK's content_block_start(thinking) + thinking_delta + signature_delta + content_block_stop sequence into runtime-level thinking_delta events and a final thinking_end event carrying the full text plus Anthropic's HMAC signature. The same thinking option is honored by the OpenAI provider (maps to reasoning_effort) and Gemini provider (maps to thinkingConfig.thinkingBudget); see Extended Thinking below for the full per-provider pipeline.

OpenAI-Compatible

import { OpenAIProvider } from '@nsys-cortex/runtime';

const provider = new OpenAIProvider({
  apiKey: process.env.OPENAI_API_KEY,
  baseUrl: 'https://api.openai.com/v1',
});

Supported OpenAI models (Oct 2026): gpt-6.1-sol (the default — "near-Astra performance for complex work at a lower cost", 1.05M context, $2 / $10 per MTok), gpt-6-astra (OpenAI's flagship, $10 / $50), gpt-6.1-terra (fast), gpt-6-luna (most efficient), and the still-served gpt-5 / gpt-5-mini / gpt-4o family. The GPT-6 and GPT-5 lines and the o-series are reasoning models: the provider never sends temperature to them (openaiSendsTemperature()) and maps --think to reasoning_effort; gpt-4o and the OpenAI-compatible siblings (Grok, Mistral, the local runtimes) keep receiving temperature.

Gemini (Google)

import { GeminiProvider } from '@nsys-cortex/runtime';

const provider = new GeminiProvider({ apiKey: process.env.GEMINI_API_KEY });

Uses raw fetch() + SSE parsing against the Gemini streamGenerateContent API. Supported models (Oct 2026): gemini-4-argon (the default — Google's flagship reasoning model, Stable, 1M context), gemini-3.8-flash (the current Flash, Stable, 1,048,576 context), gemini-3.6-flash, gemini-3.5-flash-lite, gemini-3.1-pro-preview (still a preview; gemini-3.1-pro is not a real id), and the limited-access gemini-2.5-pro / gemini-2.5-flash (retiring); the 2.0 line is shut down. On Gemini 3.x and 4 the provider sends no temperature (Google: remove the sampling parameters from every 3.x request) and maps --think to thinkingConfig.thinkingLevel (geminiThinkingStyle()); on 2.5 it sends temperature and thinkingBudget as before.

Grok (xAI)

Reuses OpenAIProvider with the xAI base URL (https://api.x.ai/v1). Set NSYS_CORTEX_LLM_PROVIDER=grok and XAI_API_KEY. Supported models (Oct 2026): grok-4.7 (current flagship, 500k context, $2 / $6 per MTok), grok-4.6, grok-4.5 (500k), grok-4.3 (1M, cheaper); grok-4 and grok-3 stay registered while xAI serves them. Grok keeps receiving temperature — xAI's model page says nothing against it.

Mistral

Reuses OpenAIProvider with the Mistral base URL (https://api.mistral.ai/v1). Set NSYS_CORTEX_LLM_PROVIDER=mistral and MISTRAL_API_KEY. The default model is mistral-large-latest — Mistral's own moving pointer to the current Large flagship; pin to a dated id (mistral-large-4 is Large 4, public preview since 2026-10-06 with a 1M window; mistral-medium-3-5, mistral-small-2603) only if you need reproducibility across their version cuts. mistral-large-2411 is in Mistral's deprecated table.

Registered models in MODEL_CONTEXT_BUDGETS: mistral-large-4 (1M), mistral-large-latest / mistral-medium-3-5 / mistral-medium-latest / mistral-small-2603 / mistral-small-latest (256k — the pointers are budgeted at the smallest model they can now resolve to), mistral-large-2411 / ministral-8b-latest / ministral-3b-latest (128k), and codestral-latest (256k, code-specialist). Any other model string passes through to the API verbatim and falls through to DEFAULT_CONTEXT_BUDGET (128k/4096).

Limitations vs the other OpenAI-compat providers:

• No prompt caching — Mistral's API does not expose a prompt_cache_key field. cacheReadTokens will always be 0 on this provider.
• No reasoning surface — Mistral's reasoning lineup (Magistral) uses a non-OpenAI-compat API, so --think is silently dropped here. Calling code that sets thinkingBudgetTokens won't error; the budget just has no effect.
• Function calling — fully supported via OpenAI-compat tools / tool_choice; the agent loop and 27-tool registry work unchanged.

Ollama (Local)

Reuses OpenAIProvider with Ollama's OpenAI-compatible endpoint (http://localhost:11434/v1). Set NSYS_CORTEX_LLM_PROVIDER=ollama and NSYS_CORTEX_LLM_MODEL to whichever model name you ollama pull locally (defaults to llama4 — Llama 4 Scout, Ollama's llama4:16x17b build, 67 GB, the last open Llama Meta released; llama3.3 at 43 GB remains the lighter pick, llama4:maverick the 245 GB larger one). No API key required. Override the base URL with OLLAMA_BASE_URL.

vLLM (Local / Self-Hosted)

Reuses OpenAIProvider with vLLM's built-in OpenAI-compatible HTTP server (vllm serve, default http://localhost:8000/v1). Set NSYS_CORTEX_LLM_PROVIDER=vllm and NSYS_CORTEX_LLM_MODEL to the id you started vLLM with — the Hugging Face id, or the --served-model-name alias when the server sets one (defaults to Qwen/Qwen3.8-27B, the BF16 release, ~54 GB; Qwen/Qwen3.8-27B-FP8 is ~28 GB, and a GPTQ-Int4 build such as sokada4/Qwen3.8-27B-GPTQ-Int4 (~18 GB) fits one 24 GB Ampere card — the KB's two-A10 box serves it as qwen3.8-27b; 262K native context, thinking on by default; the default config's temperature is 1, Qwen's thinking-mode sampling). vLLM has no auto-pull — the model must already be loaded on the server; set NSYS_CORTEX_LLM_CONTEXT_TOKENS to its --max-model-len when that is below the model's window. API key is optional and only enforced if vLLM was started with --api-key; set VLLM_API_KEY to mirror that. Override the base URL with VLLM_BASE_URL.

SGLang (Local / Self-Hosted)

Reuses OpenAIProvider with SGLang's OpenAI-compatible server (python -m sglang.launch_server, default http://localhost:30000/v1). Set NSYS_CORTEX_LLM_PROVIDER=sglang and NSYS_CORTEX_LLM_MODEL to the id you launched with — the Hugging Face id, or the --served-model-name alias when the server sets one (defaults to Qwen/Qwen3.8-27B; the same checkpoints and the same thinking-mode temperature 1 as vLLM above). Like vLLM, it's a GPU-server runtime with no auto-pull — the model must already be loaded; set NSYS_CORTEX_LLM_CONTEXT_TOKENS to its context length when that is below the model's window. API key is optional and only enforced if you passed --api-key; set SGLANG_API_KEY to mirror that. Override the base URL with SGLANG_BASE_URL.

SGLang vs vLLM: both are datacenter-GPU OpenAI-compatible servers optimized for throughput; they're direct alternatives, and Cortex treats them identically at the wire level. SGLang's distinguishing feature is RadixAttention — automatic prefix caching across requests that share a common prefix. That's a strong fit for an agent platform: every turn of a tool-use loop resends the same growing system-prompt-plus-history prefix, and RadixAttention lets the server skip recomputing it. If your workload is agentic (long multi-turn tool loops on one model) rather than many-short-independent-prompts, SGLang is worth benchmarking against vLLM. Deployment shape is otherwise the same: pick by which server your infra team already runs and which one wins your throughput benchmark on your model + hardware.

LM Studio (Local / Desktop GUI)

Reuses OpenAIProvider with LM Studio's built-in local server (default http://localhost:1234/v1, configurable in the UI under Developer → Local Server). Set NSYS_CORTEX_LLM_PROVIDER=lmstudio and NSYS_CORTEX_LLM_MODEL to the identifier LM Studio reports in GET /v1/models for the loaded model (defaults to qwen/qwen3.8-27b, LM Studio's catalog key for Qwen3.8-27B — the GGUF is lmstudio-community/Qwen3.8-27B-GGUF: Q4_K_M 16.8 GB, Q6_K 22.4 GB, Q8_0 29 GB; the default config's temperature is 1, Qwen's thinking-mode sampling and LM Studio's own default for the model). LM Studio handles downloads and quantization selection through its UI — there's nothing to pull from the CLI; the identifier is the catalog key for a catalog download, not the Hugging Face path. No API key is required by default; set LMSTUDIO_API_KEY if your setup is behind a reverse proxy that enforces one. Override the base URL with LMSTUDIO_BASE_URL.

LM Studio overlaps with Ollama on hardware (both target consumer machines) but is aimed at users who want a GUI for model browsing, quant comparison, and parameter tuning — Ollama is CLI-first and opinionated about quants.

Ollama vs vLLM vs LM Studio — which to pick

All the local providers speak the same OpenAI-compatible wire format, so the agent loop and tools don't care which one is upstream. They're not really competing for the same use case — pick by hardware, concurrency profile, and how you like to manage models. (SGLang isn't a separate column: it occupies the same slot as vLLM in every row — datacenter GPUs, HF model IDs, high-concurrency batching — differing mainly in RadixAttention prefix caching; see SGLang for when to prefer it.)

For nsys-cortex specifically: if you're driving nsys-cortex chat from your terminal, use Ollama — least friction, scriptable, no GUI. If you're a UI-first user who wants to browse models and try different quants without learning CLI tags, use LM Studio — same hardware envelope, friendlier discovery. If you're pointing the HTTP API at a shared internal inference server hosting one workhorse model for a team, use vLLM. There's no reason to pick one universally — that's why we wired them as siblings.

Rule of thumb: Ollama for "my model" in the terminal, LM Studio for "my model" with a UI, vLLM or SGLang for "our model" behind an API (SGLang when the workload is agentic — its prefix caching pays off on long tool-use loops).

Provider Factory

import { createProvider } from '@nsys-cortex/runtime';

// Auto-selects based on NSYS_CORTEX_LLM_PROVIDER env var
const provider = createProvider();

// Or specify explicitly
createProvider({ provider: 'gemini', apiKey: 'key' });
createProvider({ provider: 'grok', apiKey: 'key' });
createProvider({ provider: 'mistral', apiKey: 'key' });
createProvider({ provider: 'ollama' });
createProvider({ provider: 'vllm', baseUrl: 'http://gpu-host:8000/v1' });
createProvider({ provider: 'lmstudio' });
createProvider({ provider: 'sglang' });

Model Failover Chain

FailoverProvider wraps an ordered list of (provider, model?) candidates behind the standard LLMProvider interface. When a candidate fails before yielding its first event (auth error, rate limit, model not found, network), the next candidate is tried immediately — no backoff, that's the point of a fallback. Once any event has been yielded, errors propagate unchanged: switching models mid-response would stitch torn output from two models together.

Configure via env var or config file — when set, the chain fully defines provider selection (takes precedence over NSYS_CORTEX_LLM_PROVIDER):

# provider[:model], comma-separated; only the FIRST colon splits, so
# HF-style model paths (vllm:Qwen/Qwen3.8-27B) work.
export NSYS_CORTEX_LLM_FAILOVER="anthropic:claude-opus-5-5,openai:gpt-6.1-sol,ollama:llama4"

{ "llm": { "failover": ["anthropic:claude-opus-5-5", "openai:gpt-6.1-sol", "ollama:llama4"] } }

A candidate without a model uses the run's requested model (sensible for same-provider chains); cross-provider candidates almost always need an explicit model, since names don't transfer between providers. Each candidate reaches its own endpoint: the factory reads the per-provider <PROVIDER>_BASE_URL (exported from the config file's llm.providers.<p>.baseUrl), never the shared NSYS_CORTEX_LLM_BASE_URL, which overrides only the provider selected by NSYS_CORTEX_LLM_PROVIDER. Unknown provider names fail fast at createProvider() time. Each failover hop is logged (runtime:failover, warn level) with from/to and the triggering error.

Semantics worth knowing:

• Layering with retry. The agent loop's withRetry() wraps the whole chain, so a transient blip on the primary fails over instantly, while a full outage (every candidate failing with a retryable status) backs off exponentially and re-attempts the chain from the top — same behavior a single provider would get.
• Aborts don't hop. A user Ctrl-C surfaces as an error on the active candidate; the chain propagates it instead of burning the remaining candidates.
• Context budget follows the requested model. Compaction thresholds are chosen from the run's model before any failover happens — prefer chains of comparable context windows, or put the smallest-window candidate's budget in charge by requesting that model.
• Capability drift is on the operator. --think maps per-provider as usual (a fallback without a reasoning surface silently drops it), and prompt-cache hits won't carry across providers.

Retry fix shipped alongside this: providers are async generators, so the agent loop's previous withRetry(() => provider.stream(...)) retried only the (never-failing) generator construction — a single 529 killed the turn despite the documented backoff. The loop now prefetches the first stream event inside withRetry, so pre-first-token failures genuinely retry; post-first-token errors still propagate (torn output can't be redone).

Structured Outputs

On Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 the Anthropic provider uses the GA output_config.format JSON schema instead of the forced-tool translation below — those models reject forced tool_choice with a 400. That path coexists with tools and thinking, so nothing is dropped; the older models keep the translation.

AgentRunOptions.responseFormat / LLMProviderOptions.responseFormat constrain the model's final text response to a JSON schema:

{ type: 'json_schema', name: 'ticket', schema: { type: 'object', properties: {...}, required: [...], additionalProperties: false }, strict: true }

Per-provider mapping:

Notes:

• No beta headers on Anthropic — the forced-tool pattern works on every Claude model via the GA Messages API. (Anthropic's native structured-outputs beta can be enabled separately via NSYS_CORTEX_ANTHROPIC_BETA if you want to experiment, but the runtime doesn't depend on it.)
• OpenAI strict: true requires the schema to set additionalProperties: false and list every property in required — that's OpenAI's constraint, not ours. Set strict: false to relax.
• Surfaces: HTTP API POST /sessions/:id/message and /run accept a responseFormat: { name?, schema, strict? } body field (schema capped at 32KB serialized); CLI non-interactive mode takes --output-schema <file> pointing at a JSON Schema file.
• With the agent's tool registry active (the common chat case), only the OpenAI-compat family honors the schema end-to-end. On Anthropic/Gemini, enforced structure requires a tool-less request — construct the agent loop with an empty tool registry for dedicated extraction sessions, or call the provider's stream() directly with tools: [].

Vision Input

AgentRunOptions.images attaches base64 images to the current user message:

agentLoop.run({ agentId, sessionId, message: 'what is in this screenshot?', images: [{ data: base64Png, mimeType: 'image/png' }] });

Per-provider mapping — all native, no preprocessing:

Semantics:

• The user message becomes structured content (text block first, then images) and is persisted with attachments on the HistoryEntry, so a resumed session still shows the model the original images — same fidelity guarantee as tool calls.
• Accepted types: png, jpeg, gif, webp. Practical cap: 5MB per image (Anthropic's per-image limit, the strictest); the HTTP API enforces ≤4 images per message and a 25mb body limit.
• The model must actually be vision-capable — text-only local models (most Ollama/vLLM/LM Studio checkpoints) will ignore or garble image parts. Pick a VLM (e.g. llama3.2-vision, qwen2-vl) for local vision.

Surfaces: HTTP API images: [{ data, mimeType }] on /message + /run; CLI --image <file> (repeatable, non-interactive) and inline @path/to/image.png tokens in interactive chat.

Provider Interface

All providers implement LLMProvider:

interface LLMProvider {
  readonly name: string;
  stream(messages: LLMMessage[], options: LLMProviderOptions): AsyncIterable<LLMStreamEvent>;
  countTokens(text: string): number;
  countMessageTokens(messages: LLMMessage[]): number;
}

Message Conversion

Each provider converts the internal LLMMessage[] format to the provider-specific API format:

Anthropic:

• system role → injected as user message with [system] prefix (system prompt is passed separately via the system API parameter)
• tool_result role → user message containing a tool_result content block with tool_use_id
• assistant role with toolCalls → adds tool_use content blocks
• Images → base64 source blocks

OpenAI (and every OpenAI-compat sibling: Grok, Mistral, Ollama, vLLM, LM Studio, SGLang):

• System prompt → first system message in the messages array
• tool_result role → tool role message with tool_call_id
• assistant role with toolCalls → tool_calls array with function type
• Content blocks → extracted as plain text (images not supported in text extraction)

Gemini:

• System prompt → systemInstruction field (separate from contents)
• system role → appended to systemInstruction
• user role → { role: 'user', parts: [{ text }] }
• assistant role → { role: 'model', parts: [{ text }, { functionCall: { name, args } }] }
• tool_result role → { role: 'user', parts: [{ functionResponse: { name, response } }] }
• Tools → { functionDeclarations: [{ name, description, parameters }] }

Agent Loop

The core execution loop is an AsyncGenerator that yields events as they happen. The caller decides what to do with them (stream to SSE, log, persist, ignore).

import { CortexAgentLoop } from '@nsys-cortex/runtime';

const loop = new CortexAgentLoop({
  provider,
  toolRegistry: registry,
  sessionManager,
  systemPromptBuilder: (agentId, sessionId) => buildCortexSystemPrompt({ ... }),
});

for await (const event of loop.run({
  agentId: 'my-agent',
  sessionId: 'ses-123',
  message: 'Create a hello.txt file',
  model: 'claude-sonnet-4-6',
  maxTurns: 25,
  timeout: 600_000,
})) {
  console.log(event.type, event);
}

Event Types

run_complete is guaranteed to fire on every termination path because the main loop is wrapped in try/finally. tool_call_end always fires after a tool finishes; tool_error fires additionally when result.isError === true so UIs can show a distinct failure marker. The three tool-call events carry the tool's effectClass (ADR-011: read, mutate-reversible, mutate-irreversible or destructive, read from the registry) so a consumer routes approvals and audits from the stream itself; the events of a tool the registry does not know carry none. (#12)

Tool Execution

When the model returns multiple tool calls in one response, the agent loop partitions them into parallel-safe groups by parallelSafe flag + disjoint resourceKeys. Each group is executed via Promise.all; groups run serially.

• A tool is parallel-safe if parallelSafe === true (or undefined, falling through to readOnly).
• Two parallel-safe calls share a group when their resourceKeys (e.g. file paths, PIDs, memory file names) don't intersect.
• A non-parallel-safe call always starts its own group of size 1.

Read tools (read, grep, find, ls, stat) parallelize by default. Mutating tools (write, edit, apply_patch, process_kill, memory_append, memory_replace) declare parallelSafe: false and surface their target path/PID/file as resource keys so the planner serializes on overlap.

If a tool is unknown or blocked by policy, the agent loop returns a CortexToolResult with isError: true to the model. If a tool's execute method throws, the error is caught and returned with isError: true as well. The Anthropic provider propagates isError as is_error: true on the tool_result block so the LLM can reliably distinguish failure from prose.

Malformed tool input JSON from the model is silently replaced with {} — the tool still executes with empty input rather than crashing the run.

Hooks and approvals

Before every tool dispatch and before every model call the loop consults the hooks its caller installed (ADR-012, #16): AgentLoopOptions.hooks is { beforeToolCall?(ctx, signal), beforeLlmCall?(ctx, signal) }, each answering { action: 'allow' } or { action: 'deny', reason } — synchronously, or as a promise the loop awaits with the run's signal (a deferred decision; an abort ends the wait). A beforeToolCall deny yields hook_denied and a tool_call_end whose isError result is denied: <reason>, so the model is told why and history stays tool-pair balanced — the call never starts (tool_call_start is emitted at dispatch, after the hook and the gate, with the call's input). A beforeLlmCall deny ends the run (hook_denied, error hook denied the model call: <reason>, run_complete): there is no model to tell. beforeLlmCall is consulted once per attempt sequence, before withRetry, never inside it. Hooks narrow: the policy engine's own deny inside executeTool stays, and a hook cannot widen it.

The product's approval gate rides on the same point: a tool whose effect class AgentLoopOptions.approvalPolicy.requireFor lists (default ['destructive']; agent.approval.requireFor / NSYS_CORTEX_APPROVAL_REQUIRE_FOR), a tool whose policy.requiresApproval is set, and the elevated tool at the ask level (createElevatedTool(hostExec, policy, { level: 'ask' }); off returns no tool) are announced as approval_required (approvalId, toolCallId, toolName, toolInput, effectClass, expiresAt) and run only after an allow: AgentLoopOptions.approvals is an ApprovalBroker (request() / decide() / pending(); InMemoryApprovalBroker ships) and the decision arrives as approval_decided (decision, reason, decidedBy). No decision within approvalPolicy.timeoutMs (default 300 000; NSYS_CORTEX_APPROVAL_TIMEOUT_MS) is a deny (timeout), an abort is a deny (aborted), and without a broker a required approval is an immediate deny (no approval surface) — a gated tool never runs on silence. A parallel group is consulted as a whole: its approvals are requested together and decided independently, then the calls that passed are dispatched together. readApprovalPolicy() reads the two variables (a bad value is ignored with a warn). The surfaces — the interactive chat's prompt, the HTTP decision route — are #16 stage 2.

Run Lifecycle

1. Generate runId, resolve context budget for the model
2. Load session history (up to 200 messages), append user message
3. Build system prompt, convert tools to LLM definitions
4. Main loop (try/finally) (up to maxTurns iterations):
   • Check abort signal and timeout
   • If contextManager.needsCompaction() → run memory flush, then compact (tool-pair-safe — never splits tool_use from its tool_result)
   • Assert tool-pair balance before each provider call (assertToolPairsBalanced())
   • Stream model response via withRetry() (auto-retries 408/425/429/500/502/503/504/529 + Node net errors with exponential backoff + jitter, up to 4 attempts; honors Retry-After). The first stream event is prefetched inside the retry wrapper — async-generator providers run no code until iteration, so this is what makes pre-first-token failures actually retryable
   • Collect text + tool calls; if stopReason is end_turn → mark completedNormally, return
   • Plan parallel-safe groups; execute tool calls per group; fire tool_error for any failures
   • If loop detected → yield loop_detected, return
5. If maxTurns exceeded → yield error
6. finally — always yield run_complete with totalTurns, totalUsage, and stopReason ('aborted_or_error' if the run didn't complete normally)

Abort, Timeout & Cancellation

• AbortSignal — Pass signal in AgentRunOptions. Propagated end-to-end: through LLMProviderOptions.signal into the underlying fetch()/SDK call, and into every tool.execute(_, _, signal) invocation. Ctrl-C in the CLI chat actually cancels the in-flight Anthropic request and any running bash subprocess — not just suppresses further events.
• Timeout — Pass timeout in milliseconds (default: 600,000). Checked at the start of each turn against the run's start time.
• Per-attempt timeouts — withRetry honors signal between retries; an aborted run never gets stuck in the backoff sleep.

Prompt Caching

Provider-level prompt caching dramatically reduces input-token cost on multi-turn sessions.

• Anthropic — System prompt is converted to a structured block with cache_control: { type: 'ephemeral' }. The last tool definition gets cache_control (caches the entire tools array, the largest stable prefix). The 2nd-to-last user message becomes a cache breakpoint so older conversation history caches as the chat grows. Cache stats (cache_read_input_tokens, cache_creation_input_tokens) surface via TokenUsage.cacheReadTokens / cacheCreationTokens.
• OpenAI — prompt_cache_key is set from sessionId so multiple turns route to the same cache shard. prompt_tokens_details.cached_tokens surfaces via TokenUsage.cacheReadTokens.
• The CLI chat per-turn footer and /context output show cached tokens when caching is active.

Extended Thinking

Reasoning-capable models on Anthropic (Claude 4.x), OpenAI (o-series, GPT-5), and Gemini (2.5+) can spend extra tokens reasoning before they answer. Enable via AgentRunOptions.thinkingBudgetTokens (CLI: chat --think [budget]).

Wire (top to bottom):

Constraints to be aware of (intentional, not bugs):

• Reasoning text visibility varies. Anthropic and Gemini stream the reasoning text the user sees in the [thinking] gutter. OpenAI only exposes summaries via reasoning_content on some models — the raw chain-of-thought is hidden server-side. Pricing-wise, all three bill the reasoning tokens (Anthropic folds them into outputTokens; OpenAI / Gemini surface a separate reasoningTokens bucket).
• Temperature handling diverges. The agent loop sets temperature=1 when thinking is on (the pre-4.6 Anthropic constraint). The Anthropic provider sends temperature only to Haiku 4.5 / Sonnet 4.5 and earlier — on 4.6+ and every 5.x model the Messages API rejects it (temperature is deprecated for this model), with or without thinking. The OpenAI provider drops temperature to avoid a 400 on reasoning models. Gemini accepts any temperature alongside thinkingConfig.
• Tokens billed separately — budget_tokens (pre-4.6 Anthropic) and thinkingBudget (Gemini) are reasoning allowances distinct from max_tokens for the response. Both count against API usage. On Anthropic 4.6+ and OpenAI there is no numeric knob — the budget is bucketed to an effort level (output_config.effort / reasoning_effort), and the model decides how much to think.
• Block ordering matters on Anthropic. Thinking blocks must precede text/tool_use blocks in the assistant message. The agent loop guarantees this; if you ever construct an assistant message manually, mirror the same order.

CLI rendering: streaming thinking is shown inline as dim italic with a [thinking] gutter, transitioning cleanly into the markdown-rendered response on first text_delta. The non-interactive -p flow drops thinking output (it's reasoning, not response — including it would garble the JSON / piped text contract).

Context Window Management

The ContextManager tracks token usage and triggers automatic compaction when the conversation approaches the model's context limit.

Budget Defaults

1M context on Claude Sonnet 4.x, Opus 4.7+, and Fable 5 is GA — the Anthropic provider does not send any beta header. The older anthropic-beta: context-1m-2025-08-07 opt-in is no longer needed and would pin you to a legacy SKU; we deliberately omit it.

For operators who need to enable a different Anthropic beta (extended caching variants, computer use, etc.), set NSYS_CORTEX_ANTHROPIC_BETA to a comma-separated list — the provider attaches it as anthropic-beta: <list> on every request. The default is empty.

Compaction Threshold

usableBudget = maxContextTokens - reserveForOutput
needsCompaction = (systemPromptTokens + historyTokens) >= usableBudget * compactionThreshold

Token Counting

ContextManager accepts any Tokenizer implementation; three are shipped:

Selection is automatic: every provider exposes getTokenizer(model) on the LLMProvider interface, and the agent loop injects the result into the ContextManager it builds per run. Anthropic hands back AnthropicTokenizer; the OpenAI-compatible family hands back OpenAITokenizer; Gemini also hands back OpenAITokenizer as an offline proxy (its REST countTokens endpoint would put a network round-trip on the hot compaction check; o200k_base is within ~10-15% of Gemini's SentencePiece counts, which the conservative 85% threshold absorbs). FailoverProvider delegates to its primary candidate. SimpleTokenizer is only used when a provider offers nothing (e.g. test mocks). The CLI's /context readout measures with the same tokenizer via agentLoop.getContextTokenizer(model), so what you see is what compaction acts on.

Per-element overheads (used by SimpleTokenizer and OpenAITokenizer.countMessages):

Compaction triggers on the result, so an under-counted tokenizer can spill over the model's actual budget — which is why the loop injects the provider's calibrated tokenizer automatically rather than leaving SimpleTokenizer as the effective default.

Compaction Process

1. Context exceeds compactionThreshold * (maxContextTokens - reserveForOutput)
2. Memory flush: A silent turn prompts the agent to save important context to memory files (see below)
3. Snap tail boundary: The desired tail-start index is moved outward via snapTailBoundary() so the cut never falls between a tool_use block and its matching tool_result. Without this, the next provider call would 400 with tool_use ids must be followed by a tool_result whenever an old turn was a tool-use followed by a tool-result.
4. Split: History is divided into three parts using the snapped boundary:
   • First — the original user message (always preserved)
   • Middle — messages between first and the snapped tail start (to be summarized)
   • Tail — preserved tail (10 messages by default, possibly more if the snap moved the boundary outward)
5. Serialize middle: Each message is serialized with its role, text content, [tool_use: name(input)] markers, [tool_result: text] markers, and [called: name(input)] for tool calls. If the serialized text exceeds 50,000 characters, it is truncated with a [...truncated...] marker.
6. Summarize: The serialized middle is sent to the LLM with a summarization prompt (max 1,024 output tokens by default, configurable via summaryMaxTokens). The prompt instructs the model to preserve key decisions, file paths, errors, and task state.
7. Fallback: If summarization fails (provider error), a fallback message is used: "[Compaction summary unavailable — N messages were removed to free context space]"
8. Reassemble: [first message] + [user message with [CONTEXT_SUMMARY] prefix] + [tail messages]. The summary is injected as a user-role message, not system — several providers' message converters mishandle interleaved system messages.
9. Invariant assertion: assertToolPairsBalanced() runs on the reassembled history; throws loudly in dev if the snap logic ever produces an unbalanced result, so bugs surface as a clear assertion instead of an opaque provider 400.

Memory Flush

Before compaction, the agent runs a silent memory flush turn — an internal turn that is not exposed to the caller's event stream:

1. A user message is injected: "Context is about to be compacted. If there are important decisions, file paths, errors, or task state that should be preserved, write them to memory now using memory_update."
2. Only memory group tools are made available (all other tools are filtered out)
3. The agent can execute up to 3 sub-turns of tool use
4. If the model stops calling tools or an error occurs, the flush ends silently
5. This is best-effort — failures are caught and ignored

Tool Loop Detection

The ToolLoopDetector monitors tool call patterns and aborts runs that enter infinite loops.

Detection Patterns

All thresholds are configurable via LoopDetectorConfig.

How Detection Works

• record(call) is called before each tool execution. It checks max_calls_exceeded, identical_repeat (consecutive calls with the same name + input hash), and ping_pong (A→B→A→B pattern in the tail of call history).
• recordResult(toolName, resultText) is called after each tool execution. It tracks the last N results per tool and checks no_progress (all N results identical).
• Input hashing uses JSON.stringify(input, sortedKeys) for deterministic comparison. Inputs that differ only in key order are considered identical.
• When a loop is detected, the agent loop yields a loop_detected event and stops the run.

Session Persistence

CortexSessionManager (in @nsys-cortex/tools) accepts an optional SessionStorage adapter at construction time. When wired, every state-changing method (createSession, sendToSession, addHistory, addHistoryEntry, destroySession) persists to disk. FileSessionStorage (in @nsys-cortex/runtime) is the canonical implementation.

import { CortexSessionManager } from '@nsys-cortex/tools';
import { FileSessionStorage } from '@nsys-cortex/runtime';

const storage = new FileSessionStorage('~/.nsys-cortex/sessions');
const sessionManager = new CortexSessionManager({
  storage,
  config: { idleTimeoutMs: 4 * 3600_000, maxAgeMs: 24 * 3600_000, maxHistoryEntries: 200 },
});
await sessionManager.initialize(); // rehydrate caches; drops expired sessions

Both the CLI chat and the HTTP API server wire FileSessionStorage and call initialize() before use. The CLI uses it for resume (--continue picks the most recent session for the agent, --session <id> a specific one, /sessions lists them); the API server reads NSYS_CORTEX_SESSION_IDLE_TIMEOUT / NSYS_CORTEX_SESSION_MAX_AGE from the env and rehydrates before accepting traffic.

Faithful history + reconstruction

HistoryEntry carries the flattened content string plus optional structured fields so tool interactions survive a reload: toolCalls and thinking on assistant turns, and toolCallId / toolContent / isError on tool_result turns. The agent loop persists these via addHistoryEntry after each assistant turn and each tool result. Only truthy structured fields are serialized, so plain text entries stay compact and pre-existing transcripts round-trip unchanged.

On load, the agent loop's reconstructHistory() rebuilds provider-shaped LLMMessage[]:

• assistant entries with toolCalls become assistant messages with the tool calls attached;
• tool_result entries become tool_result messages with their toolContent blocks (and is_error flag);
• balanceReconstructed() drops leading orphan tool_results (whose tool_use was trimmed by the getHistory head-cap) and an interrupted trailing tool-use turn (results never persisted), so the first provider.stream() never 400s and assertToolPairsBalanced passes;
• thinking blocks are intentionally not replayed into provider messages — a provider's reasoning signature is bound to its original request context and could be rejected on replay, and the tool_use/tool_result pairs already carry the actions. They remain on disk for transcript fidelity (/history, export).

Storage errors are logged but never surface to callers — the in-memory state stays consistent regardless of disk state.

Cumulative usage

SessionInfo.usage accumulates token spend across the session's whole lifetime — { inputTokens, outputTokens, cacheReadTokens, cacheCreationTokens, reasoningTokens, turns, runs }. The agent loop calls sessionManager.recordUsage(sessionId, delta) in its run_complete finally with the run's totalUsage, merging it into the running total and persisting it (best-effort — wrapped so a usage-accounting failure never breaks a run; recorded even for aborted/errored runs since partial spend still counts). Because it lives on the persisted session, it survives reload (CLI --continue, HTTP-API restart) and rides along in GET /sessions/:id and the CLI /cost command. Token counts only — dollar cost depends on per-provider pricing and would go stale in the codebase.

Storage layout

~/.nsys-cortex/sessions/        # 0o700
├── sessions.json               # Session metadata index (atomic via tmp+rename)
├── <agentId-1>/                # 0o700
│   ├── ses-abc123.jsonl        # Per-agent namespaced transcripts
│   ├── ses-abc123.events.jsonl # The session's event record (ADR-014) — same lifetime, same mode
│   └── ses-def456.jsonl
└── <agentId-2>/
    └── ses-ghi789.jsonl

Per-agent namespacing (PR 2) means a prompt-injected agent reading filesystem entries cannot enumerate or read another agent's session log. The agentId is looked up from the sessions.json index, not from caller input, so callers can't forge a session ID to redirect into another agent's directory.

Each JSONL line:

{"role":"user","content":"Create a file","timestamp":"2026-02-19T10:00:00Z"}

loadHistory() wraps each line's JSON.parse and skips malformed lines instead of failing the whole transcript — a crash mid-append can leave one bad line, but the rest of the conversation survives.

The event record

Every run's events are recorded per session with a sequence number (ADR-014, #14), so a consumer can replay a session from any point and follow it live without having started the run. SessionStorage gains two optional methods, appendEvent(sessionId, event) and loadEvents(sessionId, afterSeq?); FileSessionStorage keeps the record in <agentId>/<sessionId>.events.jsonl beside the transcript — 0o600, deleted with the session, a malformed line skipped with a warning like the transcript's. SessionInfo.lastEventSeq carries the last sequence number issued, so the next run continues after it, a restart included.

recordRun(run, { sessionId, storage, bus, nextSeq }) in @nsys-cortex/runtime is the seam: it wraps CortexAgentLoop.run() without editing the loop. Every event goes to the consumer and onto the SessionEventBus (in-process, per session: subscribe(sessionId, fn) returns the unsubscribe; a throwing listener is logged and never fails the run); a non-delta event gets the next seq, is appended to the record and is yielded with it; text_delta / thinking_delta are published and yielded as they are and never recorded — at turn_complete the recorder appends and publishes one thinking event (when any) and one text event holding the turn's coalesced text, so the record stays bounded by the transcript. A storage adapter without the two methods keeps the stream live-only (logged once). seq is per session, monotonic from 1; the caller's nextSeq is per session and advances the manager's in-memory lastEventSeq, the recorder persists the last seq it issued at the end of the run, and lastRecordedSeq(storage, sessionId) reads the record's tail so a restart after a crash mid-run continues after the last line actually written. error events are recorded with their text (server-side, 0o600); the HTTP route sanitises on the way out. The attach route is #14 stage 2.

Session lifecycle

SessionConfig (also in @nsys-cortex/tools) controls expiry:

Lifecycle checks happen lazily on read (no background sweep timer) — keeps the manager stateless from the operator's perspective and means tests run deterministically.

Configuration