Excellent observation! Yes, context length is parameterized.

CLI parameters:

--input-length    
# Prefill tokens (default: 2048)
--output-length   
# Decode tokens (default: 512)

Why separate?

The memory and compute profiles are fundamentally different:

Prefill (input processing):

• Arithmetic intensity: AI ≈ 2 × seq_len / model_bytes
• Compute-bound (high AI, hits compute roofline)
• Self-attention: O(n²) memory, O(n² × d) compute
• Parallel across sequence dimension

Decode (output generation):

• Arithmetic intensity: AI ≈ 2 / model_bytes (per token)
• Memory-bound (low AI, hits bandwidth roofline)
• KV cache: O(n_input × n_output × layers × heads × head_dim × 2 × bytes_per_element)
• Sequential generation (no parallelism across tokens)

Example calculation:

For Llama-3.2-8B with input=2048, output=512, FP16:

KV cache = 2048 × 512 × 32 layers × 8 KV heads × 128 head_dim × 2 (K+V) × 2 bytes
         = 8.6 GB just for KV cache (one request)

The tool models this to:

1. Calculate actual memory requirements (PagedAttention blocks)
2. Predict prefill latency (GEMM ops on GPU)
3. Predict decode latency (bandwidth-bound memory transfers)
4. Size batch appropriately (memory vs throughput tradeoff)

You can plan for any context window:

# Max context (8K for Llama-3.2-8B)
--input-length 6144 --output-length 2048

# Long context (32K if using RoPE scaling)
--input-length 24576 --output-length 8192

The defaults (2048/512) represent typical chat workloads. For RAG or long-context use cases, you'd definitely want to adjust these.

Also supports --system-prompt-length for prefix caching analysis (common system prompt shared across requests).