What is Prompt caching?
Prompt caching is a provider-billed discount for reusing a stable prompt prefix. What it caches, what it costs, and where the breakpoints go.
1 min read↗
Ideas, collected
Local AI.
Essays, guides and observations. Find something worth sitting with.
What is Prefix caching?
Prefix caching reuses stored attention work across requests that share the same opening tokens. How it hits, and how builders keep it hot.
2 min read↗
Caching Layers Explained: KV Cache vs Prefix vs Prompt vs Semantic Cache
KV cache, prefix caching, prompt caching, and semantic cache compared: what each reuses, what it saves, and how local runners configure it.
7 min read↗
Jev vs Open Decision Models: Laya, Cua-S1-Forms, and When Scoring Beats Sampling
Jev vs Laya vs Cua-S1-Forms: when scoring fixed options beats sampling text, and which open model to self-host.
8 min read↗
What is KV Cache?
KV cache reuses past attention keys and values so long chats and agents run faster. What it costs, and how builders shrink it.
2 min read↗
Open Weights vs Open Source: What You Can Actually Download and Run
Open weights vs open source, explained for builders: what you can download, the VRAM math, and which licences let you self-host.
4 min read↗
AI news digest — September 19, 2026: legal agents, tiny models, and kill switches
AI news digest Sep 19: Astra for Law legal index, TypeSafe Jev model, 706K-param forms planner, kill-switch order, 27B in 5.9GB.
4 min read↗
RAG on Your Own Machine: A Local Retrieval Walkthrough
Build retrieve-then-write locally: chunking, a resident embeddings model, a JSONL index to Chroma/Qdrant, cited grounded answers.
6 min read↗
Quantized Models Explained: Which Quant for Which VRAM
From 16-bit weights to 4-bit quants: what quantization does, the VRAM math per level, and a practical which-quant-for-which-card table for 4 to 16 GB GPUs.
6 min read↗
Local Coding Agents Compared: Kilo Code vs Coder Agents vs Copilot Agent Mode
Kilo Code vs self-hosted Coder Agents vs Copilot agent mode: which local-friendly coding agent fits solo builders and teams.
12 min read↗
Embeddings on Consumer GPUs: Local Search That Stays Resident
Which embedding models fit beside a chat quant on a 6GB card, real VRAM costs, and three local runners: Ollama, llama.cpp, Python.
6 min read↗
AI news digest — September 8, 2026: launches, models, safety
Six builder stories: launches, safeguards, open models, agents.
3 min read↗
Small Models That Fit a 6 GB RTX 2060
What really runs in 6 GB VRAM on an RTX 2060: quantized 7–8B models, partial offload for 13B, and keeping embeddings hot on the GPU.
3 min read↗
No articles found.
Try a different word, or return to the full collection.