AI terminology explained for people who ran the old IT world — every term comes with an old-IT translation, so you can map what you already know onto what's new.
37 terms
Old-IT translation: The new general-purpose server: one box that runs almost any text workload you throw at it.
A neural network trained on massive amounts of text to predict the next token, which in practice lets it write, summarize, translate, and reason in natural language. GPT, Claude, and Gemini are LLMs. Everything else in this glossary exists to feed, steer, or constrain one.
Related: Token · Parameters · Inference · Frontier model
Old-IT translation: The byte of the AI world — the smallest unit models read, and the unit you're billed in.
A chunk of text, roughly three-quarters of an English word, that models process and providers meter. API pricing is quoted per million tokens, split into input (what you send) and output (what the model writes). A page of prose is about 500 tokens.
Related: Context window · Inference · API pricing
Old-IT translation: RAM for a conversation: fixed capacity, and when it's full something gets evicted.
The maximum number of tokens a model can consider at once — your prompt, the conversation history, retrieved documents, and the reply all share it. Models range from 64K to over 1M tokens. Overflow doesn't error; the model just stops seeing what fell out.
Old-IT translation: The new CLI: precise input in, useful output out — vague input in, garbage out.
The text you send a model to make it do something. Prompting rewards the same skills as good shell commands and tickets: explicit requirements, examples, constraints, and expected output format. It is the primary programming interface of the AI age.
Related: System prompt · Zero-shot / few-shot · Prompt injection
Old-IT translation: Group Policy for a model: standing rules applied before any user request.
Instructions set by the application (not the end user) that define a model's role, tone, rules, and boundaries for the whole session. User messages are interpreted inside whatever the system prompt established.
Related: Prompt · Guardrails · Prompt injection
Old-IT translation: Runtime, as opposed to build time — serving requests with a model that's already trained.
Running a trained model to produce output. Every chat message, API call, and agent step is inference. It's what you pay for per token, as opposed to training, which happened once at enormous cost before you ever showed up.
Related: Training · Token · Latency vs throughput
Old-IT translation: Burning the firmware: done in the factory, not something you do in production.
The process of adjusting a model's billions of parameters against training data until it predicts well. It costs millions of dollars in GPU time and is done by the model provider — almost nobody else trains models from scratch. What most teams call 'training' is actually fine-tuning or RAG.
Related: Fine-tuning · Parameters · Inference
Old-IT translation: The spec-sheet number, like core count — bigger usually means more capable and more expensive to run.
The learned numeric weights inside a model; '7B' means seven billion of them. Parameter count roughly tracks capability and directly drives memory needs: a 7B model runs on a laptop, a 70B model wants a serious GPU or a Mac with lots of unified memory.
Related: Quantization · Local model · VRAM / unified memory
Old-IT translation: A custom image on top of the vendor OS: same core, your specifics baked in.
Continuing a model's training on your own examples so behavior you'd otherwise cram into every prompt becomes the default. Useful for consistent format and tone at volume; usually the wrong first move — most teams get further with better prompts and RAG.
Old-IT translation: Giving the model read access to your file share instead of asking it to remember everything.
An architecture where relevant documents are fetched (usually via embeddings and a vector database) and pasted into the prompt so the model answers from your data instead of its training memory. It's how 'chat with your docs' products work, and the standard cure for hallucinated facts about your own systems.
Related: Embedding · Vector database · Hallucination · Context window
Old-IT translation: A hash for meaning: similar content produces nearby values, so you can index by 'what it's about.'
A list of numbers representing a text's meaning, produced by an embedding model. Texts about similar things land near each other in that number space, which is what makes semantic search possible: you match by meaning, not keywords.
Related: Vector database · RAG
Old-IT translation: An index tuned for one query: 'find records that mean roughly this.'
A database that stores embeddings and answers nearest-neighbor queries fast. It's the storage layer of RAG. Pinecone, pgvector, and Qdrant are common choices; for small datasets a plain array and cosine similarity works fine.
Old-IT translation: A confident wrong answer from a system with no error code — the silent data corruption of AI.
When a model states something false as fact, fluently and without any signal of uncertainty. It's a structural property of next-token prediction, not a bug that will be patched out. Mitigations: RAG for grounding, asking for sources, verification steps, and never shipping unreviewed model output where correctness matters.
Related: RAG · Evals · Temperature
Old-IT translation: A jitter knob: zero for reproducible runs, higher for variety.
A sampling setting controlling how random the model's word choices are. Low (0–0.3) gives consistent, focused output for extraction and code; higher (0.7+) gives variety for brainstorming and creative work. It does not control accuracy — a temperature-0 model still hallucinates.
Related: Hallucination · Inference
Old-IT translation: Cron plus a script that can improvise: work delegated end-to-end, not step-by-step.
A model given tools and a goal, running in a loop: assess, act, check results, act again until done. Unlike a chatbot answering one message, an agent might search, run code, edit files, and retry failures. Power and risk both come from the same place — it acts without you approving each step.
Related: Tool use · MCP · Orchestration · Guardrails
Old-IT translation: Syscalls for models: the defined interface through which the model touches the real world.
The mechanism by which a model invokes external capabilities — search, database queries, code execution, APIs — by emitting structured calls the host application executes. Everything an agent does beyond generating text goes through tool use.
Old-IT translation: ODBC for AI: one standard connector instead of a custom integration per app-and-tool pair.
An open protocol that standardizes how AI applications connect to tools and data sources. An MCP server exposes capabilities (your database, your ticket system); any MCP-capable assistant can use them without custom integration code. It's becoming the middleware layer of the AI stack.
Old-IT translation: Compression for models: trade a little fidelity for a lot less memory.
Storing model weights with fewer bits. Q4 shrinks a model to roughly a quarter of full size with modest quality loss, which is what makes running 7B–70B models on consumer hardware possible at all. Most local-AI users run Q4 or Q8.
Related: Parameters · Local model · VRAM / unified memory
Old-IT translation: On-prem instead of cloud: your hardware, your data, no per-request bill.
A model whose weights you download and run yourself with tools like Ollama or LM Studio. Weaker than frontier hosted models, but free per token, offline-capable, and private — data never leaves the machine. The practical question is memory: parameters times quantization must fit your RAM or VRAM.
Related: Quantization · Open weights · VRAM / unified memory · Frontier model
Old-IT translation: Freeware, not open source: you get the binary to run, not the build system.
Models whose trained weights are published for download (Llama, Mistral, DeepSeek) under licenses of varying permissiveness. 'Open weights' is not full open source — training data and code usually stay private. It's what makes local AI and self-hosting possible.
Related: Local model · Frontier model
Old-IT translation: The flagship SKU: the most capable tier on the market, priced accordingly.
The most capable models available at a given time — currently the top GPT, Claude, and Gemini tiers. They cost the most per token and handle the hardest reasoning. Most workloads don't need one: routing easy work to cheaper tiers is the single biggest AI cost lever.
Related: API pricing · Local model · LLM
Old-IT translation: Metered hosting instead of a flat-rate plan — pay for exactly what you use.
Direct model access billed per million tokens, input and output priced separately. Light users usually beat a $20/month subscription with API pricing; heavy users often don't. Batch endpoints (~50% off) and cached input (up to 90% off) change the math substantially.
Related: Token · Frontier model · BYOK
Old-IT translation: Using your own license with someone else's tooling.
A pattern where an application lets you plug in your own API key, so you pay the provider directly for usage instead of paying the app a marked-up subscription. Often the cheapest way to get frontier AI into a workflow you control.
Related: API key · API pricing
Old-IT translation: A service account credential — and it deserves the same paranoia.
The secret that authenticates your requests to a model provider and bills your account. Treat like any credential: environment variables or a secrets manager, never in client-side code or a repo, rotate on exposure, set spend limits. Leaked AI keys get abused within hours.
Related: BYOK · Rate limit
Old-IT translation: QoS on the provider's side: exceed your tier and requests get throttled, not queued.
Provider-enforced caps on requests and tokens per minute, by account tier. Production AI systems need the same discipline as any rate-limited API: backoff with retries, request queuing, and monitoring for 429s.
Related: API key · Latency vs throughput
Old-IT translation: SQL injection's successor: untrusted input that gets executed as instructions.
An attack where content the model processes — a web page, an email, a document — contains instructions the model then follows as if they came from the user. It's the top security issue for agents and RAG systems. Defenses are partial: treat all fetched content as untrusted, limit tool permissions, and require human approval for consequential actions.
Related: Jailbreak · Guardrails · Agent
Old-IT translation: Privilege escalation against the model's policy layer instead of the OS.
Prompting techniques that trick a model into ignoring its safety rules. Distinct from prompt injection: jailbreaks target the model's own limits, injections hijack it through content it processes. Providers patch known jailbreaks continuously; none stay reliable for long.
Related: Prompt injection · Guardrails
Old-IT translation: Firewall rules for model behavior: allow, deny, and log at the boundary.
The checks around a model that constrain what goes in and what comes out — input filtering, output validation, permission limits on tools, human approval gates. In practice guardrails live in your application code, not inside the model.
Related: System prompt · Prompt injection · Evals
Old-IT translation: The test suite: without one, every model or prompt change is a deploy on faith.
Systematic tests for AI behavior — a set of inputs with expected properties, run against every prompt change or model upgrade. Evals are how you know the new model version didn't quietly break your extraction format. The AI equivalent of CI.
Related: Benchmark · Hallucination · Guardrails
Old-IT translation: Synthetic load tests: useful signal, but never mistake them for your production workload.
Standardized public test sets (MMLU, SWE-bench, and dozens more) used to compare models. Directionally useful, heavily marketed, and increasingly gamed. Your own evals on your own tasks beat any leaderboard.
Related: Evals · Frontier model
Old-IT translation: One port that accepts more than one protocol: text, images, audio through the same interface.
Models that accept and/or produce more than text — reading screenshots, diagrams, and PDFs, or generating images and speech. Practically: you can paste a screenshot of an error dialog instead of retyping it.
Old-IT translation: tail -f on the response instead of waiting for the job to finish.
Receiving a model's output token-by-token as it generates rather than all at once at the end. It's why chat UIs feel responsive despite multi-second total generation times. API consumers handle it as server-sent events.
Related: Latency vs throughput · Inference
Old-IT translation: The same tradeoff you tuned in every system: response time versus jobs per hour.
Time-to-first-token and tokens-per-second determine how fast one response feels; throughput is how much total work you push through. Interactive use optimizes the former, batch pipelines the latter — batch APIs trade hours of delay for ~50% lower cost.
Related: Streaming · API pricing · Inference
Old-IT translation: The disk-space check of the AI age: the model file must fit or nothing runs.
GPU memory (VRAM) is the hard constraint for running models locally at speed — the quantized model must fit in it. Apple Silicon's unified memory lets the GPU use most of system RAM, which is why a 32 GB MacBook can run models a PC would need a 24 GB graphics card for.
Related: Local model · Quantization · Parameters
Old-IT translation: Spec-only versus spec-plus-examples — and examples win more arguments than specs.
Zero-shot means asking a model to do a task from instructions alone; few-shot means including worked examples in the prompt. Two or three good examples routinely fix output-format problems that paragraphs of instructions can't.
Related: Prompt · Fine-tuning
Old-IT translation: Verbose mode for thinking: slower, costlier, and far better on hard multi-step problems.
Having a model work through intermediate steps before answering — either prompted ('think step by step') or built in, as with dedicated reasoning models that spend extra 'thinking' tokens. You pay for those tokens; use reasoning tiers for genuinely hard problems, not lookups.
Related: Frontier model · Token · Prompt
Old-IT translation: The job scheduler layer: many workers, dependencies, retries — the pipeline thinking you already know.
Coordinating multiple model calls or agents into a workflow — routing easy tasks to cheap models, fanning out parallel subtasks, verifying outputs before the next stage. This is where old-IT pipeline and queue experience transfers directly to AI systems.
Related: Agent · Evals · API pricing
Put the vocabulary to work with the free AI Ops tools: