> For the complete documentation index, see [llms.txt](https://docs.themspkb.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.themspkb.com/ai-for-msps/foundational-ai-concepts/large-language-models.md). # Large Language Models ### Summary A **large language model (LLM)** is a [Transformer Architecture](/ai-for-msps/foundational-ai-concepts/transformer.md) trained on a vast amount of text to do one deceptively simple thing: predict the next word. The 2020 paper *Language Models are Few-Shot Learners* introduced **GPT-3**, an autoregressive (left-to-right) LLM with **175 billion parameters** — roughly 10× larger than any comparable model before it. Its headline finding was that sheer scale unlocks a new ability: the model can perform many tasks it was never specifically trained for, learning them from a few examples in the prompt with **no gradient updates** (see [In-Context Learning](/ai-for-msps/foundational-ai-concepts/in-context-learning.md)). LLMs are the engine behind ChatGPT, Copilot, and most generative "AI" features now appearing in MSP tooling. ### In plain terms An LLM is trained by covering the next word in billions of sentences and adjusting itself until its guesses are good. Do that at enormous scale and the model absorbs grammar, facts, styles, and reasoning patterns as a side effect of getting good at prediction. GPT-3's contribution was to show what happens when you make the model *much* bigger and train it on *much* more text (hundreds of billions of words scraped largely from the web, books, and Wikipedia). Capabilities that smaller models lacked — translation, question answering, arithmetic, unscrambling words, writing news articles humans struggle to distinguish from real ones — appeared without task-specific training. GPT-3 is **decoder-only**: unlike [BERT](/ai-for-msps/foundational-ai-concepts/bert.md), it reads and writes strictly left to right, which makes it a natural text *generator*. Importantly, the paper is candid about limits: GPT-3 sometimes only *approaches* the performance of specially fine-tuned systems, struggles on some tasks, and inherits biases and factual errors from its web training data. ### Why it matters for MSPs * **This is what "generative AI" means.** When a tool drafts a ticket reply, summarizes a call, or writes documentation, an LLM is generating that text word by word — probabilistically. This is precisely the *probabilistic AI* the [AI vs. Automation](/ai-for-msps/ai-in-the-msp-stack/ai-vs.-automation.md) page contrasts with deterministic scripts. * **It explains hallucination.** An LLM is optimized to produce *plausible* next words, not *true* ones. That is the mechanical reason for the "confidently wrong" answers flagged in [What It Can't Do Yet](/ai-for-msps/ai-in-the-msp-stack/what-it-cant-do-yet.md) — the model has no built-in fact-checker. * **Scale drives cost.** 175 billion parameters is expensive to run. LLM pricing (per-token, per-seat) and latency trace back to model size, which matters when you assess [implementation and ROI](/ai-for-msps/ai-in-the-msp-stack/where-it-shows-up/implementation-and-roi.md). * **Web-trained means data-provenance questions.** LLMs learn from scraped text of unknown licensing and accuracy — part of why [Data Handling & Privacy](/ai-for-msps/ai-security/data-handling-and-privacy.md) and vendor **no-training** guarantees deserve scrutiny. ### Related concepts [In-Context Learning](/ai-for-msps/foundational-ai-concepts/in-context-learning.md) · [Transformer Architecture](/ai-for-msps/foundational-ai-concepts/transformer.md) · [Foundation Models](/ai-for-msps/foundational-ai-concepts/foundation-models.md) · [RLHF and Alignment](/ai-for-msps/foundational-ai-concepts/rlhf-and-alignment.md) · [BERT](/ai-for-msps/foundational-ai-concepts/bert.md) **In the MSP KB:** [AI vs. Automation](/ai-for-msps/ai-in-the-msp-stack/ai-vs.-automation.md) · [What It Can't Do Yet](/ai-for-msps/ai-in-the-msp-stack/what-it-cant-do-yet.md) ### Contradictions & debates * **Scale vs. alignment — the sharpest disagreement in this section.** GPT-3's thesis is that scaling up is the path to capability. The [RLHF and Alignment](/ai-for-msps/foundational-ai-concepts/rlhf-and-alignment.md) paper (InstructGPT, 2022) directly complicates it: human raters preferred the outputs of a **1.3B-parameter aligned model over the 175B GPT-3** — a model over 100× larger. Raw scale makes a model *capable*; it does not make it *helpful, honest, or harmless*. Both can be true, but the "bigger is automatically better" reading of GPT-3 does not survive InstructGPT. * **Generation-first (GPT) vs. understanding-first (BERT).** GPT-3 keeps the left-to-right design that [BERT](/ai-for-msps/foundational-ai-concepts/bert.md) argued was a limitation — and still reaches strong, sometimes state-of-the-art results. The two lineages settled into different jobs rather than one displacing the other. * **Capability vs. risk framing.** GPT-3 showcases broad capability; the [Foundation Models](/ai-for-msps/foundational-ai-concepts/foundation-models.md) paper (published a year later) reframes that same generality as concentrated societal risk. Same phenomenon, opposite emphasis. ### Source paper Brown et al. (2020), *Language Models are Few-Shot Learners* — *** **Key terms**: *large language model (LLM)*, *GPT-3*, *autoregressive*, *decoder-only*, *parameters*, *next-token prediction*, *scaling*. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.themspkb.com/ai-for-msps/foundational-ai-concepts/large-language-models.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.