Foundational AI Concepts

Plain-language reference pages for the foundational research concepts behind the AI tools in the MSP stack — transformers, attention, BERT, large language models, in-context learning, foundation model

Why this section exists

Elsewhere in this KB we treat AI as an augmentation layer and focus on how to evaluate it responsibly (see AI in the MSP Stack and AI Security). This section goes one layer deeper: it explains the core research ideas that make tools like ChatGPT, Copilot, and the "AI" features in RMM/PSA platforms work.

You do not need a machine-learning background to sell, buy, or govern these tools — but a working mental model of why AI outputs vary, why models hallucinate, and why bigger is not automatically better helps you separate real capability from marketing. Each page is a self-contained entity page: a summary, a plain-language explanation, why it matters for MSPs, related concepts, noted contradictions between the source papers, and a link to the original research.

This section is descriptive, not prescriptive. It explains what these concepts are so you can understand the tools you already use — it is not a tutorial on building models.

The five papers behind modern AI

Every mainstream AI assistant in use today descends from a short lineage of research papers. Read in order, they tell the story of how we got from spell-check to ChatGPT:

Year	Paper	Concept it introduced	Entity page
2017	Attention Is All You Need	The Transformer architecture	Transformer Architecture
2018	BERT	Bidirectional pretraining + fine-tuning	BERT · Pretraining & Fine-Tuning
2020	Language Models are Few-Shot Learners (GPT-3)	Scale + in-context learning	Large Language Models · In-Context Learning
2021	On the Opportunities and Risks of Foundation Models	The term "foundation model"	Foundation Models
2022	Training LMs to Follow Instructions (InstructGPT)	Alignment via human feedback (RLHF)	RLHF & Alignment

How the concepts connect

Attention ──▶ Transformer ──┬──▶ BERT (bidirectional, fine-tuned)
                            │
                            └──▶ GPT-3 / LLMs (autoregressive, in-context learning)
                                     │
                                     ├──▶ Foundation Models (the category + its risks)
                                     └──▶ RLHF / InstructGPT (aligning models to humans)

The Attention Mechanism is the engine; the Transformer is the machine built around it. From that same machine, two families branched: BERT-style models that read text bidirectionally and are fine-tuned per task, and GPT-style large language models that generate text left-to-right and learn tasks in-context. As these models grew general enough to underpin everything, the research community named the category foundation models — and warned about its risks. RLHF was the technique that turned a raw text-predictor into a helpful assistant.

Where the papers disagree

These papers are a lineage, but they are not a consensus. The most useful tensions to understand:

• Bidirectional vs. one-directional. BERT argues that reading text in both directions at once is necessary for strong language understanding and explicitly frames left-to-right-only models as limited. GPT-3 keeps the left-to-right approach and still reaches state-of-the-art results — suggesting the limitation BERT identified was not fatal.

• Fine-tuning vs. in-context learning. BERT's whole paradigm is fine-tune a copy of the model for each task. GPT-3 pushes back, listing the downsides of fine-tuning and showing a model can often learn a task from a few examples in the prompt with no weight updates at all.

• Scale vs. alignment. GPT-3's headline is bigger is better. InstructGPT directly complicates this: human raters preferred a 1.3B-parameter aligned model over the 175B GPT-3 — a model over 100× larger, showing that how you train can beat how big you build.

• Capability optimism vs. risk caution. GPT-3 celebrates broad capability; the foundation models paper reframes that same generality as a concentration of risk — one flawed base model propagating its flaws to everything built on top.

Each entity page carries a Contradictions & debates section with the specifics.

Start here

If you are new to the topic, read in this order: Attention Mechanism → Transformer Architecture → Large Language Models → In-Context Learning → RLHF and Alignment. Then loop back to BERT, Pretraining and Fine-Tuning, and Foundation Models for the full picture.

---

Key terms: transformer, attention, large language model (LLM), pretraining, fine-tuning, in-context learning, foundation model, RLHF, alignment.