bundle-course-sap-technical-abap-abap-on-hana-bo-data-services-bw4hana-hana By Uplatz<\/a><\/h3>\nI. The Foundation: Pretraining and the “Base Model”<\/b><\/h2>\n
<\/p>\n
A. The Foundational Objective: Learning from the World’s Text<\/b><\/h3>\n
<\/p>\n
Pretraining is the initial, resource-intensive stage where an LLM is trained from scratch on a vast and diverse corpus of text and code.<\/span>2<\/span> The objective is not to teach the model to perform any specific user-facing task, but rather to force it to learn the statistical patterns, syntactic rules, semantic relationships, and vast “world knowledge” embedded within human language.<\/span>2<\/span><\/p>\nThis process is powered by a paradigm known as <\/span>self-supervision<\/b>.<\/span>16<\/span> In this approach, the training labels (e.g., the next word in a sentence) are derived from the input data itself, eliminating the need for costly and time-consuming human annotation.<\/span>2<\/span> From the perspective of downstream applications like sentiment analysis, this pretraining phase is often described as “unsupervised” because the model learns useful, general-purpose representations without exposure to any task-specific labels.<\/span>17<\/span><\/p>\nThis distinction is not merely pedantic; it is the core economic and practical justification for the entire pretrain-finetune paradigm. Because the model first learns general linguistic competence from massive, cheap, <\/span>unlabeled<\/span><\/i> data <\/span>5<\/span>, it requires significantly <\/span>less<\/span><\/i> specialized, expensive, <\/span>labeled<\/span><\/i> data during the subsequent fine-tuning stage to achieve high performance on a new task.<\/span>16<\/span> This data efficiency is what makes adapting LLMs practical.<\/span><\/p>\n <\/p>\n
B. Core Pretraining Mechanisms: NTP vs. MLM<\/b><\/h3>\n
<\/p>\n
The specific self-supervised objective used during pretraining fundamentally defines the model’s architecture and its innate capabilities. The two dominant objectives create an architectural schism in LLM design.<\/span><\/p>\n\n- Autoregressive \/ Next-Token Prediction (NTP):<\/b><\/li>\n<\/ol>\n
\n- Mechanism:<\/b> Employed by decoder-only models such as the GPT series.<\/span>2<\/span> This objective trains the model to predict the next token (word) in a sequence given all preceding tokens.<\/span>2<\/span> It is a “left-to-right, causal” approach <\/span>2<\/span>, mathematically defined as maximizing the probability of a sequence $w$ by modeling $P(w) = \\prod_{i=1}^{n} P(w_i | w_1, \\ldots, w_{i-1})$.<\/span><\/li>\n
- Strengths:<\/b> This method excels at coherent, long-form text generation, as its entire objective is to produce the next logical word.<\/span>19<\/span> This training also leads to surprising emergent abilities in areas like mathematics and reasoning, even without specific training.<\/span>2<\/span><\/li>\n
- Weaknesses:<\/b> NTP-based models can struggle with tasks requiring precise information retrieval from a long context.<\/span>19<\/span> Research indicates this may be a fundamental trade-off of the objective itself; as the model’s layers process information, they learn to “forget” previous tokens to better predict <\/span>future<\/span><\/i> tokens, which is antithetical to tasks requiring perfect recall of early context.<\/span>20<\/span><\/li>\n<\/ul>\n
\n- Denoising \/ Masked Language Modeling (MLM):<\/b><\/li>\n<\/ol>\n
\n- Mechanism:<\/b> Employed by encoder-only models like BERT.<\/span>19<\/span> This approach is a form of a Denoising Auto-Encoder (DAE).<\/span>21<\/span> It “corrupts” the input by masking a percentage of its tokens (e.g., 15%) and trains the model to reconstruct the original, uncorrupted text.<\/span>21<\/span><\/li>\n
- Strengths:<\/b> To predict a masked token, the model must use <\/span>bidirectional attention<\/span><\/i>, or look at the text both before and after the mask.<\/span>19<\/span> This “cloze-type” objective makes MLM-based models exceptionally effective at tasks requiring deep contextual understanding, sentence-level information retrieval, and the generation of rich text embeddings.<\/span>19<\/span><\/li>\n
- Weaknesses:<\/b> Because they are not trained to sequentially generate text, MLM models are inherently unsuited for coherent, long-form text generation.<\/span>19<\/span><\/li>\n<\/ul>\n
The choice between NTP and MLM dictates the model’s innate utility. The division is so fundamental that to make a decoder-only (NTP) model effective at text <\/span>embedding<\/span><\/i> tasks (an encoder’s strength), one must modify it by enabling bidirectional attention and adding a masked prediction objective\u2014in essence, temporarily forcing the GPT-style model to behave like a BERT-style model.<\/span>22<\/span><\/p>\n <\/p>\n
C. The Artifact: The “Base Model” (e.g., Llama 2-base, GPT-3)<\/b><\/h3>\n
<\/p>\n
The end product of the pretraining phase is the “base model”.<\/span>4<\/span> Prominent examples include the Llama 2 and Llama 3 base models <\/span>24<\/span>, and the original GPT-3 model.<\/span>26<\/span> These models are foundational artifacts, possessing immense general knowledge (the Llama 2 base model, for instance, was trained on 2 trillion tokens).<\/span>24<\/span> However, they are “uncensored” and not tuned for dialogue or instruction-following.<\/span>4<\/span> They are powerful, but raw, repositories of statistical knowledge.<\/span><\/p>\n <\/p>\n
D. The “Alignment Gap”: Why Base Models Are Not Helpful Assistants<\/b><\/h3>\n
<\/p>\n
A base model is not a usable product for most applications, creating an “alignment gap” that necessitates the subsequent tuning stages.<\/span>27<\/span><\/p>\n\n- The Completion Engine Problem:<\/b> Base models are trained to <\/span>predict the next word<\/span><\/i>, not to <\/span>answer a user’s question<\/span><\/i> or <\/span>follow an instruction<\/span><\/i>.<\/span>5<\/span> Their objective is statistical pattern matching, not adherence to user intent.<\/span>13<\/span> If a user inputs “Summarize this article:”, a base model is just as likely to complete the prompt with “in 500 words or less.” as it is to actually perform the summarization.<\/span><\/li>\n
- The User Expectation Mismatch:<\/b> Usability studies reveal that most users have an “inaccurate mental model” of LLMs, often equating them to sophisticated search engines.<\/span>28<\/span> They do not instinctively understand that the quality of the model’s output is highly dependent on careful prompt engineering. This gap between user expectation and model capability <\/span>necessitates<\/span><\/i> a model that can understand and follow instructions directly.<\/span><\/li>\n
- Lack of Helpfulness and Safety:<\/b> A base model’s outputs simply reflect the (often undesirable) patterns of its training data. They can generate responses that are untruthful, toxic, biased, or simply unhelpful.<\/span>29<\/span><\/li>\n<\/ol>\n
A clear case study is the comparison between the original GPT-3 base model and its aligned successor, InstructGPT. The base GPT-3 often failed to follow simple instructions.<\/span>6<\/span> When asked to perform a task, it might instead generate text <\/span>about<\/span><\/i> the task.<\/span>31<\/span> This demonstrated a profound gap between the model’s <\/span>capability<\/span><\/i> (knowledge) and its <\/span>usability<\/span><\/i> (behavior), a gap that alignment tuning is designed to close.<\/span>30<\/span><\/p>\n <\/p>\n
II. The “Alignment” Imperative: Bridging the Gap from Prediction to Utility<\/b><\/h2>\n
<\/p>\n
A. Defining the AI Alignment Problem for LLMs<\/b><\/h3>\n
<\/p>\n
AI alignment is the broad technical field focused on steering AI systems toward human-intended goals, preferences, and ethical principles.<\/span>32<\/span> In the context of LLMs, this problem is often simplified to achieving the “HHH” triad: making the model <\/span>Helpful<\/b> (it correctly follows user intent), <\/span>Honest<\/b> (it is truthful and does not fabricate information), and <\/span>Harmless<\/b> (it refuses to produce toxic, biased, or unsafe content).<\/span>33<\/span><\/p>\nThis is a non-trivial challenge. AI designers cannot specify the full range of desired and undesired behaviors, so they often use simpler “proxy goals”.<\/span>32<\/span> A model may find loopholes in these goals (“reward hacking”) or develop emergent, undesirable behaviors (like strategic deception) to achieve its objectives in unintended ways.<\/span>32<\/span><\/p>\n <\/p>\n
B. The Two-Phase Alignment Pipeline<\/b><\/h3>\n
<\/p>\n
The industry-standard solution to the alignment gap, popularized by OpenAI’s research on InstructGPT <\/span>30<\/span>, is a multi-stage process <\/span>35<\/span>:<\/span><\/p>\n\n- Phase 1: Supervised Fine-Tuning (SFT):<\/b> The base model is first trained on a smaller, high-quality dataset of examples demonstrating desired behavior. This dataset is typically human-written or generated by a more advanced AI.<\/span>30<\/span><\/li>\n
- Phase 2: Preference Tuning (e.g., RLHF\/DPO):<\/b> The SFT model is then further refined using feedback on its outputs. This often involves Reinforcement Learning from Human Feedback (RLHF), where a “reward model” is trained on human <\/span>preferences<\/span><\/i> (e.g., “Answer A is better than Answer B”).<\/span>38<\/span> This reward model then “steers” the SFT model toward generating outputs that humans would rate highly.<\/span>30<\/span> Newer methods like Direct Preference Optimization (DPO) achieve similar results without the complexity of reinforcement learning.<\/span>40<\/span><\/li>\n<\/ol>\n
The common confusion surrounding “Fine-Tuning” vs. “Instruction Tuning” <\/span>8<\/span> stems from a misunderstanding of <\/span>Phase 1: SFT<\/b>. SFT is the <\/span>general mechanism<\/span><\/i> (supervised training on a labeled dataset). This mechanism can be applied to two different, parallel goals: injecting domain-specific <\/span>knowledge<\/span><\/i> or teaching general-purpose <\/span>behavior<\/span><\/i>
![\"\"](\"https:\/\/uplatz.com\/blog\/wp-content\/uploads\/2025\/11\/LLM-Training-Strategies-Compared-1024x576.jpg\")
<\/p>\n