Large Language Models (LLMs)
Deep learning models trained on massive text datasets to understand and generate human language.
Transformer Architecture
The backbone of LLMs, leveraging self-attention mechanisms for language understanding.
Self-Attention
A mechanism allowing the model to weigh the importance of each word relative to others in a sentence.
GPT
Generative Pre-trained Transformer models (GPT) excel in text generation and language tasks.
BERT
Bidirectional Encoder Representations from Transformers, optimized for understanding context in both directions.
Tokenization
The process of breaking down text into smaller units (tokens) for input into LLMs.
Fine-Tuning
Adjusting a pre-trained LLM on a smaller, domain-specific dataset for better performance.
Prompt Engineering
Crafting effective input prompts to guide the output of LLMs toward desired responses.
RAG (Retrieval-Augmented Generation)
Combining retrieval from external sources with generation from LLMs to improve factual accuracy.
LoRA
Low-Rank Adaptation: a technique for efficient fine-tuning of large models by freezing base weights.
Hallucination
When LLMs generate outputs that are plausible-sounding but factually incorrect or fabricated.
Model Evaluation
Metrics such as BLEU, ROUGE, perplexity, and human feedback used to assess LLM performance.