The rapid evolution of deep learning models, particularly Large Language Models (LLMs), has led to unprecedented capabilities across various domains. However, this advancement comes with a significant cost: models are growing exponentially in size and complexity, demanding immense computational resources and memory. This escalating demand poses a substantial challenge for deploying these powerful AI systems on ubiquitous, resource-constrained devices or in applications requiring real-time inference. Extreme low-bit quantization emerges as a critical solution, addressing these limitations by drastically reducing the precision of model parameters. This technique fundamentally transforms the computational landscape of deep learning, enabling broader accessibility and more sustainable operation of advanced AI.<\/span><\/p>\n <\/p>\n Quantization in deep learning is a process that reduces the numerical precision of model parameters, typically weights and activations, from high-precision floating-point formats to lower-precision integer representations.<\/span>1<\/span> Deep neural networks (DNNs) traditionally operate with 32-bit floating-point (FP32) or 16-bit floating-point (FP16) values, which offer high numerical fidelity but are computationally and memory intensive.<\/span>1<\/span> The primary objective of low-bit quantization is to compress these models, thereby shrinking their memory footprint and accelerating inference operations.<\/span>2<\/span><\/p>\n Extreme low-bit quantization extends this concept to its practical and theoretical limits, generally involving bit-widths of 4-bit integer (INT4) and below. This includes highly aggressive reductions to 2-bit, 1.58-bit (ternary), and 1-bit (binary) representations.<\/span>1<\/span> This aggressive precision reduction distinguishes extreme low-bit quantization from more conventional 8-bit integer (INT8) quantization, which often achieves acceptable accuracy with relatively straightforward methods. At sub-4-bit levels, maintaining model performance becomes significantly more challenging, necessitating specialized techniques and architectural modifications.<\/span>2<\/span><\/p>\n A critical observation in this field is that the impact of precision reduction on model accuracy is not linear. When moving from 8-bit to sub-4-bit quantization, there is a qualitative shift in the challenges encountered. Research consistently indicates severe performance degradation and drastic precision loss at lower bit-widths.<\/span>2<\/span> A notable learning transition occurs between 2 and 3 bits, where the internal representations of models change drastically for 2-bit and below.<\/span>8<\/span> This suggests the existence of an “accuracy cliff” below approximately 3-4 bits, where traditional quantization methods become largely ineffective. To recover performance at these extreme levels, a fundamental rethinking of model architectures, training algorithms, and optimization strategies is required, moving beyond simple extrapolation from higher-bit techniques. This necessity drives the development of novel frameworks like OneBit and ParetoQ, specifically designed to navigate this complex landscape.<\/span><\/p>\n Table 1: Overview of Common Quantization Bit-widths<\/b><\/p>\n This table provides a systematic comparison of different quantization precisions, serving as a foundational reference for understanding the trade-offs inherent in this field. By explicitly detailing the bit-width, representation type, typical value range, relative memory footprint, primary computational operations, and typical accuracy impact, the table clarifies the landscape of quantization. It visually and quantitatively demonstrates the critical balance between efficiency and accuracy, which is the central challenge in developing quantized models. Furthermore, by illustrating the evolution of computational primitives from floating-point multiplications to simpler bitwise operations, the table reinforces the fundamental mechanisms driving efficiency gains across different quantization levels.<\/span><\/p>\n <\/p>\n The push for extreme low-bit quantization is driven by the escalating demands of modern deep learning and the imperative to deploy AI in diverse, resource-constrained environments. The sheer scale of contemporary deep learning models, particularly LLMs, necessitates innovative solutions to overcome their inherent computational and memory burdens.<\/span><\/p>\n Current LLMs, with their billions of parameters, consume immense memory and incur prohibitive computational costs, typically confining their deployment to high-performance GPUs or cloud-based servers.<\/span>3<\/span> For instance, a moderately sized LLaMA-13B model in FP16 format still requires 26GB of memory, making its use impractical on anything less than a high-end NVIDIA A100 GPU.<\/span>6<\/span> Extreme low-bit quantization directly addresses this by compressing models to a mere fraction of their original size, making them manageable for less powerful hardware.<\/span><\/p>\n A significant impetus for this research is the ambition to deploy sophisticated AI capabilities directly onto edge devices, such as smartphones, IoT gadgets, wearables, and embedded systems. These devices are inherently limited in RAM, storage, and processing power, rendering full-precision models impractical.<\/span>1<\/span> Extreme low-bit quantization makes it feasible to run powerful AI locally, unlocking new applications like gesture recognition, object detection, and voice synthesis on consumer-grade hardware.<\/span>16<\/span> This capability is not merely an optimization; it represents a fundamental enabler for the broader societal impact of AI. The massive size and computational demands of LLMs currently exceed the capabilities of most mainstream hardware, thereby limiting widespread adoption and restricting deployment. This situation highlights a critical hardware bottleneck that extreme low-bit quantization directly alleviates. By making powerful AI models compatible with ubiquitous, affordable hardware, it facilitates the democratization of AI, allowing it to permeate new domains and applications previously inaccessible. This decentralization of AI processing offers enhanced privacy, reduced latency, and offline capabilities, fostering a new wave of innovation and potentially transforming numerous industries and daily life.<\/span><\/p>\n Furthermore, many modern applications, from autonomous systems to interactive assistants, require AI models to respond with extremely low latency. The computational burden of full-precision models can introduce significant delays. By drastically reducing the computational load, extreme low-bit quantization accelerates inference speed, enabling the real-time performance crucial for such applications.<\/span>1<\/span> Concurrently, quantized models, due to their reduced computational requirements, consume significantly less power. This translates to extended battery life for portable devices and, in large-scale cloud data centers, leads to reduced operational costs and a lower carbon footprint, aligning with growing sustainability goals.<\/span>1<\/span> The high resource demands of large LLMs also contribute to elevated operational costs. By enabling models to run on less expensive, lower-power hardware or fewer high-end GPUs, extreme low-bit quantization helps to lower the overall cost of deploying and maintaining AI systems.<\/span>6<\/span><\/p>\n <\/p>\n <\/p>\n The core of extreme low-bit quantization’s efficiency gains lies in its ability to fundamentally alter the nature of arithmetic operations within a neural network. Instead of relying on computationally expensive floating-point multiplications, this technique transforms these operations into much faster and cheaper bitwise or integer arithmetic.<\/span><\/p>\n When network weights and activations are constrained to binary (1-bit) values (e.g., -1 or 1) or powers of two (e.g., 0, \u00b12^-2^, \u00b12^-1^, \u00b12^0^), the complex floating-point multiplication operations, which dominate convolutional and fully-connected layers, can be replaced by simpler additions, subtractions, or bit shifts.<\/span>2<\/span> For example, multiplication by 2^-N^ is equivalent to a right bit shift by N positions. In the most aggressive forms of quantization, particularly with binary weights and activations, the traditional multiply-accumulate (MAC) operations can be entirely replaced by logical XNOR and POPCOUNT (counting the number of ones in a bitstring) commands.<\/span>20<\/span> This transformation is significant because bitwise operations are inherently faster and require less hardware area and power than floating-point units. Binary neural networks, for instance, have demonstrated up to 58 times speedup in convolutional operations and 32 times memory savings.<\/span>19<\/span><\/p>\n This shift in computational primitives profoundly impacts hardware efficiency. Specialized deep learning hardware, such as ASICs and FPGAs, can be designed or optimized to perform these simpler operations much more efficiently, leading to dramatic speedups and energy savings.<\/span>11<\/span> The fundamental advantage of extreme low-bit quantization is not merely a reduction in the<\/span><\/p>\n number<\/span><\/i> of bits, but a qualitative change in the <\/span>type<\/span><\/i> of mathematical operations performed. Moving from complex floating-point multiplications to simpler integer additions, bit shifts, or logical XNOR\/POPCOUNT operations fundamentally changes the computational primitives. This is a deeper technical advantage than simply achieving faster inference or less memory. This change in primitives directly translates to less complex hardware logic. Beyond the computational savings, representing parameters with fewer bits also reduces the amount of data that needs to be moved between memory and processing units. Less memory transfer directly contributes to lower power consumption and faster processing, as memory access often represents a significant bottleneck in modern computing systems.<\/span>1<\/span> This highlights a critical causal relationship: the algorithmic innovation of extreme low-bit quantization directly drives the need for and benefits from specialized hardware design. By simplifying the core arithmetic, it opens the door for the development of highly efficient, custom-built AI accelerators (e.g., ASICs, FPGAs, LUT Tensor Cores) that are purpose-built for these low-precision operations, rather than general-purpose floating-point computations.<\/span>11<\/span> This synergy between software algorithms and hardware architecture is essential for fully realizing the transformative potential of extreme low-bit AI.<\/span><\/p>\n <\/p>\n <\/p>\n The pursuit of extreme low-bit quantization has necessitated the development of sophisticated techniques and frameworks that address the inherent challenges of precision reduction while striving to maintain model performance. These methodologies can be broadly categorized by their approach to binarization\/ternarization, their training paradigms, and their specialized adaptations for complex models like LLMs.<\/span><\/p>\n <\/p>\n <\/p>\n Binarization and ternarization represent the most aggressive forms of quantization, aiming to represent model parameters with the absolute minimum number of bits. Their effectiveness hinges on clever approximations and specialized training procedures.<\/span><\/p>\n <\/p>\n <\/p>\n Binarization is a technique that constrains network weights (and sometimes activations) to only two discrete values, typically {-1, 1} or {0, 1}.<\/span>2<\/span> This extreme reduction maximizes memory savings and enables the replacement of multiplications with simpler additions or bitwise operations.<\/span><\/p>\n Pioneering approaches in this area include <\/span>BinaryConnect (BC)<\/b>, which introduced the foundational concept of training Deep Neural Networks (DNNs) with binary weights during both forward and backward propagations.<\/span>22<\/span> In BinaryConnect, weights are binarized using a sign function (e.g.,<\/span><\/p>\n wb = +1 if w \u2265 0, else -1). Stochastic binarization is also explored, where wb = +1 with a probability p = \u03c3(w) (using a hard sigmoid function \u03c3), which can act as a regularizer.<\/span>23<\/span> A critical innovation of BinaryConnect is its handling of gradients. Since the sign function is non-differentiable, it employs the<\/span><\/p>\n Straight-Through Estimator (STE)<\/b> to approximate its gradient during backpropagation, allowing training to proceed.<\/span>22<\/span> Importantly, while binarized weights are used in the forward and backward passes, the gradients are accumulated in full-precision weights, which are then updated and optionally clipped (e.g., to<\/span><\/p>\n [-1, 1]) to maintain the necessary precision for the optimizer.<\/span>23<\/span><\/p>\n Another pivotal development is <\/span>XNOR-Net<\/b>, which extended binarization by quantizing <\/span>both<\/span><\/i> the weights and the activations to binary values.<\/span>19<\/span> XNOR-Net approximates convolutions using primarily binary operations like XNOR and POPCOUNT.<\/span>20<\/span> Unlike earlier binary network methods, XNOR-Net also preserves magnitude information via a channel-wise scaling factor (\u03b1).<\/span>25<\/span> Similar to BinaryConnect, XNOR-Nets face challenges with the non-continuous sign function, necessitating the use of the Straight-Through Estimator for gradient approximation.<\/span>25<\/span> Gradients are accumulated in full-precision weights, which are then binarized before each forward pass.<\/span>25<\/span> To mitigate information loss, the first and last layers of XNOR-Nets often retain full-precision weights.<\/span>25<\/span><\/p>\n <\/p>\n <\/p>\n Ternary Neural Networks (TNNs) constrain weights and activations to three discrete values, typically {-1, 0, 1}.<\/span>2<\/span> The inclusion of the zero state (0) differentiates TNNs from BNNs, offering an additional level of flexibility and efficiency.<\/span>21<\/span> When either a weight or an activation is zero, or both, the corresponding computational unit remains inactive, leading to simplified computations where multiply-accumulate operations can be replaced by control gates and binary logical operations like XNOR.<\/span>21<\/span> This event-driven computational paradigm makes TNNs computationally efficient, often on par with BNNs.<\/span>21<\/span><\/p>\n Similar to binarization, the non-differentiable nature of the quantization function in TNNs necessitates gradient approximation. The Straight-Through Estimator (STE) is widely adopted for approximating partial gradient calculations.<\/span>21<\/span> More advanced frameworks, such as Ternary Residual Quantization (TRQ), refine this process by recursively performing quantization on full-precision weights for a more accurate reconstruction, combining binarized “stem” and “residual” parts.<\/span>21<\/span> TRQ also introduces a learnable coefficient (\u03b1) that determines the scale of the binarized stem and residual, allowing the quantizer to automatically fine-tune the optimal mapping for each layer during backward propagation.<\/span>21<\/span> This learnable scaling, combined with a finer gradient calculation for \u03b1, significantly improves performance compared to fixed \u03b1 values. TRQ’s methodology can also be generalized to higher N-bit quantization by recursively encoding the residual.<\/span>21<\/span><\/p>\n <\/p>\n <\/p>\n The decision of when to apply quantization\u2014during or after model training\u2014defines two primary paradigms: Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ). Each approach presents distinct advantages and limitations, particularly in the context of extreme low-bit quantization.<\/span><\/p>\n <\/p>\n <\/p>\n Post-Training Quantization (PTQ) involves applying quantization to an already trained full-precision model, converting its floating-point representation to a lower-precision fixed-point integer format without requiring additional retraining.<\/span>1<\/span> This method is generally faster and requires less training data compared to QAT, making it suitable for scenarios where a working model already exists and the primary goal is to increase speed and efficiency.<\/span>1<\/span><\/p>\n Despite its simplicity, PTQ faces significant challenges, particularly at extremely low bit-widths (sub-4-bit). Naively applying PTQ to FP32 models at these low precisions typically results in severe accuracy degradation.<\/span>2<\/span> The drastic precision loss in the weight matrix significantly increases the error in linear projections, which are fundamental to LLMs.<\/span>6<\/span> For 1-bit quantization, the standard Round-To-Nearest (RTN) operation can undermine the practical significance of quantization scale and zero-point parameters.<\/span>6<\/span><\/p>\n To mitigate these issues, advanced PTQ methods have emerged. Techniques like PTQ1.61, for instance, push the limits of extremely low-bit PTQ for LLMs by introducing a one-dimensional structured mask based on input activation magnitudes. This mask selectively preserves critical (salient) weights by allocating a higher bit-width (e.g., 4 bits) to them, while quantizing non-salient weights to 1-bit, drastically reducing memory overhead compared to unstructured masks.<\/span>27<\/span> PTQ1.61 also employs a block-wise scaling factors optimization framework that considers implicit row-wise correlations and angular biases, using a joint metric of Mean Squared Error (MSE) loss and cosine similarity to minimize binarization error.<\/span>28<\/span> Furthermore, a novel “quantization preprocessing” paradigm is introduced, which involves a lightweight restorative Low-Rank Adaptation (LoRA) to transform the weight distribution of the pre-trained model into a more row-wise format, making it more amenable to per-channel quantization.<\/span>27<\/span><\/p>\n Another notable PTQ framework, SqueezeLLM, achieves lossless compression up to 3-bit by using sensitivity-based non-uniform quantization to assign optimal bit precision based on second-order information, and a Dense-and-Sparse decomposition to efficiently store outliers and sensitive weight values in a sparse format.<\/span>30<\/span> For image super-resolution tasks, 2DQuant employs a dual-stage PTQ method that addresses distinctive activation distributions in transformer-based models through a coarse-to-fine optimization process, including Distribution-Oriented Bound Initialization (DOBI) and Distillation Quantization Calibration (DQC). This approach has shown significant improvements in PSNR, compression ratio, and speedup for 2-bit quantization.<\/span>31<\/span><\/p>\n <\/p>\n <\/p>\n Quantization-Aware Training (QAT) integrates the quantization process directly into the model’s training or fine-tuning phase.<\/span>1<\/span> This allows the model to “learn” to adapt to the reduced precision from the outset, typically leading to enhanced performance compared to PTQ.<\/span>1<\/span> QAT is generally preferred when higher accuracy is paramount and sufficient training data and computational resources are available.<\/span>1<\/span><\/p>\n However, QAT demands substantial computational power and representative training data.<\/span>1<\/span> For LLMs with billions of parameters, the training cost can be prohibitive, making QAT impractical in many scenarios.<\/span>10<\/span> Despite these challenges, QAT is crucial for achieving reasonable accuracy at extremely low bit-widths (INT4 and lower), where the training loop must be modified to explicitly account for quantization.<\/span>2<\/span> Bootstrapping the quantized model with trained full-precision weights or using a trained FP32 model as a starting point or “teacher network” in a knowledge distillation setup can lead to higher accuracy.<\/span>2<\/span><\/p>\n The ParetoQ framework exemplifies state-of-the-art QAT. It unifies binary, ternary, and 2-to-4 bit quantization-aware training, demonstrating robustness and yielding state-of-the-art models across all bit widths.<\/span>4<\/span> ParetoQ emphasizes fine-tuning pre-trained full-precision models as a more effective approach than training from scratch, systematically allocating training budget (e.g., 90% for full-precision pre-training and 10% for QAT fine-tuning) to achieve optimal performance.<\/span>8<\/span> It also identifies that lower-bit quantization (binary, ternary, 2-bit) requires more fine-tuning tokens compared to higher-bit quantization (3-bit, 4-bit), attributing this to different QAT behaviors: “compensation” for higher bits and “reconstruction” for lower bits.<\/span>8<\/span> ParetoQ refines quantization functions by demonstrating that learnable scales consistently outperform statistics-based methods, and introduces Stretched Elastic Quant (SEQ) for ternary and 2-bit quantization to balance output quantized levels and evenly divide the full-precision weight span.<\/span>4<\/span><\/p>\n <\/p>\n <\/p>\n The unique characteristics and immense scale of Large Language Models (LLMs) necessitate specialized techniques to effectively apply extreme low-bit quantization while preserving their complex capabilities. LLMs present distinct challenges, such as the presence of outliers in weight distributions and the sensitivity of their performance to precision loss.<\/span><\/p>\n OneBit is a framework specifically designed for 1-bit weight quantization of LLMs, aiming to achieve extremely low bit-width deployment.<\/span>6<\/span> It addresses the severe performance degradation typically associated with 1-bit quantization by introducing a novel linear layer architecture. This architecture incorporates two FP16 (16-bit floating-point) value vectors (<\/span><\/p>\n g and h) alongside a 1-bit sign matrix (W\u00b11).<\/span>6<\/span> The sign matrix maintains the high rank and information capacity of the original weight matrix, while the value vectors provide the necessary floating-point precision at minimal cost, effectively compromising the inherent precision loss.<\/span>6<\/span> The computational order within this layer is optimized for time and space efficiency. Furthermore, these value vectors contribute to both forward and backward stability by restoring the range of output activations and limiting fluctuation during training, preventing overflow and gradient explosion.<\/span>6<\/span><\/p>\n OneBit also employs a parameter initialization method called Sign-Value-Independent Decomposition (SVID). SVID mathematically decomposes the original high-bit weight matrix into a sign matrix and a value matrix, which is then approximated by the outer product of two vectors (rank-1 approximation).<\/span>6<\/span> This decomposition bridges the architectural gap between the quantized and original models, providing an effective starting point for training and improving convergence speed. Knowledge transfer through quantization-aware knowledge distillation further helps the 1-bit model retain the performance of the original LLM.<\/span>6<\/span><\/p>\n Another approach, Vector Post-Training Quantization (VPTQ), focuses on extremely low-bit quantization of LLMs by formulating the problem using Second-Order Optimization. VPTQ uses vector quantization (VQ), a data compression technique that maps high-dimensional vectors to lower-dimensional ones stored in codebooks, leveraging correlations across data dimensions for more effective compression than scalar quantization.<\/span>13<\/span> VPTQ aims to overcome the limitations of current VQ by offering a lightweight and efficient approach for extreme low-bit weight quantization, achieving state-of-the-art accuracy and improved inference throughput (1.6-1.8x faster than existing methods).<\/span>13<\/span><\/p>\n The challenge of hardware compatibility for extreme low-bit quantization is also a significant focus for LLMs. Most hardware supports symmetric computations, which creates difficulties for mixed-precision calculations during General Matrix Multiplication (GEMM), a critical operation for LLMs.<\/span>11<\/span> The rapid development of LLMs often outpaces hardware upgrades, leaving many new data formats unsupported and complicating deployment.<\/span>11<\/span> Solutions are being developed to bridge this gap, such as the Ladder data type compiler, which converts unsupported low-precision data types into hardware-compatible ones without data loss.<\/span>11<\/span> The T-MAC mpGEMM library replaces traditional multiplication operations with bit-wise table lookups, eliminating dequantization overhead and enhancing CPU computational efficiency.<\/span>11<\/span> Furthermore, the LUT Tensor Core hardware architecture represents a software-hardware co-design for low-bit LLM inference, achieving significant performance gains, computational density, and energy efficiency.<\/span>11<\/span> These advancements are crucial for enabling LLMs to run efficiently on edge devices, expanding their applicability across a wider range of scenarios.<\/span><\/p>\n <\/p>\n <\/p>\n Despite the significant advancements, extreme low-bit quantization faces several formidable challenges that hinder its widespread adoption and full potential realization. These limitations span accuracy, training stability, and hardware compatibility.<\/span><\/p>\n <\/p>\n <\/p>\n The most immediate and pervasive challenge is the severe performance degradation that occurs when bit-width is drastically reduced, particularly below 4 bits.<\/span>2<\/span> This is primarily due to the drastic precision loss inherent in representing parameters with very few values. The limited dynamic range and resolution of extremely low bit-widths (e.g., INT4 and lower) are particularly problematic for activation functions like ReLU, which are unbounded, leading to significant information loss.<\/span>2<\/span> The long-tail effect in weight and activation distributions, especially in transformer-based models, means that the vast majority of floating-point numbers are compressed into only one or two candidate values, resulting in poor parameter homogenization and substantial performance degradation.<\/span>31<\/span> For asymmetric distributions, traditional symmetric quantization methods are ineffective, wasting a significant portion of candidate values.<\/span>31<\/span><\/p>\n1.1 Defining Low-bit Quantization: From Full Precision to Sub-4-bit<\/b><\/h3>\n
\n\n
\n Bit-width<\/span><\/td>\n Representation Type<\/span><\/td>\n Typical Value Range\/States<\/span><\/td>\n Memory Footprint (Relative to FP32)<\/span><\/td>\n Primary Computational Operations<\/span><\/td>\n Typical Accuracy Impact (Relative)<\/span><\/td>\n Common Use Cases\/Benefits<\/span><\/td>\n<\/tr>\n \n FP32<\/b><\/td>\n Floating-point<\/span><\/td>\n ~4 billion values<\/span><\/td>\n 1x<\/span><\/td>\n Floating-point multiplication<\/span><\/td>\n Baseline<\/span><\/td>\n General training\/inference, High fidelity<\/span><\/td>\n<\/tr>\n \n FP16<\/b><\/td>\n Floating-point<\/span><\/td>\n ~65,500 values<\/span><\/td>\n 0.5x<\/span><\/td>\n Floating-point multiplication<\/span><\/td>\n Minimal\/Near-lossless<\/span><\/td>\n Faster training\/inference, Reduced memory<\/span><\/td>\n<\/tr>\n \n INT8<\/b><\/td>\n Integer<\/span><\/td>\n [-128, 127]<\/span><\/td>\n 0.25x<\/span><\/td>\n Integer multiplication, Addition<\/span><\/td>\n Minimal\/Near-lossless, Acceptable degradation<\/span><\/td>\n General inference, Edge\/Mobile AI, CPU\/GPU optimization<\/span><\/td>\n<\/tr>\n \n INT4<\/b><\/td>\n Integer<\/span><\/td>\n [-8, 7]<\/span><\/td>\n 0.125x<\/span><\/td>\n Integer multiplication, Addition<\/span><\/td>\n Acceptable degradation (with effort)<\/span><\/td>\n Resource-constrained devices, Faster inference<\/span><\/td>\n<\/tr>\n \n 2-bit<\/b><\/td>\n Integer<\/span><\/td>\n e.g., {-2, -1, 1, 2}<\/span><\/td>\n ~0.0625x<\/span><\/td>\n Addition\/Subtraction, Bitwise operations<\/span><\/td>\n Severe degradation (requires advanced methods)<\/span><\/td>\n Extreme resource constraints, Specialized hardware<\/span><\/td>\n<\/tr>\n \n 1.58-bit<\/b><\/td>\n Ternary<\/span><\/td>\n {-1, 0, 1}<\/span><\/td>\n ~0.049x<\/span><\/td>\n Addition\/Subtraction, Bitwise (XNOR)<\/span><\/td>\n Severe degradation (requires advanced methods)<\/span><\/td>\n Extreme resource constraints, Sparse computation<\/span><\/td>\n<\/tr>\n \n 1-bit<\/b><\/td>\n Binary<\/span><\/td>\n {-1, 1} or {0, 1}<\/span><\/td>\n ~0.031x<\/span><\/td>\n Addition\/Subtraction, Bitwise (XNOR, POPCOUNT)<\/span><\/td>\n Severe degradation (requires advanced methods)<\/span><\/td>\n Maximum compression, Specialized hardware<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n 1.2 The Driving Force: Why Extreme Quantization is Imperative<\/b><\/h3>\n
1.3 Fundamental Principles: Replacing Floating-Point Operations with Bitwise Arithmetic<\/b><\/h3>\n
2. Core Techniques and Methodologies<\/b><\/h2>\n
2.1 Binarization (1-bit) and Ternarization (1.58-bit, 2-bit)<\/b><\/h3>\n
2.1.1 Binarization (1-bit)<\/b><\/h4>\n
2.1.2 Ternarization (1.58-bit, 2-bit)<\/b><\/h4>\n
2.2 Quantization-Aware Training (QAT) vs. Post-Training Quantization (PTQ)<\/b><\/h3>\n
2.2.1 Post-Training Quantization (PTQ)<\/b><\/h4>\n
2.2.2 Quantization-Aware Training (QAT)<\/b><\/h4>\n
2.3 Specialized Techniques for Large Language Models (LLMs)<\/b><\/h3>\n
2.4 Challenges and Limitations<\/b><\/h3>\n
2.4.1 Accuracy Degradation<\/b><\/h4>\n