Help
Options
Didn't know it?
click below
click below
Knew it?
click below
click below
Don't Know
Remaining cards (0)
Know
retry
shuffle
restart
0:00
Embed Code - If you would like this activity on your web page, copy the script below and paste it into your web page.
Normal Size Small Size show me how
Normal Size Small Size show me how
Enterprise Mobility
Strengthen your understanding of enterprise mobility with quick, easy-to-review
| Question | Answer |
|---|---|
| What is the primary definition of quantization in Large Language Models (LLMs)? | A technique that reduces the numerical precision of model parameters to decrease memory usage and improve inference speed. |
| In the 1930s, _____ used quantization to represent analog signals in digital form by sampling their amplitudes. | Pulse-code modulation (PCM) |
| Which historical field forms the core of nearly all lossy compression algorithms and influenced AI quantization? | Digital signal processing (DSP) |
| What development in the 2010s made researchers apply low-precision weights to neural networks? | The rapid growth in size and complexity of deep learning models. |
| Why is quantization considered an "urgent necessity" for models like Llama? | It is the only way to realistically deploy models with billions of parameters on accessible hardware. |
| What is the primary objective of Post-Training Quantization (PTQ)? | To convert high-precision weights to lower precision after a model is fully trained without needing retraining. |
| What is the typical accuracy dip range associated with Post-Training Quantization? | $2\%$ to $5\%$. |
| How does Quantization-Aware Training (QAT) differ from PTQ? | QAT integrates the quantization process directly into the training loop so the model adapts to lower precision. |
| Which technique generally protects model accuracy better: PTQ or QAT? | Quantization-Aware Training (QAT). |
| Which precision format is considered the trusted workhorse for production systems, offering a $4x$ memory reduction compared to FP32? | INT8 (8-bit Integer). |
| What is the typical accuracy loss when using INT8 quantization? | Less than $1\%$ |
| Term: FP16 | Definition: A 16-bit floating point format that provides a $2x$ reduction in memory and native speed boosts on modern GPUs. |
| By how much does INT4 quantization reduce memory compared to the original FP32 format? | $8x$. |
| Which method uses layer-wise quantization with approximate second-order information to minimize output error? | GPTQ (General Pre-Trained Transformer Quantization). |
| What is the core assumption of Activation-aware Weight Quantization (AWQ)? | Not all weights are equally important, requiring the identification of salient weight channels using activation statistics. |
| In AWQ, what percentage of weight channels are typically identified as "salient" and scaled up? | Approximately $1\%$ |
| How does FP4 differ from integer quantization formats? | It uses a floating-point representation to handle bell-shaped and long-tail distributions more flexibly. |
| Term: NF4 (4-bit NormalFloat) | Definition: An information-theoretically optimal format that normalizes weights assuming they follow a Gaussian distribution. |
| Which hardware architecture integrates native FP4 tensor core support for potentially doubling performance over FP8? | NVIDIA Blackwell architecture. |
| What is the main benefit of using FP8 in modern data center GPUs like the H100? | It offers near-lossless accuracy with $2x$ performance and memory improvements compared to FP16. |
| Which emerging format serves as a middle ground between FP4 and FP8 and uses microscaling (MX) for better accuracy? | FP6 (6-bit Floating Point). |
| What compression ratio can 3-bit quantization achieve? | $10x$ to $11x$. |
| What is the primary drawback of 2-bit quantization? | Significant accuracy degradation. |
| Term: HQQ (Half-Quadratic Quantization) | Definition: A calibration-free method capable of quantizing models to 2-bit precision in minutes. |
| Why are 70B parameter models more resilient to aggressive quantization than 8B models? | Larger models possess more parameter redundancy, making them more "quantization-friendly." |
| What is the purpose of per-layer quantization? | To apply different precision levels to different layers based on their specific sensitivity to noise. |
| Which quantization format is optimized for CPU and Apple Silicon using variants like Q4_K_M? | GGUF. |
| The _____ format is optimized for low-latency inference with customizable bit allocations. | EXL2 |
| How does quantization enable a 70B model to fit on a single RTX PRO 6000 GPU? | By reducing the original FP16 size of $\sim 140$ GB down to $\sim 35$-$45$ GB via INT4 quantization. |
| Quantization allows a Llama 3.1 405B model to run on 8x H100 GPUs instead of _____. | $16x$. |
| What is a primary storage and transfer benefit of quantized models? | Significantly faster download speeds and reduced version control storage requirements. |
| Concept: Perplexity | Definition: A performance metric measuring how well a probability model predicts a sample by gauging prediction uncertainty. |
| In terms of predictive performance, what does a lower perplexity value signify? Stronger predictive performance and greater model confidence. | Stronger predictive performance and greater model confidence. |
| What does a perplexity of 20 imply about a model's next-word prediction? | The model is as uncertain as if it were choosing from 20 equally likely options. |
| How does a significant post-quantization increase in perplexity affect a model? | It indicates the model has lost considerable predictive power. |
| Which quantization technique generally yields a lower (better) perplexity: GPTQ or AWQ? | AWQ. |
| What does accuracy loss represent in the context of quantized parameters? | The decline in model performance resulting from reduced numerical precision. |
| How does fine-tuning with QLoRA help quantized models? | It allows the model to recalibrate and adapt to lower precision in a parameter-efficient manner. |
| Which model layers often require higher precision in mixed-precision training to maintain performance? | Critical attention layers and output layers. |
| How do regularization techniques like weight clipping improve quantization outcomes? | By encouraging a more robust model during training that is less sensitive to precision loss. |
| What is the primary infrastructure challenge of hosting private LLMs without quantization? | The prohibitive cost and requirement for stacks of high-memory GPUs and massive storage. |
| How does quantization make private LLM hosting viable for smaller teams? | It reduces the hardware requirements, allowing sophisticated models to run on less powerful or consumer-grade hardware. |
| What is the typical return on investment (ROI) timeframe for hardware when switching from cloud APIs to private hosting via quantization? | 9 to 12 months. |
| A startup can save between _____% to _____% by switching from per-token cloud pricing to GPU-leased instances for private LLMs. | $50\%$; $70\%$. |
| What is per-channel quantization? | A strategy where different scales are applied to individual channels within a layer to improve precision. |
| Which emerging trend involves adjusting precision per-token based on the complexity of the input? | Dynamic quantization. |
| How does quantization impact inference latency on consumer hardware? | It can increase latency slightly due to the computational overhead of dequantization. |
| What is the effect of Double Quantization (DQ)? | It provides an additional $0.4$ bits per parameter savings by quantizing the quantization constants themselves. |
| Why is data privacy a key driver for LLM quantization in enterprise settings? | It enables on-premises deployment, ensuring sensitive information never leaves the internal network. |
| Which format is specifically integrated with the bitsandbytes library for easy deployment and fine-tuning? | NF4 (NormalFloat 4-bit). |
Created by:
mobisoftinfotech
Popular DAT sets