# Post-training quantization vs QAT: which one preserves model accuracy better?

Chase Pierce · August 22, 2026

> The Direct Answer: PTQ Wins on Speed, QAT Wins on Accuracy When comparing post-training quantization (PTQ) against quantization-aware training (QAT)...

## The Direct Answer: PTQ Wins on Speed, QAT Wins on Accuracy

When comparing post-training quantization (PTQ) against quantization-aware training (QAT), the short answer is that QAT recovers more accuracy at low bit-widths, while PTQ is dramatically cheaper and faster. PTQ takes an already-trained model and converts its weights and activations to lower precision — typically INT8 or INT4 — using calibration data and no gradient updates. QAT, by contrast, inserts simulated quantization operations into the training loop itself, so the network learns weight distributions that tolerate quantization noise from the start. NVIDIA's technical documentation on Model Optimizer and quantization-aware training describes this as the standard path to low-precision accuracy recovery: when simple PTQ degrades a model beyond acceptable limits, QAT is the fallback that closes most of the gap.

**Also worth reading:** [What is quantization aware training and how do I actually do it? A practical QAT tutorial for 2026?](https://technician.dev/knowledge/what_is_quantization_aware_training_and_how_do_i_actually_do_it_a_practical_qat_tutorial_for_2026.php) · [Pruning vs quantization for edge deployment: which model compression technique should you actually use?](https://technician.dev/knowledge/pruning_vs_quantization_for_edge_deployment_which_model_compression_technique_should_you_actually_use.php) · [What is the real ROI of AI technician training in 2026, and how do you measure it?](https://technician.dev/knowledge/what_is_the_real_roi_of_ai_technician_training_in_2026_and_how_do_you_measure_it.php)

The practical rule of thumb that has held through 2025 and into 2026 is this: at 8-bit precision, well-implemented PTQ usually costs less than 1% relative accuracy on large models, making QAT unnecessary. At 4-bit precision and below, PTQ degradation becomes erratic — some models lose 2–3%, others lose far more depending on architecture and outlier behavior — and QAT typically recovers 1–3 percentage points of that loss at the cost of retraining compute. Google's release of Gemma 3 QAT checkpoints in 2025 made this concrete: their QAT versions of Gemma 3 were tuned specifically so that 4-bit models performed close to their bf16 originals, bringing usable quality to consumer GPUs with limited VRAM.

For teams running inference in production — including field-service AI platforms like technician.dev, where dispatch assistants, diagnostic classifiers, and vision models run on edge hardware — the decision is rarely ideological. It comes down to how much accuracy you can afford to lose, how much calibration or training data you have, and whether your deployment timeline tolerates a retraining cycle.

## How Post-Training Quantization Actually Works

PTQ operates entirely after training is complete. The process loads the trained floating-point checkpoint, runs a few hundred to a few thousand representative samples through the network (the calibration set), and records activation statistics — min/max ranges, percentiles, or mean-square-error-minimizing scales. Those statistics determine the scaling factors that map float32 or bf16 values onto the integer grid. Weight-only quantization can even skip calibration entirely by analyzing the weight tensors directly.

NVIDIA's Model Optimizer implements several PTQ flavors worth knowing by name. Max calibration uses absolute min/max values, which is simple but fragile when a single outlier inflates the range and crushes resolution for everything else. Percentile calibration clips extreme values, trading a small amount of saturation error for much better effective resolution. Entropy (KL-divergence) calibration minimizes the information loss between the original and quantized distributions. Beyond basic calibration, modern PTQ methods apply algorithmic corrections: SmoothQuant migrates scale from hard-to-quantize activations into easier-to-quantize weights; GPTQ and AWQ use small calibration sets to find quantization-aware rounding of weights that minimizes output reconstruction error; and AutoQuant-style pipelines search over these options automatically per layer.

Qualcomm's AIMET library documents a similar progression of PTQ techniques, from simple min/max calibration up to cross-layer equalization and bias correction, which compensate for systematic shifts introduced by quantizing adjacent layers differently. The takeaway is that "PTQ" is not one method but a family, and choosing the right variant often determines whether you lose 0.2% or 2%.

## How Quantization-Aware Training Recovers Lost Accuracy

QAT attacks the problem at its root. During fine-tuning, fake-quantization nodes simulate the rounding and clamping of INT8 or INT4 arithmetic in the forward pass, while the backward pass uses the straight-through estimator (STE) to approximate gradients through the non-differentiable rounding operation. Over thousands of steps, the optimizer adjusts weights so the model's internal representations land comfortably inside quantization bins rather than straddling boundaries where rounding error hurts most.

This is why QAT shines precisely where PTQ fails: aggressive compression. At INT8, both approaches converge to similar results, and QAT's extra cost buys little. At INT4, and especially at mixed 4-bit weight / 8-bit activation configurations, QAT's advantage becomes measurable and repeatable. Google's Gemma 3 QAT work demonstrated that with enough careful training, a 4-bit model can approach bf16-level benchmark scores — something naive PTQ on the same checkpoints could not match. NVIDIA frames QAT explicitly as the "accuracy recovery" stage in its optimization workflow: run PTQ first, measure the drop, and if the drop exceeds your tolerance, escalate to QAT rather than abandoning low precision altogether.

The costs are real, though. QAT requires training infrastructure, a labeled or self-supervised dataset representative of production traffic, hyperparameter care around learning rate (typically 10–100x smaller than original training), and patience — fine-tuning runs of days to weeks are common for billion-parameter models. It also introduces failure modes of its own: unstable STE gradients, oscillating quantization ranges early in training, and sensitivity to how many epochs you train before freezing the quantization parameters.

## Side-by-Side Comparison

| Feature | Post-Training Quantization (PTQ) | Quantization-Aware Training (QAT) |
| --- | --- | --- |
| Requires retraining | No | Yes (fine-tuning with fake-quant nodes) |
| Typical time investment | Minutes to hours | Days to weeks |
| Data needed | 500–5,000 unlabeled calibration samples | Full representative training dataset |
| Compute cost | Single GPU, negligible | Multi-GPU training cluster |
| Accuracy at INT8 | Usually

Canonical: https://technician.dev/knowledge/post-training_quantization_vs_qat_which_one_preserves_model_accuracy_better.php
Markdown: https://technician.dev/knowledge/post-training_quantization_vs_qat_which_one_preserves_model_accuracy_better.php/index.md