⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠ You can decompress Drawing data with the command palette: ‘Decompress current Excalidraw file’. For more info check in plugin settings under ‘Saving’
Excalidraw Data
Text Elements
NF4 vs Standard Integer Quantization
Neural Network Weights Distribution
(Bell Curve / Gaussian)
-8
0
+7
Weight Values
Frequency
Most weights cluster near zero!
Standard Integer Quantization
-8
-6
-4
-2
0
+2
+4
+7
Equal-sized buckets (Uniform)
Wastes precision at extremes!
NF4 (NormalFloat 4-bit)
-8
0
+7
More buckets near zero (Bell-curve aligned)
Matches weight distribution!
Result:
NF4 captures ~99% of 16-bit precision
Standard 4-bit loses significant info
• Standard: Assumes uniform distribution
• NF4: Designed for Gaussian distribution
• More resolution where data lives = less loss